Multimodal fusion techniques integrate data from multiple imaging modalities to enhance diagnostic accuracy and clinical decision-making. In the context of SPECT/CT imaging for sports injury diagnosis, fusion strategies aim to combine the functional insights of SPECT with the anatomical precision of CT (Zhou et al., 2023). Early approaches focused on rigid registration methods, aligning images through affine transformations to ensure spatial coherence. However, these methods often struggled with soft tissue deformations, leading to inaccuracies in localization (Joseph et al., 2023). Recent breakthroughs in deep learning-based fusion techniques have greatly enhanced the integration of multimodal data. Convolutional neural networks (CNNs) and transformer-based architectures have demonstrated their capability to extract complementary features from SPECT and CT scans (Shi et al., 2022). Hybrid models that incorporate attention mechanisms further enhance the fusion process by dynamically weighting relevant features from each modality. These models not only improve the interpretability of fused images but also facilitate automated lesion detection and classification (Zhang et al., 2022). Generative adversarial networks (GANs) have also been explored for multimodal fusion, particularly in the synthesis of high-resolution hybrid images. By training the generator network to learn cross-domain mappings, GAN-based approaches enhance the contrast and spatial resolution of SPECT/CT images, enabling more precise localization of sports-related injuries (Bayoudh et al., 2021). Moreover, multi-scale feature extraction techniques have been employed to preserve fine-grained anatomical details while integrating functional information, reducing noise and artifacts in the fused images (Lian et al., 2022). Despite these advancements, challenges remain in achieving optimal fusion performance. Variability in patient anatomy, motion artifacts, and differences in imaging protocols pose significant obstacles (Malviya et al., 2024). Domain adaptation techniques and self-supervised learning frameworks are being investigated to improve generalization across diverse datasets. Future research directions include the incorporation of physiological priors to guide the fusion process and the development of real-time fusion systems for enhanced clinical workflows (Malviya et al., 2020).

Deep learning has revolutionized medical image analysis by providing automated solutions for segmentation, classification, and anomaly detection. In SPECT/CT imaging for sports injury diagnosis, deep learning models leverage large-scale datasets to learn discriminative patterns indicative of pathological conditions (Ma et al., 2021). CNN-based architectures, such as U-Net and DeepLab, have been extensively used for image segmentation, enabling precise delineation of injured regions. These models outperform traditional thresholding and region-growing techniques by capturing complex spatial dependencies in multimodal data (Du et al., 2022). Transformer-based models have recently gained traction for medical image analysis, particularly in capturing long-range dependencies across imaging modalities. Vision transformers (ViTs) leverage self-attention mechanisms to aggregate information across spatial dimensions, enhancing feature extraction from SPECT and CT images (Fan et al., 2022). Hybrid models combining CNNs and transformers achieve state-of-the-art performance in lesion detection and classification by integrating local and global contextual information. One of the major challenges in applying deep learning to SPECT/CT analysis is the limited availability of labeled medical datasets (Chango et al., 2022). To address this issue, researchers have explored self-supervised learning and contrastive learning techniques, which enable models to learn meaningful representations from unlabeled data. Few-shot learning approaches have also been investigated to improve model generalization in scenarios with limited training samples (Yan et al., 2022). Another critical aspect of deep learning for multimodal image analysis is explainability. Black-box nature of deep learning models raises concerns in clinical settings, necessitating the development of explainable AI (XAI) techniques (Malviya et al., 2016). Saliency maps, class activation mappings (CAMs), and attention visualization methods have been integrated into SPECT/CT analysis pipelines to enhance model transparency. Future research should focus on robust model validation, interpretability, and integration with clinical decision support systems to facilitate adoption in sports injury diagnosis (Athertya et al., 2024).

The integration of multimodal learning for SPECT/CT imaging in sports injury diagnosis presents significant clinical opportunities and challenges. SPECT imaging provides functional insights into metabolic activity, enabling early detection of stress fractures, ligament injuries, and inflammatory conditions that may not be apparent in conventional radiography or MRI (Ektefaie et al., 2022). CT imaging complements this by offering detailed anatomical structures, aiding in accurate localization and characterization of injuries. One of the primary clinical applications of multimodal learning in this domain is the early detection of overuse injuries in athletes (Wu et al., 2022). Stress fractures, common among endurance athletes, often exhibit subtle metabolic changes before structural abnormalities become visible. Multimodal deep learning models enhance diagnostic sensitivity by identifying these early-stage abnormalities, allowing for timely intervention and injury prevention (Yang et al., 2022). Another critical application is in post-injury rehabilitation monitoring. SPECT/CT imaging can assess bone healing progression and detect potential complications such as avascular necrosis or delayed union. AI-driven image analysis facilitates quantitative assessment of injury recovery, providing objective metrics for clinicians to tailor rehabilitation protocols (Chai and Wang, 2022). Personalized treatment plans based on AI-generated insights contribute to optimized recovery timelines and reduced risk of reinjury. Despite these advancements, several challenges hinder widespread clinical adoption. Radiation exposure remains a concern, particularly for young athletes undergoing repeated imaging (Yu et al., 2021). AI-driven dose optimization strategies are being developed to minimize radiation while preserving image quality. Standardization of imaging protocols across different institutions is another challenge, as variations in acquisition parameters can impact model performance. Federated learning approaches, which enable decentralized model training across multiple centers without data sharing, offer a potential solution for improving model robustness (Yu et al., 2023). Future research should focus on integrating multimodal learning into real-world clinical workflows. Seamless integration with picture archiving and communication systems (PACS) and electronic health records (EHRs) is essential for efficient deployment. Prospective clinical trials are needed to validate the clinical utility of AI-driven SPECT/CT analysis in sports injury diagnosis (Hu et al., 2022). Addressing these challenges will pave the way for more accurate, efficient, and personalized sports medicine applications.

Recent research has also highlighted the value of deep learning in joint-level medical imaging tasks. For instance, a study on automated detection of synovial fluid in the human knee using MRI and transfer learning demonstrated the feasibility of identifying subtle biomechanical abnormalities through deep models (Iqbal et al., 2020). Although focused on gastrointestinal analysis, another CNN-based system for endoscopic image classification similarly underscores the utility of convolutional architectures for spatial anomaly detection. These works support our design choice to leverage CNNs and attention mechanisms in injury localization and validate the broader applicability of such approaches across clinical imaging domains (Iqbal et al., 2022).

Sports injuries are a significant concern for athletes across various disciplines, affecting performance and long-term health. Traditional injury diagnosis relies heavily on expert evaluation, which can be subjective and time-consuming. Recent advancements in artificial intelligence and computer vision have enabled automated methods for detecting and diagnosing sports injuries using real-time data from various sources, including video footage, wearable sensors, and medical imaging.

This section presents a novel approach to sports injury detection that integrates deep learning models with biomechanical analysis to achieve high accuracy in identifying injuries. This study begin by defining the problem in a formalized mathematical framework in Section 3.2, where this study introduce key notations and the theoretical foundation underlying our approach. In Section 3.3, this study describe our proposed model, which leverages a hybrid architecture combining convolutional neural networks (CNNs) with transformer-based temporal analysis to capture both spatial and sequential injury patterns. This study introduce an innovative strategy in Section 3.4, where domain knowledge from sports science is incorporated into the learning process through biomechanical constraints and multi-modal fusion techniques. Unlike conventional approaches that focus solely on visual cues, our method integrates kinematic data to enhance detection accuracy.

Sports injury detection involves analyzing the movements and biomechanics of athletes to identify potential injuries. This process requires a formalized mathematical representation of human motion and injury characteristics. In this section, this study introduce the notation and fundamental concepts that form the basis of our proposed method.

Let X ∈ R T × J × D represent the motion data of an athlete, where T denotes the number of time steps, J the number of key joints being tracked, and D the spatial dimensionality. Each joint j at time step t is represented as x t , j ∈ R D . The full-body kinematic state at time t is given by X t = { x t , 1 , x t , 2 , … , x t , J } ∈ R J × D .

To analyze movement patterns, this study extract kinematic features such as joint velocity and acceleration (Equation 1): V t = X t − X t − 1 Δ t , A t = V t − V t − 1 Δ t , (1) where V t ∈ R J × D is the velocity matrix, A t ∈ R J × D is the acceleration matrix, and Δ t is the time interval between frames.

The likelihood of injury is modeled as a function I : R T × J × D → [ 0,1 ] , where I ( X ) provides the probability of an injury occurring. A common approach is to use a biomechanical threshold (Equation 2): I X = σ ∑ j = 1 J w j ‖ A t j ‖ , (2) where σ ( ⋅ ) is the sigmoid function and w j are learned weights indicating the importance of each joint.

Human motion can be represented as a graph G = ( V , E ) , where the vertices V symbolize the joints, and the edges E capture the anatomical or dynamic relationships between them. The adjacency matrix A ∈ R J × J encodes connectivity, and the node feature matrix F t = X t contains joint positions.

Injuries often result from abnormal energy distribution. The total mechanical energy of a joint is given by Equation 3: E t j = 1 2 m j ‖ V t j ‖ 2 + m j g h t j , (3) where m j is the mass of the joint, g is the gravitational acceleration, and h t j is the vertical height of the joint. Large deviations from expected values indicate potential injuries.

To enhance injury detection, this study integrate multiple data sources (Equation 4): X total = λ 1 X vision + λ 2 X IMU + λ 3 X pressure , (4) where X vision represents pose data extracted from video, X IMU is motion data from inertial measurement units, and X pressure is foot pressure data. The coefficients λ i control the contribution of each modality.

Given a sequence of motion data X , the goal of sports injury detection is to learn a function f : R T × J × D → { 0,1 } , where f ( X ) = 1 indicates an injury. The function is trained using a dataset D = { ( X ( i ) , y ( i ) ) } i = 1 N with labels y ( i ) ∈ { 0,1 } indicating injury occurrence. This formulation provides a structured foundation for developing a data-driven sports injury detection system, which will be elaborated in subsequent sections.

While imaging modalities like SPECT/CT provide static snapshots of anatomical and metabolic activity, biomechanical data captures the dynamics of movement—how joints accelerate, decelerate, and absorb force during activity. These patterns often reveal early signs of dysfunction before visible structural damage occurs. For example, a runner with an early-stage stress injury may exhibit unusually high force in the tibial region with subtle changes in gait symmetry, even if CT images appear normal. By incorporating motion capture and plantar pressure data, our model detects these abnormal patterns of movement and loading. This is akin to a mechanic detecting issues in a car not by looking at its body, but by sensing unusual vibrations or performance shifts during a test drive. Physiological signals—such as energy transfer between joints—act as functional biomarkers that complement anatomical imaging, allowing for earlier and more accurate injury identification.

To address accessibility for readers from clinical and sports medicine backgrounds, this study have added technical terms in Table 1, providing concise definitions and contextual explanations for complex concepts such as transformer-based models, feature refinement, and multimodal fusion. This addition is intended to improve readability and facilitate interdisciplinary understanding of our methodology.

To accurately detect sports injuries, this study propose the Biomechanics-Aware Injury Detection Network (BID-Net), a deep learning model specifically designed to integrate principles from sports science with multimodal medical imaging. Unlike conventional AI models that primarily rely on static image-based features, BID-Net combines spatial and temporal patterns from imaging with biomechanical data—such as joint velocities, accelerations, and energy transfer—to capture functional abnormalities in movement. This architecture is tailored to the demands of sports medicine, where injuries often emerge not from obvious structural defects but from dysfunctional motion dynamics. By embedding motion capture inputs and domain-informed constraints directly into its learning process, BID-Net interprets human motion as a diagnostic signal. This hybrid perspective enables more robust, interpretable, and early-stage injury recognition, especially in cases where anatomical abnormalities are subtle or absent in imaging.

The overall structure of BID-Net, including its CNN, transformer, biomechanical fusion, and prediction pipeline, is illustrated in Figure 1.

To enhance the accuracy, robustness, and interpretability of sports injury detection, this study propose the Adaptive Multi-Modal Fusion Strategy (AMFS). This strategy integrates heterogeneous data sources—such as video-based pose estimation, inertial measurement unit (IMU) signals, and plantar pressure readings—into a unified detection framework. By leveraging domain knowledge from biomechanics, deep learning feature extraction, and sensor fusion techniques, AMFS is designed to adaptively capture both spatial and temporal injury patterns. The framework enables real-time analysis while ensuring the physiological relevance of detected anomalies (As shown in Figure 3).

The PA-HMDB51 dataset Hinojosa et al. (2022) is an extended version of the HMDB51 action recognition dataset, specifically designed to support research in pose-based activity analysis. It includes annotated human pose information for each video frame, enabling more fine-grained analysis of human actions. This dataset is particularly useful for tasks involving human motion understanding, such as pose estimation and action classification. By incorporating both spatial and temporal pose dynamics, PA-HMDB51 offers a richer representation of complex human behaviors, making it a valuable benchmark for evaluating algorithms that leverage pose information in video-based activity recognition. The Kinetics-700 Dataset Wang et al. (2022) is a large-scale video dataset developed by DeepMind for human action recognition. It contains approximately 650,000 video clips, each lasting around 10 s and covering 700 distinct human action classes. The dataset is sourced from YouTube and provides a diverse and realistic collection of actions performed in varied settings and environments. Kinetics-700 supports training deep learning models that require large amounts of data for accurate temporal and semantic understanding. Its size, diversity, and fine-grained class labels make it one of the most comprehensive benchmarks for evaluating video classification and action recognition models. The MIMIC-III Dataset Khope and Elias (2022) is a publicly available medical database that contains de-identified health-related data from over 40,000 critical care patients. Collected from the Beth Israel Deaconess Medical Center, it includes information such as demographics, vital signs, lab results, medications, and clinical notes. Researchers use MIMIC-III for a wide range of healthcare studies including predictive modeling, patient outcome analysis, and clinical decision support. The richness and depth of the data make it a cornerstone in medical AI research, especially for developing algorithms that can assist with diagnosis and treatment planning in intensive care units. The HAM10000 Dataset Houssein et al. (2024) is a large collection of multi-source dermatoscopic images used for skin lesion analysis and diagnosis. It contains over 10,000 dermatoscopic images representing seven common types of skin lesions, including both benign and malignant conditions. The dataset was created to support the development and evaluation of machine learning algorithms in dermatology. With high-quality images and verified labels, HAM10000 facilitates research in automated skin disease detection and has become a standard benchmark for training and validating image classification models in medical imaging, particularly in the early detection of skin cancer.

Our experiments are conducted on multiple trajectory prediction benchmarks to evaluate the effectiveness of the proposed method. All models are implemented in PyTorch and trained on an NVIDIA A100 GPU with 80 GB of memory. The Adam optimizer is employed with an initial learning rate of 1e-3, which is halved if the validation loss remains unchanged for three consecutive epochs. The batch size is set to 128, and training is performed for 50 epochs. To ensure stability, gradient clipping is applied with a maximum norm of 10.0. The loss function used is a combination of the displacement error and a social interaction-aware penalty, which accounts for trajectory consistency and collision avoidance. For trajectory forecasting, the input consists of historical observations spanning 2–3 s, sampled at 2.5 Hz for pedestrian datasets and 10 Hz for autonomous driving datasets. The output is a multi-modal prediction of future trajectories over a prediction horizon of 3–5 s. This study employ a graph-based recurrent architecture that models agent interactions using self-attention and relative spatial embeddings. The motion prediction module is based on a conditional variational autoencoder (CVAE) framework, ensuring diversity in the generated future trajectories. To refine the predicted paths, this study integrate map-aware constraints by encoding lane connectivity and drivable areas using a transformer-based attention mechanism. Evaluation metrics include the Average Displacement Error (ADE) and Final Displacement Error (FDE), computed over multiple predicted trajectories. For pedestrian datasets, this study also report the Collisions per Trajectory (CpT) to assess model safety in social environments. On autonomous driving benchmarks, this study consider kinematic constraints by evaluating the Heading Error (HE) and Off-Road Rate (ORR), ensuring that predictions align with road structure. Ablation studies are conducted to assess the contributions of interaction modeling, map encoding, and trajectory diversity. During inference, this study generate the top K = 6 predictions for each agent, ranking them based on likelihood estimated by the learned distribution. A non-maximum suppression strategy is applied to remove redundant trajectory samples. The proposed method is compared against state-of-the-art approaches, including graph-based motion predictors, transformer-based models, and flow-based generative models. Each method is tested under identical data preprocessing conditions to ensure fair comparisons. For hyperparameter tuning, this study perform a grid search over key parameters such as the hidden size of recurrent layers, the number of attention heads, and the weighting of regularization terms in the loss function. The optimal configuration is selected based on its performance on the validation set. To prevent overfitting, data augmentation strategies like random rotations, trajectory jittering, and time shifts are applied. The final model is selected based on the best validation ADE score and evaluated on the test set using unseen scenarios. All experiments are conducted using a unified evaluation pipeline with standardized dataset splits and pre-processing steps. To facilitate reproducibility, this study release our code, trained models, and evaluation scripts.

This study assess our method by comparing it with state-of-the-art (SOTA) approaches across four benchmark datasets: PA-HMDB51, Kinetics-700, MIMIC-III, and HAM10000. The comparison is conducted using key evaluation metrics. The results are presented in Tables 2, 3. Our method consistently outperforms existing approaches across all datasets, demonstrating superior trajectory prediction capabilities.

The results on the PA-HMDB51 and Kinetics-700 datasets in Figure 6 show that our model attains the highest values in Accuracy, Recall, and F1 Score, outperforming previous methods such as CLIP, ViT, and I3D. The improvements are particularly significant in the Kinetics-700 dataset, where pedestrian interactions and social dynamics pose challenges to trajectory forecasting. Our approach effectively captures these interactions using a graph-based motion representation combined with a transformer-based attention mechanism. The higher AUC score further indicates the robustness of our model in handling diverse scenarios with varying agent behaviors. On the MIMIC-III Dataset and HAM10000 Dataset in Figure 7, our method again outperforms SOTA baselines. The improvements in Accuracy and Recall suggest that our model can better distinguish between different motion patterns and produce more precise trajectory predictions. The higher F1 Score confirms that our method balances precision and recall effectively, reducing false-positive and false-negative predictions. The enhanced AUC score indicates a more reliable overall prediction framework, capable of generalizing to complex driving environments. The enhanced performance can be attributed to the incorporation of spatial-temporal encoding and interaction modeling, allowing the model to effectively learn realistic agent behaviors across varying traffic conditions.

The success of our model across all datasets highlights the effectiveness of incorporating scene context, interaction modeling, and multi-modal trajectory generation. Unlike existing methods that primarily rely on past trajectories for prediction, our approach leverages additional cues such as road topology, lane connectivity, and social interactions to refine the predicted paths. The use of a transformer-based attention mechanism enables efficient information aggregation across multiple agents, leading to more accurate and diverse trajectory forecasts. The ablation studies further confirm the contribution of these components to the overall performance, demonstrating the robustness and generalization ability of our approach in various real-world scenarios.

This study conduct an ablation study to assess the impact of different components in our model, using four benchmark datasets: PA-HMDB51, Kinetics-700, MIMIC-III, and HAM10000. In Tables 4, 5, this study present the results, where this study evaluate different variations of our method by excluding key modules.

In Figures 8, 9, the first ablation variant, removes the Temporal Attention Modeling, which models agent interactions within the scene. The performance drop across all datasets indicates that capturing spatial and temporal dependencies is crucial for accurate trajectory prediction. Accuracy and Recall decline significantly, suggesting that the model struggles to correctly predict agent movements when interactions are not explicitly encoded. This result confirms that effective trajectory forecasting requires understanding the relationships between agents in dynamic environments. The second ablation, eliminates the Biomechanical Constraint Fusion, which integrates road topology, lane connectivity, and scene context into the trajectory prediction process. The absence of this module leads to a noticeable drop in AUC scores across all datasets, particularly in the autonomous driving datasets, where road structure plays a vital role in agent movement. The decline in F1 Score further shows that the model generates less precise trajectory distributions, increasing the likelihood of off-road or unrealistic predictions. This result highlights the importance of incorporating scene context for ensuring accurate and feasible trajectory generation. The third ablation, removes the Temporal Prediction Consistency, which enables the model to output diverse future motion hypotheses. The results show a decrease in both Accuracy and F1 Score, suggesting that the removal of this component impairs the model’s capacity to account for uncertainty in motion prediction. Without multi-modal outputs, the model tends to generate overconfident single-modal predictions that may not align with real-world behaviors. This limitation is especially evident in the Kinetics-700 dataset, where pedestrian motion involves high variability due to social interactions.

The complete model, incorporating all three components, delivers the highest performance on all datasets. In comparison to the ablation variants, our full method achieves the highest scores. These results validate the necessity of interaction modeling, map-aware encoding, and multi-modal generation for achieving robust and generalizable trajectory predictions. The improvements over the ablation baselines highlight how our approach effectively balances scene understanding, interaction awareness, and predictive diversity, leading to superior performance in complex motion forecasting scenarios.

The evaluation results on the CAMUS and KiTS datasets in Table 6 demonstrate the superior performance of our proposed BID-Net framework in clinically relevant CT imaging scenarios. On the CAMUS dataset, which involves left ventricle (LV) and left atrium (LA) segmentation, BID-Net achieved Dice scores of 0.902 and 0.873, respectively, significantly outperforming both the baseline U-Net and the attention U-Net models. The improvement in Hausdorff Distance (HD95), reduced to 3.12 mm for LV and 3.87 mm for LA, indicates enhanced boundary delineation and spatial coherence, which are essential for accurate anatomical interpretation. Similarly, the gains in sensitivity and specificity reflect BID-Net’s strong capability in capturing relevant structures while minimizing false positives—an important consideration for cardiac-related injury detection and diagnosis. On the KiTS dataset, which presents a more complex challenge due to the heterogeneity of kidney tumors, BID-Net maintained high accuracy across both kidney and tumor segmentation tasks. Dice scores reached 0.895 for the kidney and 0.861 for tumor regions, with corresponding HD95 values of 3.45 mm and 4.02 mm. The increased sensitivity (0.88) and specificity (0.94) further confirm the model’s ability to localize and characterize abnormalities effectively. These results suggest that the biomechanics-aware design and multi-modal fusion strategy employed in BID-Net contribute not only to improved segmentation precision but also to the clinical interpretability and reliability of injury localization in real-world imaging applications.

Imagine a collegiate soccer player who experiences sudden knee pain during a match. Traditional diagnostic workflows may require delayed imaging appointments, subjective physical assessments, and limited real-time insights. With BID-Net integrated into a sideline diagnostic station, real-time motion data from wearable sensors and portable pressure mats can be fused with low-dose SPECT/CT scans. Within minutes, the system highlights abnormal joint mechanics and localized stress zones, flagging early indicators of a medial meniscus injury. This enables sports clinicians to make evidence-based decisions on whether the athlete can safely return to play or requires further medical intervention. By combining biomechanical cues with enhanced imaging interpretation, BID-Net supports faster, more objective, and actionable diagnoses in high-performance environments.