Traditional image encoders, such as convolutional neural networks (CNNs) and Vision Transformer (ViT) (Dosovitskiy et al., 2020), have achieved remarkable performance across a wide range of visual tasks. While CNNs like ResNet are known for their efficiency and effectiveness, ViT typically needs higher computational resources and larger model sizes (He et al., 2016). To address these limitations, Yu et al. (2022) proposed PoolFormer, which eschews attention mechanisms in favor of spatial pooling and global token mixing, achieving competitive performance with reduced computational demands.

Recently, Liu et al. (2024) introduced the KAN, a novel architecture based on the Kolmogorov-Arnold representation theorem. KAN replaces the linear transformations in traditional multilayer perceptron (MLP) with learnable spline-based functions, offering a flexible and interpretable modeling framework with strong approximation capabilities, particularly beneficial in low-data scenarios and for capturing complex nonlinear relationships. Several studies have explored KAN in Alzheimer's disease. For example, Verma et al. (2025) combined KAN with Visual Geometry Group (VGG) for Alzheimer's disease prediction and achieved promising results. Others have integrated KAN with graph neural networks to further enhance modeling capabilities (Wang, 2025; Ding et al., 2025). In addition to its role as a classifier, KAN has recently been applied to tabular feature extraction tasks and demonstrated notable effectiveness (Poeta et al., 2024; Eslamian et al., 2025; Gao et al., 2024).

Building on these advancements, we propose a hybrid neural backbone that combines the architectural efficiency of PoolFormer with the functional expressiveness and interpretability of KAN. Specifically, our design utilizes PoolFormer's spatial feature aggregation capabilities and enhances representation learning with KAN-based modules, providing a robust and efficient framework for processing clinical tabular data.

In visual tasks, traditional attention mechanisms include Squeeze and Excitation Networks (SENet) (Hu et al., 2018) and Convolutional Block Attention Modules (CBAM) (Woo et al., 2018). In particular, CBAM applies channel-wise attention and spatial attention sequentially to enhance feature representations by highlighting useful features and diminishing less important ones. More recently, Liu et al. (2023) proposed the CGA mechanism, a memory-efficient attention scheme that divides input tokens into separate groups and performs cascaded cross-group attention, greatly reducing computational complexity while preserving expressive power .

For AD analysis, attention mechanisms have also been explored. For instance, Li et al. (2025) combined self-attention and bi-attention to effectively fuse features from MRI and PET modalities. Similarly, Kang et al. (2023) employed cross-attention to integrate MRI data with clinical features. However, most existing approaches do not adequately consider the unique structural characteristics of each modality, particularly the disparity between high-dimensional imaging data and low-dimensional tabular data. Moreover, the high computational cost of attention is also a problem.

To address these limitations, we introduce two innovative attention modules designed for multimodal AD analysis. The first is HBAM, which extends the CBAM architecture to facilitate bidirectional attention between image and tabular modalities. The second is the MMCA, which adapts the CGA mechanism to support efficient and effective cross-modal interaction. Together, these modules form the backbone of our proposed framework, enabling precise and computationally efficient fusion of heterogeneous medical data.

Interpretability is essential in medical image analysis. Among various explainability methods (Kohlbrenner et al., 2020; Selvaraju et al., 2017; Sundararajan et al., 2017), Integrated Gradients (IG) (Sundararajan et al., 2017) has been widely adopted due to its solid theoretical foundations and practical effectiveness, which stands out as a path-based attribution method that overcomes the limitations of standard gradient-based explanations, such as saturation and noise. In the application of neuroscience in dementia detection, previous studies have successfully employed the IG attribution method (Gryshchuk et al., 2025; Wang et al., 2023). Following this line of research, we adopt IG in our study to investigate feature contributions across multimodal inputs.

For a given model F, an input x, and a baseline input x′ (often a zero vector), IG calculates the attribution for each input dimension as the path integral of the gradients along a straight-line path from the baseline to the actual input:

where IG_i(x) represents the attribution of the i-th feature. Essentially, IG quantifies each input feature's contribution to the change in the model output from the baseline to the actual input. This approach is particularly suitable for high-dimensional medical data, providing pixel-level attributions for image modalities and feature-level attributions for clinical inputs.

This study utilizes data from the Alzheimer's disease Neuroimaging Initiative (ADNI), drawing specifically from the ADNI-1 and ADNI-2 datasets (Mueller et al., 2005). The ADNI cohort includes individuals diagnosed with AD, MCI, and Normal Controls (NC). To avoid duplication, subjects appearing in both datasets were excluded from ADNI-2. We used T1-weighted sMRI, FDG-PET imaging, and clinical data, categorizing subjects into pMCI and sMCI groups. Demographic information is detailed in Table 1.

Given the challenges of missing PET data and class imbalance between pMCI and sMCI in both datasets, we focused on subjects with complete multimodal data, comprising MRI, PET, and clinical features. Therefore, we merge the ADNI-1 and ADNI-2 cohorts into a single dataset. The finalized dataset comprises 512 subjects: 149 pMCI and 363 sMCI. The dataset was randomly split into training and testing sets with a 4:1 ratio. The training set included 119 pMCI and 290 sMCI. The testing set comprised 30 pMCI and 73 sMCI.

All MRI images underwent preprocessing, including intensity normalization, skull stripping, and normalization to Montreal Neurological Institute (MNI) space. FDG-PET images were similarly processed through intensity normalization, normalization to MNI space, and co-registration with MRI images. All images were resized to a resolution of 96 × 128 × 96. For clinical data, similar to previous studies (Hu et al., 2025; Duenias et al., 2025; Kang et al., 2023) and based on clinical experience from doctors, we selected seven features. Among them, age, gender, and education belong to demographic attributes, while ApoE4 status, phosphorylated tau 181 (P-tau 181) and total tau (T-tau) are categorized as cerebrospinal fluid (CSF) biomarkers. The last attribute is a composite measure derived from 18F-fluorodeoxyglucose (FDG) and florbetapir (AV45) PET scans. It is worth noting that cognitive scores were excluded because they are directly related to the diagnosis of Alzheimer's disease and thus were excluded to avoid introducing bias into the prediction of disease progression.

This section details the architecture of TriLightNet, our lightweight triple-modal fusion network for predicting MCI-to-AD progression. We first describe the feature extraction from MRI, PET, and clinical data. For imaging modalities, ResNet is employed to capture spatial features, and the tabular encoder combines KAN with PoolFormer to enable efficient and expressive representation learning from clinical variables. Subsequently, we introduce the HBAM for nuanced image-clinical feature interaction, followed by the MMCA for efficient progressive fusion of all modalities. Finally, we discuss the loss function used for model training.

All experiments were conducted using PyTorch version 2.6.0 alongside CUDA 11.8, running on a single NVIDIA V100 32GB GPU. The model underwent training for 200 epochs with a batch size of 8, allowing for efficient data management. For optimizing model parameters, the Adam optimizer was utilized, with the learning rate fixed at 0.0001 to facilitate precise adjustments during training. Given the relatively small size of the datasets, we adopted several strategies to reduce sampling bias and alleviate overfitting. First, we performed five-fold cross-validation to ensure robust evaluation of the model's performance. Secondly, we used a cosine scheduler with a hyperparameter T_max set to 50, allowing for dynamic adjustment of the learning rate throughout the training process and enhancing the model's adaptation capabilities. Additionally, we incorporated an early stopping strategy with a patience value of 50, which effectively prevented overfitting by halting training when the validation loss ceased to improve.

For evaluation, we employ eight metrics to assess both classification performance and model efficiency: Accuracy, Sensitivity, Precision, Area Under the Receiver Operating Characteristic Curve (AUROC), F1-score, Balanced Accuracy, number of parameters (Params), and floating point operations (FLOPs). Given the dataset's class imbalance, we emphasize F1-score and Balanced Accuracy for fairer evaluation. Balanced Accuracy mitigates bias from uneven class distributions and is better suited for imbalanced binary tasks, which is defined as:

where TP, TN, FP, and FN represent the numbers of true positives, true negatives, false positives, and false negatives, respectively.

We comprehensively evaluate our framework against representative multimodal approaches under three fusion scenarios. For MRI and PET bimodal fusion, we compare with ResNet+Concat (He et al., 2016), ViT+Concat (Dosovitskiy et al., 2020), nnMamba+Concat (Gong et al., 2025), Diamond (Li et al., 2025), and MDL (Qiu et al., 2024). For MRI and clinical fusion, methods include VAPL (Kang et al., 2023) and HyperFusionNet (Duenias et al., 2025). In the trimodal setting integrating MRI, PET, and clinical data, we compare with IMF (Li et al., 2023), HFBSurv (Li et al., 2022), ITCFN (Hu et al., 2025), and MultimodalADNet (Zhang et al., 2025b). Detailed comparative metrics are presented in Table 2.

TriLightNet achieves outstanding performance across several evaluation metrics, with an accuracy of 81.25%, AUROC of 0.8146, and F1-score of 69.39%, all surpassing those of the baseline models. Although ResNet+Concat attains the highest Sensitivity at 75.04%, its relatively low Accuracy and Precision result in suboptimal overall performance. In contrast, TriLightNet ranks second in Sensitivity while maintaining the highest Balanced Accuracy, demonstrating its strong capability to effectively integrate multimodal information and deliver more reliable predictive outcomes.

In terms of computational efficiency, TriLightNet has 17.405 million Params and 10.517 billion FLOPs. While models such as MDL and MultimodalADNet have fewer Params, their predictive performance is significantly inferior. Among the tri-modal approaches, TriLightNet consistently outperforms competing methods, including HFBSurv (34.123 million Params and 141.849 billion FLOPs) and IMF (67.843 million Params and 70.925 billion FLOPs). This efficiency highlights that TriLightNet not only delivers superior accuracy but also exhibits enhanced computational resource efficiency, making it particularly suitable for real-world applications that demand accurate and efficient prediction of cognitive impairment.

We conduct the following two ablation experiments: (1) Assessing the individual contributions of the HBAM and MMCA modules. (2) Evaluating the effect of KAN for the tabular encoder.

The results of ablation experiment (1) are shown in Table 3, demonstrating the significant contributions of both the HBAM and MMCA modules to improve the model's performance. Specifically, HBAM improves the model's accuracy by 0.5%, sensitivity by a remarkable 27.03%, AUROC by 0.1353, and F1-score by 13.91%. These improvements can be attributed to the design of channel attention within the HBAM framework, which facilitates the integration of clinical features into the image feature channels. This interaction at the channel level is followed by spatial attention enhancement, enabling the model to produce more concentrated and informative feature representations.

For MMCA module, it increases accuracy by 5%, AUROC by 0.1745 and balance accuracy by 10.88%. Although there is a slight decrease in precision, the stability of the model improves. This performance can be primarily attributed to two key mechanisms: progressive fusion, which gradually refines dominant features from shallow to deep layers; and bidirectional symmetric enhancement, where each set of cross-attention operations is bidirectional, allowing mutual reinforcement between MRI and PET features, thus achieving comprehensive and balanced multimodal fusion.

Finally, when both HBAM and MMCA modules are jointly applied, the model achieves the best performance across all five evaluation metrics: an accuracy of 81.25%, precision of 65.38%, AUROC of 0.8146, F1-score of 69.39%, and balanced accuracy of 79.06%, demonstrating the complementary benefits and synergistic effect of combining both modules. This indicates that the synergy between these two modules significantly enhances the model's classification capabilities, highlighting the importance of integrating both the HBAM and MMCA modules into the TriLightNet model.

Table 4 presents the ablation results of experiment (2). It illustrates that KAN demonstrates a remarkable advantage over the MLP in clinical tabular feature extraction, achieving consistent improvements across all evaluation metrics. In particular, KAN increases F1-score by 1.72% and balanced accuracy by 1.98%. This improvement can be attributed to KAN's use of expressive B-spline basis functions for modeling clinical features, rather than simple activation functions (e.g., Sigmoid, ReLU) commonly used in MLPs. This enhanced representational capacity allows for better capture of nonlinear patterns in clinical variables, thereby improving overall performance.