This study utilized sMRI, DTI, PET, and neuropsychological data from a total of 1759 participants. The data were obtained from the Alzheimer’s disease prediction of longitudinal evolution (TADPOLE) challenge¹ (Marinescu et al., 2018), which was sourced from the Alzheimer’s Disease Neuroimaging Initiative (ADNI) database (adni.loni.usc.edu). ² The current study included 648 healthy controls (294 males), 699 individuals with MCI (411 males), and 412 individuals with AD (234 males). Table 1 presents the demographic and clinical characteristics of the participants. Unstable HC and MCI subjects with any conversions or revisions were excluded (Hojjati et al., 2018). It is important to note that subjects did not necessarily require all three neuroimaging modalities, as each modality had been used in separate prediction tasks.

The current study aimed to predict cognitive scores using imaging biomarkers. Three cognitive scores were used as targets for prediction: MMSE, CDRSB, and ADAS. Each score highlights a different aspect of an individual’s neuropsychological status, depending on their stage of AD progression. The MMSE test is scored on a scale of 0–30, with a higher score indicating better cognitive function. The CDRSB and ADAS are scored on scales ranging from 0 to 18 and 0 to 70, respectively, with higher scores indicating more severe dementia and lower scores indicating milder dementia.

We used various biomarkers from neuroimaging data as predictors in our machine learning models. Nine different types of biomarkers were extracted from the three neuroimaging modalities: sMRI, DTI, and PET. The aim of selecting these biomarkers was to establish a robust machine learning framework based on a substantial sample size. All imaging data had undergone prior processing, and post-processed measurements for each biomarker were collected (Marinescu et al., 2018). The sMRI biomarkers were grouped into four categories: volume of gray matter (VGM) in 68 cortical brain regions, average thickness (ATH) in 68 cortical brain regions, surface area (SA) in 70 cortical and subcortical brain regions, and volume of white matter (VWM) in 45 brain regions. The DTI biomarkers consist of fractional anisotropy (FA), mean diffusivity (MD), radial diffusivity (RD), and longitudinal diffusivity (LD), within 57 brain regions. Additionally, we utilized 18F-florbetapir PET imaging to capture Aβ deposition in 109 cortical and subcortical brain regions (see Supplementary Table S1). It is important to note that the regions marked with a red font in Supplementary Table S1 are similarly repeated in VGM, ATH, and Aβ, and they are based on the Desikan-Killiany cortical parcellation (Desikan et al., 2006). The selection of regions was based on the criterion of availability without any missing data across all participants and biomarkers in each modality. This rigorous approach aimed to maintain data integrity and consistency throughout the analysis, ensuring that the results accurately reflected the information from all participants without any gaps or inconsistencies due to missing values. These diverse biomarkers collectively formed the foundation of our machine learning model.

For grouping our samples, we included multiple cohorts within each group: HC/MCI, HC/AD, MCI/AD, and HC/MCI/AD. The decision to consider multiple cohorts as single group was influenced by several factors. First, examining biomarker performance across all four groups allows for a more comprehensive assessment of the data. It provides insights into how biomarkers perform across various combinations of clinical conditions, which may offer valuable insights not captured by analyzing individual conditions separately. Second, in clinical practice, patients often present with mixed symptoms or may transition between different clinical conditions over time. By considering multiple condition categories simultaneously, the analysis better reflects the real-world complexity of neurodegenerative diseases and provides a more accurate representation of biomarker performance in clinical settings. Finally, understanding how biomarkers perform across different clinical conditions is crucial for clinical decision-making and treatment planning. By considering all relevant condition categories, the analysis provides more actionable insights for clinicians and researchers working in the field of neurodegenerative diseases.

In this study, the ensemble regression tree (ERT) was used to predict cognitive scores based on neuroimaging biomarkers. To ensure generalization of our models, we used a permutation process, repeated 500 iterations, to divide subjects into training (90%) and test (10%) sets across four groups: HC/MCI, HC/AD, MCI/AD, and HC/MCI/AD.

To address the bias and variance effects, we leveraged ensemble algorithms that combined the predictions of multiple estimators (Rokach, 2005). Our approach combined two ensemble learning techniques: bagging tree (BT) and random forest (RF). Bagging is an ensemble algorithm that fits multiple models on different subsets of a training set and then combines the predictions from all models. The BT aggregation aimed to decrease decision tree variance by merging multiple weak learners into stronger ones (Dey, 2016). We accomplished this by training 50 learner decision trees, each utilizing randomly selected subsets of data from training samples. The RF is an extension of bagging that also randomly selects subsets of features used in each data sample. We grew trees with surrogate splits to compute the error function for each variable at each split point (Springer and Kegelmeyer, 2008). For each biomarker (e.g., ATH, MD, and Aβ) the importance of each feature was determined by summing the changes in node risk resulting from splits on each feature and then dividing by the number of branch nodes. Node risk is defined as the mean squared error weighted by the node probability. Since we used surrogate split, the feature importance was calculated by summing changes in node risk across all splits at each branch node, including surrogate splits. The change in node risk highlights the difference between the risk of the parent node and the combined risk of its two children.

The feature importance metrics were derived from the average values across 500 iterations. Moreover, all reported mean squared error (MSE) values were based on the test sets across 500 iterations. We used analysis of variance (ANOVA) test followed by post hoc Tukey’s honestly significant difference (HSD) test with a family-wise error rate (FWER) of 0.01 to compare MSE of the nine biomarkers. Statistical significance for the categorical variables was assessed using chi-squared (χ2) test. It is noteworthy that we investigated the significance of various biomarkers in predicting cognitive measures by comparing their performance under two aspects based on: (A) MSE of predicting the cognitive measures in Section 3.1; (B) Correlation coefficient between actual values and predicted values in Section 3.1; and (C) ranking the significant brain regions in Sections 3.2 and 3.3. Implementation of the ERT algorithm and statistical analyses were carried out using MATLAB (2019b, MathWorks, Natick, MA).

For calculating the normalized importance (NI) of each biomarker (e.g., ATH, MD, and Aβ), we started by averaging the feature importance values across 500 iterations of the ERT. Next, we computed the square root of the average feature importance matrix and normalized it across all features associated with that specific biomarker. Finally, we divided the normalized values by the average across features to standardize the NI values across biomarkers. This standardization step ensures that the variation of the NI is centered around one, facilitating clearer comparisons and interpretations across biomarkers. By employing this approach, we obtained NI values that effectively reflect the relative importance of each biomarker in predicting the target outcome, while also accommodating variations in feature importance and variability across different biomarkers.

To identify significant brain regions, we compared the NI values with those generated through a random permutation test. This involved randomly shuffling the actual cognitive scores 500 times, alongside 500 iterations of train and test separation. Subsequently, we determined the 95th percentile (calculated as the mean plus two standard deviations of the 500 permuted NI values) for each biomarker as a threshold. Brain regions with NI values surpassing this threshold were deemed to have a notable influence on predicting cognitive scores. It’s important to note that when plotting the NI for each biomarker and subject group, we arranged the indices so that the larger values of NI curves consistently descend from left to right. This trend suggests that regions with lower indices are the ones significantly contributing to outcomes.

To enhance clarity regarding our permutation test, we constructed Supplementary Figure S1 as an illustrative example. This figure displays NI values alongside permutation results for Aβ across 109 brain areas in predicting MMSE scores. The black plots represent the mean of the permutation results in each group, with the standard deviation also depicted. Significant features were identified based on the 95th percentile of the fitted normal distribution, roughly equivalent to the mean plus two standard deviations. In Supplementary Figure S1A, focusing on the HC/MCI group representing individuals with non to mild cognitive decline, our findings suggest that no statistically significant brain region exhibited an NI higher than the 95th percentile of the fitted normal distribution from the permutation test. Conversely, examining the AD cohort in HC/AD, MCI/AD, and HC/MCI/AD groups (Supplementary Figures S1B–D), we observed that NI values in several brain regions significantly surpassed the 95th percentile of the fitted normal distribution from the permutation test results. It’s important to note that for better visualization in, we only plotted the mean of the permutation results for the HC/MCI/AD groups. This simplification offers insight into the general pattern of permutations results, typically centered around 1.

Finally, to visualize the significant ranked brain regions, we standardized NI values across four groups of individuals (HC/MCI, HC/AD, MC/AD, and HC/MCI/AD) and color coded them from lowest NI value (dark blue) to highest NI value (dark red).

Figures 1A–C, 2A–C and Tables 2, 3 provide insights into the performance of ERT models for predicting three cognitive scores (MMSE, CDRSB, and ADAS) based on nine neuroimaging biomarkers across four distinct groups (HC/MCI, HC/AD, MCI/AD, and HC/MCI/AD). Figures 1A–C and Table 2 offer results of MSE between predicted values and actual values, providing an indication of the predictive accuracy of the model. On the other hand, Figures 2A–C and Table 3 present data on the correlation coefficient between predicted values and actual values, showcasing the goodness of fit of the ERT model.

As observed by the results presented in Figures 1A–C, 2A–C and Tables 2, 3, determining the most effective biomarker within each subject group requires a comprehensive evaluation. This evaluation entails not only analyzing the MSE, which quantifies the average squared difference between predicted and actual values but also considering the correlation between these predicted and actual values. It is essential to note that in certain predictive models, particularly noticeable within the HC/MCI group, the biomarker with the lowest MSE values may not consistently exhibit the highest correlation with the actual values. This observation emphasizes the complexity of the predictive model’s performance assessment. While achieving the lowest MSE suggests optimal performance in terms of error metrics, it does not guarantee that the predicted and actual values align perfectly or adhere closely to a linear assumption.

We conducted one-way ANOVA across nine biomarkers for each group, examining MSE and correlation values in Tables 2, 3. The analyses involved multiple ANOVA tests to evaluate biomarkers performances across three cognitive scores (MMSE, CDRSB, and ADAS) and four groups (HC/MCI, HC/AD, MCI/AD, and HC/MCI/AD). The results of the one-way ANOVA on MSE values (Table 2) demonstrated statistically significant differences among at least two biomarkers (p < 0.0001, F > 16.97) across all groups and cognitive scores. The results of the one-way ANOVA on correlation values (Table 3) also show the significant differences among at least two biomarkers (p < 0.0001, F > 92.20) across all groups and cognitive scores. This finding suggests the potential presence of specific biomarkers demonstrating superior performance compared to others.

An in-depth pairwise examination utilizing Tukey HSD tests (considering FWER of 0.01) on the nine biomarkers for MMSE cognitive score prediction (Figures 1A, 2A) highlighted Aβ’s exceptional predictive capabilities in the presence of an AD cohort. Aβ emerged as a notably superior predictor, yielding significantly smaller MSE values and larger correlation (average correlation >0.66) compared to the other eight biomarkers (p-value <0.0001). This finding is also consistent in Figures 1B, 2B, emphasizing Aβ’s superiority in predicting CDRSB scores, where MSE values were significantly smaller and larger correlation (average correlation >0.66) than those based on other biomarkers (p-value <0.0001). For ADAS score, based on both MSE and correlation (Figures 1C, 2C) only in the HC/MCI/AD group, Aβ significantly (p < 0.0001) outperformed all other biomarkers (average correlation >0.70) in predicting ADAS scores.

The average results of MSE and correlation for sMRI modality biomarkers across all cognitive scores, as shown in Tables 2, 3, consistently demonstrate that ATH outperforms the other three biomarkers (VGM, VWM, and SA) within this modality. Similarly, the assessment of DTI modality biomarkers across all cognitive scores indicates that MD surpasses the other three biomarkers (FA, RD, and LD) within the DTI modality across most groups and cognitive scores. These findings imply that ATH and MD exhibit superior predictive accuracy and stronger correlations with actual values across various cognitive scores compared to the other biomarkers in the sMRI and DTI modalities, respectively.

In the previous section, we compared the biomarkers based on MSE values. In this section and Section 3.3, our focus is on comparing the biomarkers with the ranking of significant brain regions to identify the most effective biomarker. Figures 3, 4 primarily aim to identify the most crucial biomarkers in detecting the transition groups of subjects from normal cognition to severe cognitive decline. Each of the nine subplots in these figures corresponds to a neuroimaging biomarker, and their NI is compared across four groups of individuals (HC/MCI, HC/AD, MCI/AD, and HC/MCI/AD) in contrast to the average random permutation test results of HC/MCI/AD (only for visualization).

Figure 3 shows NI of the brain regions for predicting the MMSE score based on nine biomarkers. As shown in this figure, VGM, ATH, and VWM biomarkers demonstrated 3, 10, and 2 brain regions, respectively, with significantly larger NI in the HC/MCI group compared to random permutation test results. In contrast, we failed to identify any brain region with significantly larger NI in the HC/MCI group compared to random permutation based on Aβ. The presence of AD cohort in the other three groups increased the likelihood of identifying important brain regions. Specifically, as shown in Figure 3I, we found Aβ in more than 30 brain regions with NI significantly larger in groups with AD (HC/AD, MCI/AD, and HC/MCI/AD) than in the random permutation or HC/MCI group. Figures 3A–C, along with Figure 3I, illustrated that despite Aβ affecting more brain regions in the presence of AD cohort, the sMRI biomarkers VGM, VWM, and especially ATH achieved higher NI values. Figure 3D highlighted almost non-existent association between SA and MMSE scores. Additionally, DTI biomarkers (FA, MD, RD, and LD), indicated a significant difference in the NI of five brain regions between groups with and without AD.

Supplementary Figure S2 shows NI values for the prediction of CDRSB. The CDRSB prediction aligns closely with the MMSE score prediction, as shown in Figure 3, but the presence of the AD group led to higher NI in CDRSB compared to MMSE scores (Supplementary Figures S2A,B).

The NI results for ADAS prediction are illustrated in Figure 4. The HC/MCI group, as shown in this figure, provided particularly interesting results. Almost all biomarkers identified at least one significant brain region, not only in groups containing AD individuals but also in the HC/MCI group. Consequently, several brain regions were affected by all nine neuroimaging biomarkers. In the HC/MCI group, ATH emerged as the best predictor of ADAS scores, with two brain regions having NIs exceeding 1.5. Among DTI biomarkers, MD proved to be the most effective. In the MCI/AD, HC/AD, and HC/MCI/AD groups, which included the AD patients, 45, 25, and 15 brain regions, respectively, contributed to the prediction of Aβ, ATH, and MD.

Based on our prediction models, we observed several associations between ADAS and neuroimaging biomarkers across four groups of individuals transitioning from normal cognition to severe dementia. Aβ, ATH, and MD were identified as more important neuroimaging biomarkers for predicting cognitive scores based on PET, sMRI, and DTI, respectively. Figures 5–7 depict the NI values of these biomarkers for ADAS prediction. In the Supplementary materials, we also present the NI values of these three biomarkers for MMSE and CDRSB prediction (Supplementary Figures S3–S8). As previously mentioned, our analysis involved calculating feature importance across 68 regions based on the Desikan-Killiany cortical parcellation for both ATH and Aβ outcomes. Additionally, for Aβ, we examined 41 subcortical regions as detailed in Supplementary Table S1. However, Aβ in the subcortical regions did not demonstrate significant feature importance in comparison to the cortical regions, consistently ranking lower in importance. To ensure clarity and facilitate a more direct comparison between ATH and Aβ, we chose to focus exclusively on the 68 cortical regions common to both biomarkers in Figures 5, 6 and Supplementary Figures S3, S4, S6, S7.

The NI of cortical brain regions based on Aβ is illustrated in Figure 5. The top-ranked significant regions showed with dark yellow to red colors include the right and left transverse temporal gyrus, and right precuneus cortex in the MCI/AD group; right transverse temporal gyrus, right precuneus cortex in the HC/MCI/AD group; and right and left superior frontal gyrus, right and left transverse temporal gyrus, and right precuneus cortex in the HC/AD group. Notably, the right transverse temporal gyrus and right precuneus cortex consistently showed high NI throughout three groups with the presence of the AD group. The NI value of each brain region in the HC/MCI group was below the assumed threshold and not significant.

Figure 6 shows the most discriminatory brain regions according to ATH. The top-ranking significant regions showed with dark yellow to red colors for the HC/MCI group were the right and left entorhinal cortex and the right fusiform cortex. The other three groups (MCI/AD, HC/MCI/AD, and HC/AD) had the same top-ranking regions: left and right entorhinal cortex, left inferior temporal gyrus, and left middle temporal gyrus. In groups from one through four (1: HC/MCI, 2: MCI/AD, 3: HC/MCI/AD, and 4: HC/AD), the NI values of the left and right entorhinal cortex gradually increased, revealing the significant progression of these two regions based on ATH.

According to previous results and as shown in Figure 7, MD was the most discriminatory DTI biomarker. In each group, the top significant regions were as follows: left cingulum and right sagittal stratum in the HC/MCI group; left uncinate fasciculus in the MCI/AD group; left uncinate fasciculus and left cingulum in the HC/MCI/AD group; and left uncinate fasciculus and left cingulum in the HC/AD group.

After careful consideration, we chose to focus on the HC/MCI/AD group, which encompasses all three cohorts and identified the three most effective biomarkers for each neuroimaging modality (Aβ, ATH, and MD). We used these biomarkers to demonstrate a consensus on the top brain regions for predicting three cognitive scores (i.e., MMSE, CDRSB, and ADAS), as illustrated in Figure 8. For each biomarker, we identified four overlap states: no consensus, one consensus on the cognitive scores, two consensuses on cognitive scores, and consensus on all three cognitive scores. Despite differences in biomarkers, we identified six brain regions that consistently emerged as the top regions for predicting all three cognitive scores: the right transverse temporal region for Aβ, the left and right entorhinal cortex, the left inferior temporal gyrus, the left middle temporal gyrus for ATH, and the left uncinate fasciculus for MD.