On the reliability of deep learning-based classification for Alzheimer’s disease: Multi-cohorts, multi-vendors, multi-protocols, and head-to-head validation - Song, Yeong-Hun; Yi, Jun-Young; Noh, Young; Jang, Hyemin; Seo, Sang Won; Na, Duk L.; Seong, Joon-Kyung (Frontiers in Neuroscience, 2022)
Structural changes in the brain due to Alzheimer’s disease dementia (ADD) can be observed through brain T1-weighted magnetic resonance imaging (MRI) images. Many ADD diagnostic studies using brain MRI images have been conducted with machine-learning and deep-learning models. Although reliability is a key in clinical application and applicability of low-resolution MRI (LRMRI) is a key to broad clinical application, both are not sufficiently studied in the deep-learning area. In this study, we developed a 2-dimensional convolutional neural network-based classification model by adopting several methods, such as using instance normalization layer, Mixup, and sharpness aware minimization. In order to train the model, MRI images from 2765 cognitive normal individuals and 1192 patients with ADD from the Samsung medical center cohort were exploited. To assess the reliability of our classification model, we designed external validation in multiple scenarios: 1) multi-cohorts validation using four additional cohort datasets including more than 30 different centers in multiple countries, 2) multi-vendors validation using three different MRI vendor subgroups, 3) LRMRI image validation, and finally, 4) head-to-head validation using ten pairs of MRI images from ten individual subjects scanned in two different centers. For multi-cohorts validation, we used the MRI images from 739 subjects from the Alzheimer’s Disease Neuroimaging Initiative cohort, 125 subjects from the Dementia Platform of Korea cohort, 234 subjects from the Premier cohort, and 139 subjects from the Gachon University Gil Medical Center. We further assessed classification performance across different vendors and protocols for each dataset. We achieved a mean AUC and classification accuracy of .9868 and .9482 in 5-fold cross-validation. In external validation, we get a comparable AUC of .9396 and classification accuracy of .8757 to other cross-validation studies in the ADNI cohorts. Furthermore, we observed the possibility of broad clinical application through LRMRI image validation by achieving a mean AUC and classification accuracy of .9404 and .8765 at cross-validation, and AUC and classification accuracy of .8749 and .8281 at the ADNI cohort external validation.