Inflammatory demyelinating diseases of the central nervous system (CNS) are important causes of nontraumatic neurological disabilities (1, 2). Multiple sclerosis (MS), neuromyelitis optica spectrum disorder (NMOSD), and myelin oligodendrocyte glycoprotein antibody-associated disease (MOGAD) are major disease entities in this field (3, 4). Increasing evidence indicates that NMOSD is an independent disorder associated with the expression of anti-aquaporin-4 (AQP4) antibodies rather than a variant of MS (5, 6). With the discovery of antibodies targeting myelin oligodendrocyte glycoprotein (MOG) in AQP4 antibody-negative NMOSD, MOGAD is now recognized as a unique immunological entity that is distinct from both MS and NMOSD (4, 7).

Indeed, MS, NMOSD and MOGAD exhibit divergent pathogeneses, treatment options for relapse prevention, and prognoses (8), and their diagnosis is mainly based on combined results involving clinical findings, radiological manifestations, and cerebrospinal fluid and serological tests. In conventional magnetic resonance imaging (MRI), bilateral periventricular white matter and cortical lesions are often considered typical features of MS (9), whereas longitudinally extensive transverse myelitis and posterior long-segment optic nerve lesions are more specific to NMOSD (10). MOGAD is considered to exhibit intermediate MRI features between those of MS and NMOSD (11). However, the MRI manifestations in some cases are indistinguishable among these conditions. Owing to the presence of overlapping clinical and radiological findings among these disorders, differential diagnosis can be challenging.

Autoantibody tests normally confer high sensitivity, but are invasive and time-consuming to obtain results, and conversion to an antibody-negative status may occur during the disease course (12). An improper choice of treatment may lead to disease deterioration. For example, disease-modifying therapies such as interferon beta and dimethyl fumarate are recommended as the standard treatment for MS but may exacerbate NMOSD and increase relapse rates (13, 14). Immunosuppressive agents such as azathioprine and mycophenolate mofetil are the first-line therapies for NMOSD and MOGAD. Moreover, several emerging therapies have shown different efficacies in controlling disease recurrence in patients with AQP4+ NMOSD and MOGAD (15). Thus, to reduce the delay from disease onset to appropriate treatment, thereby improving clinical benefits, there is an urgent need to develop an effective non-invasive approach for a rapid and precise differential diagnosis.

Many researchers have explored imaging differences among the three diseases using multi-model MRI sequences, indicating that the classification of MR image features can be helpful in the diagnosis of these diseases. Duan et al. (16) compared brain structural alterations on MRI, and demonstrated cortical and subcortical atrophy without severe white matter rarefaction in MOGAD in comparison with MS and AQP4+ NMOSD, whereas diffusion MRI measurements showed lower fractional anisotropy and higher mean diffusivity in MS. Moreover, Banks et al. (17) retrospectively compared the involvement of the brainstem or cerebellar region in CNS inflammatory demyelination diseases, and revealed that diffuse middle cerebellar peduncle MRI lesions favored a diagnosis of MOGAD over MS and AQP4+ NMOSD. They further showed that diffuse medulla, pons, or midbrain MRI lesions occasionally occurred in MOGAD and AQP4-IgG-NMOSD but never in MS. Although these findings have revealed image-dependent differentiation, few studies have been conducted using conventional MRI and its possible integration with the deep learning technique into the clinical workflow.

Recent advances in artificial intelligence have prompted the development of deep learning-based algorithms designed for the automatic classification of demyelinating diseases based on conventional MRI (11, 18–20). For example, Kim et al. (18) constructed a three-dimensional convolutional neural network (CNN) deep-learning-based model using brain MRI and clinical information to differentiate NMOSD from MS, achieving a moderate accuracy of 71.1%, sensitivity of 87.8%, and specificity of 61.6%. Rocca et al. (20) also applied a deep-learning algorithm based on CNN using brain MRI to discriminate between MS and its mimics, including NMOSD, revealing the highest accuracy (98.8%) and specificity (98.4%), and the lowest false positive rate (4.4%) for MS.

Notably, these aforementioned studies mostly used brain MRI with or without incorporation of clinical information for image-based classification using a traditional CNN. There has been minimal exploration of integrated MRI sequences and multi-site consideration of neuroimaging protocols. To address this gap, we here proposed a novel deep-learning algorithm, according to Co-scale conv-attentional image Transformers (CoaT)-based network (21), which was trained on multi-sequence (coronal T2-weighted and sagittal T2-FLAIR) and multi-location (brain, cervicothoracic and thoracolumbar spinal cord) MRI. The combined image sequences represent a better reflection of a realistic clinical setting and may contribute to increased classification accuracy.

In this study, we proposed transformer-based deep-learning model to differentiate among MS, AQP4+ NMOSD and MOGAD based on conventional brain and spinal cord MRI. This novel transformer-based model architecture was designed to be versatile in handling with images of multiple sequences and scanning locations whichever were available at clinical practice, for differentiating among MS, NMOSD, and MOGAD at a high accuracy. The fusion model using images of combined locations exhibited significantly higher accuracy than the models trained with a single location MRI as well as two experienced radiologists, which referred to possible alternative tool in assisting clinical decisions for a fast and accurate treatment referral.

To our knowledge, this is the first study to use an ensemble-location approach in the task of multi-category classification to improve the differential diagnosis of CNS inflammatory demyelinating diseases. Given patients suspected with demyelinating disease may undergo brain and/or spinal cord MRI scans, the model was developed to handle with the data in the single- or multi- location manner on conventional MR images. When MR images at one location were taken, our results showed that taking brain images as model inputs had a relative higher accuracy in classifying these three conditions than using spinal cord images only (pooled accuracy: 75.9% vs. 62.7%). Moreover, when multi-location MR images were available, the fusion model demonstrated an improved accuracy of 81.4%, which was comparable compared to the performance of two neuroradiologists (accuracy: 64.4% for rater 1 and 74.6% for rater 2). Meanwhile, the fusion model exhibited higher Kappa and MCC than that of human raters.

Previous studies have used only brain or spinal cord images (11, 20), along with accompanying clinical variables (18, 33) for prediction. Among these studies, Rocca et al. (20) used CNN and achieved an accuracy of 98.8% for the differential diagnosis of MS from NMOSD. However, it shall be noted that this high diagnostic performance of these models may be attributed to obvious differences in lesion volume and distribution, which may lead to over-estimated classification accuracy of the models. This potential bias was addressed in our study by enrolling images from patients whose MRI were taken at multi-location and the features extracted broad heterogeneity in lesion distribution were regarded as shared manifestations among target demyelinated disease.

Deep-learning algorithms are an active area of research in medical image processing. These algorithms can extract information from conventional MR images, including features that cannot be recognized by the human eye, and help to make a more accurate diagnosis (34), prognosis evaluation (35) and therapeutic guidance (36). Our proposed multimodal MIL-CoaT deep-learning network can perform multicategory classification using hybrid MRI sequences. This model was developed based on an MIL strategy and incorporated the CoaT transformer as the basic network structure. Specifically, the MIL strategy used slice as the model input, which leads to increased size of data in the training stage. Together with the slice-level data augmentation, we were able to mitigate the issue of imbalanced samples between brain and spinal cord MRI scans. Furthermore, during the training stage, each epoch has the same amount of training sample at each group. The samples of the categories with smaller sample size were transformed and reused through data augmentation technique which can be treated as the new samples in iterative cycles. Moreover, the CoaT-based transformer structure has the characteristics of dynamic attention and global context fusion, which are not available with a traditional CNN. Thus, by combining the extracted features and subsequent instance-wise feature fusion using attention pooling, we obtained a model with improved generalization ability.

We reviewed misclassified cases to determine their imaging characteristics and speculate possible reasons. We found that the cases misclassified as AQP4+ NMOSD can exhibit certain characteristics such as the presence of brain lesions adjacent to the fourth ventricle, and multiple short segment lesions that fused into long segment lesions in the spinal cord, which may resemble the appearance of AQP4+ NMOSD. Moreover, the presence of severe brain atrophy, and involvement of cortical or juxtacortical regions may be a possible cause of being misdiagnosed as MS.

This study indeed had some limitations. First, optic nerve lesions may exist in CNS demyelinating diseases, whereas the limited imaging data of optic nerve MRI were not suitable for inclusion in big data analysis. Second, the lack of thin-section MRI data due to time-cost issues may have resulted in lower image resolution and less precise information. However, it shall be mentioned that in most hospitals in China, the conventional MRI protocol with thick slices was utilized for the diagnosis of patients suspected with demyelinated diseases. This is attributed to larger number of patients and longer waiting times compared to western countries. Under this condition, we focused on research-quality MR images and developed the deep learning model that could be used to classify major types of demyelinated disease with high accuracy. Third, the loss of accuracy with the fusion model including spinal cord MRI in MS cases may have been attributed to the limited sample size and the fact that some MS patients did not have lesions in the spinal cord. This would be an important feature for co-learning to ensure reliability in applying the model to clinical practice.

Overall, our results provide evidence that deep-learning networks may be used for differential diagnosis based on brain and spinal cord MRI for patients with MS, AQP4+ NMOSD, and MOGAD. To our knowledge, this is the first study to apply the newly proposed MIL-CoaT transformer-based deep-learning algorithm to conventional MRI of multiple locations and sequences in attempt to solve the clinical challenge of diagnosing CNS demyelinating diseases. This evidence is also expected to motivate future research for delving into the clinical and radiological basis of deep-learning networks, as well as to validate the findings with a prospective study design.

The original contributions presented in the study are included in the article/ Supplementary Material . Further inquiries can be directed to the corresponding authors.

The studies involving human participants were reviewed and approved by the Ethics Committee of the Second Xiangya Hospital of Central South University. Written informed consent from the participants’ legal guardian/next of kin was not required to participate in this study in accordance with the national legislation and the institutional requirements.

JL and WL conceived and designed the study. XC collected imaging and clinical data. CH, FT, and WL reviewed the data. BL, WC, and RY analyzed and interpreted the data. CH drafted the manuscript. All authors contributed to the article and approved the final version.

This work was supported by the Clinical Research Center for Medical Imaging in Hunan Province (2020SK4001). Leading talents of scientific and technological innovation in Hunan Province in 2021 (2021RC4016). The accurate localization study of mild traumatic brain injury based on deep learning through multimodal image and neural network (2021gfcx05).

Authors BL, WC, and RY were employed by Infervision Medical Technology Co., Ltd.

The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.