Examining the human brain as an integrative network of functionally interacting brain regions can provide new insights about large-scale neuronal communication in the human brain. It also provides a platform to examine how functional connectivity and information integration relates to human behavior and how this organization may be altered in neurodegenerative diseases. Independent component analysis (ICA) is a powerful mathematical tool for simultaneously extracting a variety of coherent functional networks (Beckmann and Smith, 2004; Calhoun and Adali, 2006), while separating them from different sources of noise induced by head motion or physiological confounds (Salimi-Khorshidi et al., 2014). It decomposes resting-state functional magnetic resonance imaging (rsfMRI) data into distinct networks that are temporally correlated but maximally independent in the spatial domain. ICA has been successfully used to study a wide variety of neurological conditions such as Alzheimer’s disease (Dennis and Thompson, 2014), multiple sclerosis (Castellazzi et al., 2018), and traumatic brain injury (Iraji et al., 2015; Li et al., 2020).

There are two significant challenges in the application of ICA for functional connectivity analysis. First, the output of ICA decomposition depends on a fundamental parameter: the total number of independent components. The optimal choice of this parameter, related to determining the effective data dimension, remains an open question. Choosing too small a value for the effective data dimension might under-decompose the signal and generate “fused components.” At the same time, choosing too high a value might over-decompose the data and lead to splitting of meaningful components (Li et al., 2007; Hui et al., 2013). The other major challenge is that ICA does not provide any labeling or ordering of its components, and the user must therefore determine how the functional network corresponds to each component. When dealing with large data sets, this manual process can be labor intensive and requires expertise. One possible way to deal with these challenges is by running ICA with different numbers of components and selecting the one that extracts the best-fitted result for each desired functional network (Kairov et al., 2017). Therefore, a fully automatic algorithm that can reliably classify ICA components into various types of functional brain networks while also generating a goodness-of-fit or confidence metric would provide a unified solution to these two challenges.

One approach to mitigate the burden of identifying resting state networks (RSNs) in large data sets is to focus on group analyses. However, there is an increasing need for subject-specific analyses, such as for surgical planning (Catalino et al., 2020), brain stimulation (Jung et al., 2020), and in studies where group bias may be a concern (Michael et al., 2014). For these situations (or analyses), one can apply an automated approach for identifying particular networks from subject-specific ICA decompositions. Template matching has been the most commonly used method for automatic ICA identification in the last two decades. In (Demertzi et al., 2014), a Pearson correlation was computed between the template and each ICA component, and the network corresponding to the template yielding the highest correlation was selected as the identified network. In (Greicius et al., 2004, 2007), a linear template-matching procedure was applied, which involves taking the average z-score of voxels falling within the template minus the average z-score of voxels outside the template and selecting the component in which this difference (the goodness-of-fit) was the greatest.

Deep learning using Convolutional Neural Networks (CNNs) has become the state-of-the-art for image segmentation, classification, detection and retrieval related tasks (Khan et al., 2020). Recent work (Zhao et al., 2018) has demonstrated the capability of CNNs for automatic recognition of spatial resting-state network maps, although this and other machine learning approaches for subject-specific identification of RSNs (Hacker et al., 2013; Lv et al., 2015) were not designed to work with ICA. In Vergun et al., 2016, several machine learning approaches, including a one layer neural network, were used to label ICA components in epilepsy, achieving up to 90% accuracy. In Nozais et al., 2021, a multilayer perceptron deep neural network approach for identifying 45 RSNs based on ICA was implemented using two databases of over 250 subjects for training and over 1,900 subjects for testing, achieving an accuracy of 92%. In our previous study (Chou et al., 2018), we proposed a fully automatic deep 3D 2-channel CNN method based on stacks of inception modules (denoted as inception CNN) that can reliably identify ICA components corresponding to eight major functional networks with 98% accuracy rate over 108 testing samples.

Convolutional Neural Networks can be prone to erroneously giving high-confidence predictions to out-of-distribution examples– i.e., inputs that are markedly different from any samples in the data used to train the CNN, and that do not correspond to any particular class the CNN was trained to predict (Wang et al., 2017). A seemingly straightforward approach to handle this issue is to enlarge the training set of both in- and out-of-distribution examples. However, the number of out-of-distribution examples can be infinitely many, making training computationally expensive and intractable, since its underlying space is unbounded. Moreover, ensuring accurate classification of in-distribution samples and correct detection out-of-distribution samples may require exceedingly large architectures, further complicating the training process (Liang et al., 2018). To handle these issues, we (Chou et al., 2019) explored a semi-supervised Generative Adversarial Network (denoted as sGAN) (Augustus, 2016) for the classification on ICA components by using a shared discriminator/multi-class classifier that discriminates real data from synthetic while also predicting the class label. The sGAN achieved a high accuracy rate (98.7%) for eight functional networks without requiring additional labeled images during training or access to out-of-distribution examples. The trained generator could also be used to synthesize realistic functional networks as a source of data augmentation. However, obtaining a large enough training set for the sGAN was challenging, requiring 501 ICA manually labeled components. To extend the sGAN to classify RSNs beyond the 8 components would require a substantial number of additional components to be labeled for training.

Poor performance on a limited labeled data set is a common problem with CNNs (O’Mahony et al., 2019). In medical imaging applications, collecting this much data may not always be feasible. Furthermore, there are situations in which a user is interested in an additional (new) classification label outside the set of original labels. Standard approaches require that the network be completely retrained in order to add a new label. To circumvent these limitations, we propose to use a metric learning approach based on a deep Siamese Network (called SiameseICA) for the classification of ICA components with a relatively small amount of training data. Siamese networks provide a framework to learn metric embeddings that provide a discriminative distance function between classes. The yielded distance metric is able to enlarge the separability of samples from different classes and reduce the variations of samples from the same class simultaneously. By learning distance functions rather than explicitly learning classification, metric learning approaches have successfully been used for classification yielding state-of-the-art performance on problems with small numbers of labeled samples without overfitting (Hoffer and Ailon, 2014; Cao et al., 2020). Siamese Networks have been applied to various problems, including image recognition and verification, visual tracking, novelty and anomaly detection, one-shot and few-shot learning (Bromley et al., 1994; Koch et al., 2015; Schroff et al., 2015; Bertinetto et al., 2016; Snell et al., 2017; Rana and Kisku, 2020). They are useful in cases where there are large numbers of classes with small numbers of observations of each because they avoid the problem of directly classifying an image. Rather, they take two images as inputs and compute feature vectors in a low-dimensional space to measure the similarity between the images. We show how this can be used to address the issues of labeling ICA components and determining the number of components. Although SiameseICA can be applied for group ICA analysis, we focus our evaluation primarily on single subject ICA because this leads to noisier components that are typically more difficult to identify.

This paper makes the following contributions: (1) we propose a novel technique for ICA classification with low training samples that outperformed CNN based deep learning methods and traditional template matching methods; (2) we demonstrate that the model can generalize to new categories, unseen in the training process; (3) we show that the best-fit ICA components representing the functional networks can be effectively identified by the proposed technique, thereby removing the requirement to know the number of ICA components a priori; (4) we show that our technique is robust to the scan-rescan variation, demonstrating high reproducibility on classifying ICA components; (5) we perform an analysis on mild TBI, severe TBI and healthy subjects using networks identified by the proposed technique and show significant group differences. The source code of SiameseICA, as well as a trained model, is available on Github: http://github.com/chouyiyu/deepICAclassifier.

Four sets of data were used to train and evaluate the proposed method. The first dataset, denoted as CNRM, consisted of 179 scans including 41 healthy, 93 mild-TBI and 45 severe-TBI subjects enrolled in a natural history study of TBI and scanned on a Siemens Biograph mMR 3T scanner. The rsfMRI images (TR = 2,000 ms, TE = 27 ms, flip angle = 90 degrees, voxel size = 3.43 mm × 3.43 mm × 3 mm, dimensions = 64 × 64 × 36, Time points = 206) were used for training, testing (sections “Classification Performance,” “One-Shot Classification for the Caudate Network”) and TBI group analysis (section “Group comparison between mild TBI, severe TBI and healthy subjects”). T1-weighted MPRAGE images (TR = 2,530 ms, TE = 3 ms, flip angle = 7 degrees, TI = 1,100 ms, voxel size = 1 mm × 1 mm × 1mm, dimensions = 256 × 256 × 176) were also used for registration purposes. Participants in the study were enrolled under a protocol approved by the institutional review board at the National Institutes of Health and consented to research use of their imaging data. Training data were used for optimizing the SiameseICA network parameters, and the test data were used to examine the accuracy of identifying a single ICA component. For training and testing data, 30 ICA components were extracted from each subject using MELODIC (Multivariate Exploratory Linear Optimized Decomposition into Independent Components) (Beckmann and Smith, 2004) and manually identified via visual inspection as being one of eleven resting-state networks (RSNs): default mode, medial visual, occipital visual, lateral visual, motor, cerebellum, auditory, executive, salience, left dorsal attention and right dorsal attention. RSNs were only selected when they were identified with high confidence. Since some components may represent split or merged networks, all 11 RSNs were not necessarily identified for each subject. In addition to individual subject ICA components, group ICA processing was performed by creating random subsets of n subjects to generate additional training and testing image sets with more reproducible and fully-formed network representations (Zuo et al., 2010). The final sizes of the training set and testing set were 20 and 50, respectively, for each of the eleven RSNs considered.

The second dataset consisted of rsfMRI from 10 healthy subjects acquired using a GE Signa 3T scanner (TR = 2,000 ms, voxel size = 3.75 mm × 3.75 mm × 4 mm, dimensions = 64 × 64 × 36, Time points = 175), denoted as Milwaukee-b, obtained from the 1,000 Functional Connectomes Project (FCP) at http://fcon_1000.projects.nitrc.org/fcpClassic/FcpTable.html, Biswal et al. (2010). A T₁-weighted MPRAGE image (voxel size = 0.938 mm × 0.938 mm × 1 mm, dimensions = 256 × 256 × 144; detailed information not available) was also acquired for each subject. This dataset was used for demonstrating the generalizability and separability between classes of the proposed SiameseICA method (section “Generalizability, separability and reproducibility”).

The third dataset (section “Classification Performance”), denoted as BeijingFCP, was also obtained from FCP and includes 198 healthy subjects with 621 independent components manually labeled for the validation. MRI was acquired using a Siemens Trio 3T scanner. Specifically, an echo-planar imaging (TR = 2,000 ms, voxel size = 3.125 mm × 3.125 mm × 3.6 mm, dimensions = 64 × 64 × 33, Time points = 225) sequence was performed to obtain resting state fMRI images. A T₁-weighted MPRAGE (voxel size = 1.0 mm × 1.0 mm × 1.0 mm, dimensions = 256 × 256 × 176; detailed information not available) was carried out to acquire a high-resolution anatomical image of the brain structure.

The fourth dataset, denoted as MMRR-21 (section “Group Comparison Between Mild TBI, Severe TBI and Healthy Subjects”) consists of scan-rescan image sessions performed on a Philips Achieva 3T (TR = 2,000 ms, TE = 30 ms, flip angle = 75 degrees, voxel size = 3 mm × 3 mm × 3.973 mm, dimensions = 80 × 80 × 37, Time points = 210) on 21 healthy volunteers¹. This dataset was intended to be a resource for statisticians and imaging scientists to be able to quantify the reproducibility of their imaging methods. Each subject from this dataset also had a T₁-weighted MPRAGE image (TR = 6.7 ms, TE = 3.1 ms, TI = 842 ms, flip angle = 8 degrees, voxel size = 1 mm × 1 mm × 1.2 mm, dimensions = 256 × 256 × 170).

This section provides details on the proposed method for identifying RSNs from the output of an ICA algorithm. The method comprises four major stages: (1) pre-processing; (2) application of ICA; (3) training the SiameseICA with a triplet loss function; and (4) finding the most likely class for a test image using the trained model.

In this study, we present a deep learning approach to ICA component classification based on a Siamese Network that is trained from a small dataset and still demonstrates superior performance. The architecture creates a low-dimensional embedding space for RSNs by mapping images with the same class to nearby points in a low-dimensional space using a triplet loss function. The reduced-feature representation is then used to identify images from the dataset that are most similar to a test image. We found that five examples of each RSN were sufficient to achieve accurate performance. An advantage of our approach is that it obviates the need for estimating the number of components required when applying ICA.

SiameseICA can be generalized to classify new categories without the need for additional retraining. We showed that given sufficient initial training data to define an embedding space for the support set, only one sample was required as a reference for a completely new RSN class. This approach is referred to as one-shot classification, drastically reducing the need for labeled datasets. In addition, we expect the model is more robust to class imbalance as it can be used on a dataset where very few examples exist for some classes.

The identification of the best ICA components for each RSN on different datasets showed good qualitative performance and reproducibility on independent test data, confirming the feasibility of the proposed method in research studies that may involve heterogeneous data. We also demonstrated the value of the approach in an example research study of traumatic brain injury. Using SiameseICA, we found that the default mode and salience networks were altered in mild and severe traumatic brain injury patients compared to healthy controls.

Several avenues will be explored in future work. Currently, the network outputs relative distances between the input and all of the reference images. Converting these distances into probabilities will be more intuitive as similarity scores. In addition, further optimization of the model, such as choices of hyperparameter tuning, the number of layers, number of nodes for each layer, and the learning rate of the network may yield higher performance.

Publicly available datasets were analyzed in this study. This data can be found here: http://fcon_1000.projects.nitrc.org/fcpClassic/FcpTable.html and http://www.nitrc.org/projects/multimodal. TBI imaging data is not yet publicly available but will be made available upon request to LC, chanle@cc.nih.gov.

The studies involving human participants were reviewed and approved by National Institutes of Health IRB. The patients/participants provided their written informed consent to participate in this study.

YC and DP contributed to conception and design of the study and wrote sections of the manuscript. LC provided data for the study. YC labeled data for this study with guidance from CC and JB. SR helped the analysis of data. YC wrote the first draft of the manuscript. All authors contributed to manuscript revision, read, and approved the submitted version.

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.