<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<!DOCTYPE article PUBLIC "-//NLM//DTD Journal Publishing DTD v2.3 20070202//EN" "journalpublishing.dtd">
<article xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink" article-type="research-article">
<front>
<journal-meta>
<journal-id journal-id-type="publisher-id">Front. Neuroinform.</journal-id>
<journal-title>Frontiers in Neuroinformatics</journal-title>
<abbrev-journal-title abbrev-type="pubmed">Front. Neuroinform.</abbrev-journal-title>
<issn pub-type="epub">1662-5196</issn>
<publisher>
<publisher-name>Frontiers Media S.A.</publisher-name>
</publisher>
</journal-meta>
<article-meta>
<article-id pub-id-type="doi">10.3389/fninf.2017.00059</article-id>
<article-categories>
<subj-group subj-group-type="heading">
<subject>Neuroscience</subject>
<subj-group>
<subject>Original Research</subject>
</subj-group>
</subj-group>
</article-categories>
<title-group>
<article-title>Multimodal Discrimination of Schizophrenia Using Hybrid Weighted Feature Concatenation of Brain Functional Connectivity and Anatomical Features with an Extreme Learning Machine</article-title>
</title-group>
<contrib-group>
<contrib contrib-type="author">
<name><surname>Qureshi</surname> <given-names>Muhammad Naveed Iqbal</given-names></name>
<xref ref-type="aff" rid="aff1"><sup>1</sup></xref>
<xref ref-type="author-notes" rid="fn002"><sup>&#x02020;</sup></xref>
<uri xlink:href="http://loop.frontiersin.org/people/134222/overview"/>
</contrib>
<contrib contrib-type="author">
<name><surname>Oh</surname> <given-names>Jooyoung</given-names></name>
<xref ref-type="aff" rid="aff1"><sup>1</sup></xref>
<xref ref-type="author-notes" rid="fn002"><sup>&#x02020;</sup></xref>
<uri xlink:href="http://loop.frontiersin.org/people/425376/overview"/>
</contrib>
<contrib contrib-type="author">
<name><surname>Cho</surname> <given-names>Dongrae</given-names></name>
<xref ref-type="aff" rid="aff1"><sup>1</sup></xref>
<uri xlink:href="http://loop.frontiersin.org/people/433997/overview"/>
</contrib>
<contrib contrib-type="author">
<name><surname>Jo</surname> <given-names>Hang Joon</given-names></name>
<xref ref-type="aff" rid="aff2"><sup>2</sup></xref>
<uri xlink:href="http://loop.frontiersin.org/people/7815/overview"/>
</contrib>
<contrib contrib-type="author" corresp="yes">
<name><surname>Lee</surname> <given-names>Boreom</given-names></name>
<xref ref-type="aff" rid="aff1"><sup>1</sup></xref>
<xref ref-type="author-notes" rid="fn001"><sup>&#x0002A;</sup></xref>
<uri xlink:href="http://loop.frontiersin.org/people/255410/overview"/>
</contrib>
</contrib-group>
<aff id="aff1"><sup>1</sup><institution>Department of Biomedical Science and Engineering, Institute of Integrated Technology, Gwangju Institute of Science and Technology</institution> <country>Gwangju, South Korea</country></aff>
<aff id="aff2"><sup>2</sup><institution>Department of Neurologic Surgery, Mayo Clinic</institution> <country>Rochester, MN, United States</country></aff>
<author-notes>
<fn fn-type="edited-by"><p>Edited by: Sergey M. Plis, Mind Research Network, United States</p></fn>
<fn fn-type="edited-by"><p>Reviewed by: Thomas Natschl&#x000E4;ger, Software Competence Center Hagenberg, Austria; K. Jarrod Millman, University of California, Berkeley, United States</p></fn>
<fn fn-type="corresp" id="fn001"><p>&#x0002A;Correspondence: Boreom Lee <email>leebr&#x00040;gist.ac.kr</email></p></fn>
<fn fn-type="other" id="fn002"><p>&#x02020;These authors have contributed equally to this work</p></fn></author-notes>
<pub-date pub-type="epub">
<day>08</day>
<month>09</month>
<year>2017</year>
</pub-date>
<pub-date pub-type="collection">
<year>2017</year>
</pub-date>
<volume>11</volume>
<elocation-id>59</elocation-id>
<history>
<date date-type="received">
<day>25</day>
<month>04</month>
<year>2017</year>
</date>
<date date-type="accepted">
<day>25</day>
<month>08</month>
<year>2017</year>
</date>
</history>
<permissions>
<copyright-statement>Copyright &#x000A9; 2017 Qureshi, Oh, Cho, Jo and Lee.</copyright-statement>
<copyright-year>2017</copyright-year>
<copyright-holder>Qureshi, Oh, Cho, Jo and Lee</copyright-holder>
<license xlink:href="http://creativecommons.org/licenses/by/4.0/"><p>This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.</p></license>
</permissions>
<abstract><p>Multimodal features of structural and functional magnetic resonance imaging (MRI) of the human brain can assist in the diagnosis of schizophrenia. We performed a classification study on age, sex, and handedness-matched subjects. The dataset we used is publicly available from the Center for Biomedical Research Excellence (COBRE) and it consists of two groups: patients with schizophrenia and healthy controls. We performed an independent component analysis and calculated global averaged functional connectivity-based features from the resting-state functional MRI data for all the cortical and subcortical anatomical parcellation. Cortical thickness along with standard deviation, surface area, volume, curvature, white matter volume, and intensity measures from the cortical parcellation, as well as volume and intensity from sub-cortical parcellation and overall volume of cortex features were extracted from the structural MRI data. A novel hybrid weighted feature concatenation method was used to acquire maximal 99.29% (<italic>P</italic> &#x0003C; 0.0001) accuracy which preserves high discriminatory power through the weight of the individual feature type. The classification was performed by an extreme learning machine, and its efficiency was compared to linear and non-linear (radial basis function) support vector machines, linear discriminant analysis, and random forest bagged tree ensemble algorithms. This article reports the predictive accuracy of both unimodal and multimodal features after 10-by-10-fold nested cross-validation. A permutation test followed the classification experiment to assess the statistical significance of the classification results. It was concluded that, from a clinical perspective, this feature concatenation approach may assist the clinicians in schizophrenia diagnosis.</p></abstract>
<kwd-group>
<kwd>Schizophrenia</kwd>
<kwd>COBRE</kwd>
<kwd>neuroimaging</kwd>
<kwd>global functional connectivity</kwd>
<kwd>group ICA</kwd>
<kwd>hybrid weighted feature concatenation</kwd>
<kwd>machine learning</kwd>
</kwd-group>
<counts>
<fig-count count="6"/>
<table-count count="6"/>
<equation-count count="4"/>
<ref-count count="75"/>
<page-count count="14"/>
<word-count count="9217"/>
</counts>
</article-meta>
</front>
<body>
<sec sec-type="intro" id="s1">
<title>Introduction</title>
<p>Schizophrenia is a major psychiatric disorder that reportedly affects one percent of the population (Mao et al., <xref ref-type="bibr" rid="B45">2009</xref>). This disorder can cause chronic impairments in cognition, perception, reality testing, and emotion among others (Bhugra, <xref ref-type="bibr" rid="B5">2005</xref>). Its underlying mechanism is still unclear, even though it is considered to involve structural and functional brain abnormalities (Ho et al., <xref ref-type="bibr" rid="B35">2003</xref>; Karlsgodt et al., <xref ref-type="bibr" rid="B37">2010</xref>; Oh et al., <xref ref-type="bibr" rid="B52">2015</xref>, <xref ref-type="bibr" rid="B53">2017</xref>). Schizophrenia is usually diagnosed on the basis of history and mental status examination by psychiatrists under the guidelines of specified diagnostic criteria. Given that the diagnostic process is mainly performed by assessment of visible symptoms and the criteria require long-term observation to evaluate, a standard diagnostic test is needed to reduce the possibility of misdiagnosis.</p>
<p>Nowadays, there is growing interest in the use of machine learning techniques for the diagnosis of diseases (Mirzaei et al., <xref ref-type="bibr" rid="B48">2016</xref>; Qureshi et al., <xref ref-type="bibr" rid="B58">2016</xref>). Especially psychiatric disorders, including schizophrenia, have been the focus of research on automatic diagnosis by machine learning techniques and neuroimaging data (Davatzikos et al., <xref ref-type="bibr" rid="B16">2005</xref>; Fan et al., <xref ref-type="bibr" rid="B22">2008</xref>; Nieuwenhuis et al., <xref ref-type="bibr" rid="B51">2012</xref>; Schnack et al., <xref ref-type="bibr" rid="B64">2014</xref>). Previous studies have mainly utilized unimodal data acquired by structural magnetic resonance imaging (MRI), resting state functional MRI, task related functional MRI, or diffusion MRI, although some studies have included multimodal neuroimaging data. Considering that patients with schizophrenia have both structural and functional abnormalities, multimodal image data can provide more information, enhancing the accuracy of diagnosis. To wit, multimodal data can provide information for classification that is unavailable when using unimodal data. However, it is still unclear whether multimodal data can boost accuracy effectively as previous multimodal classification studies included relatively small samples with less than 35 participants in each group (Du et al., <xref ref-type="bibr" rid="B20">2012</xref>; Ota et al., <xref ref-type="bibr" rid="B54">2013</xref>; Sui et al., <xref ref-type="bibr" rid="B70">2013</xref>).</p>
<p>In this study, we classified patients with schizophrenia and healthy controls using both anatomical (sMRI) and resting state functional (rs-fMRI) features. To date, few studies have used combined sMRI and rs-fMRI features to discriminate patients with schizophrenia from healthy controls (Silva et al., <xref ref-type="bibr" rid="B67">2014</xref>). In addition, previous studies combining sMRI and fMRI utilized only a few parameters such as independent component analysis (ICA) features and gray matter densities. Considering the accuracy was under 95% in these studies, the classifier may be ameliorated with use of more parameters such as global connectivity, surface area, and curvature features among others. Although there exist many previous studies using ICA for extracting the features for classification, however, to the best of our knowledge, the group ICA method used in the present study has never been used before for this purpose.</p>
<p>To utilize multiple modalities, they should be integrated elaborately and the calculation of the importance of each feature type is crucial for high accuracy acquisition. Concurrent with this, we applied and proposed a simple and hybrid weighted feature concatenation method. The method is simple and robust, and can automatically calculate the weight of each feature type. In addition, we compared the accuracy of five types of classifier [linear and non-linear (radial basis function) support vector machine (SVM), linear extreme learning machine (ELM), linear discriminant analysis (LDA), and random forest bagged tree classifier] by applying multimodal brain features and the proposed feature concatenation method. We hypothesized that the combination of whole brain sMRI and rs-fMRI features would be superior to the utilization of unimodal data, and the accuracy would be sufficient for use in clinical practice.</p>
<p>The remainder of this paper is organized as follows: the materials and methods section provides information on the dataset, subject selection, preprocessing of the s/fMRI data, feature extraction, structural measures, global connectivity, group ICA, the hybrid weighted feature concatenation method, the introduction of the classification algorithms, and permutation testing for assessing the significance of the results. The results section presents the comparative results of both the unimodal and multimodal binary classifiers using the ELM, SVM-L, SVM-RBF, LDA, and Random forest classifiers. The discussion section includes the commentary on the results, comparison with previous research, and the clinical significance of the results. The conclusion section includes the limitations of the present study and concludes the paper.</p>
</sec>
<sec sec-type="materials and methods" id="s2">
<title>Materials and methods</title>
<sec>
<title>Subjects</title>
<p>We used 72 subjects each from the normal control and schizophrenia group from the Center for Biomedical Research Excellence (COBRE) dataset (Calhoun et al., <xref ref-type="bibr" rid="B9">2011</xref>; Hanlon et al., <xref ref-type="bibr" rid="B34">2011</xref>; Mayer et al., <xref ref-type="bibr" rid="B47">2013</xref>; Stephen et al., <xref ref-type="bibr" rid="B69">2013</xref>). Age ranged from 18 to 65 years in each group. Information on this dataset is available at <ext-link ext-link-type="uri" xlink:href="http://cobre.mrn.org/">http://cobre.mrn.org/</ext-link>. A structured clinical interview based on DSM-IV was used by trained clinical psychiatrists for the diagnosis of patients with Schizophrenia in this dataset. For making a balanced study design, we used only 72 subjects from each subgroup of the COBRE dataset.</p>
<p>All the subjects were screened and excluded if they had a history of substance abuse or dependence present in the 12 months prior to the scanning date. In addition, the patients were also screened for history of neurological disorders, intellectually disabled, and severe head trauma with a loss of consciousness for more than 5 min. COBRE is a publically available dataset distributed with &#x0201C;Creative Commons License: Attribution&#x02014;Non-Commercial&#x0201D; and written informed consent was obtained from all subjects in accordance with the institutional review board (IRB) protocols of the University of New Mexico (UNM) (Cabral et al., <xref ref-type="bibr" rid="B7">2016</xref>). Table <xref ref-type="table" rid="T1">1</xref> shows the demographic information of the subjects.</p>
<table-wrap position="float" id="T1">
<label>Table 1</label>
<caption><p>Demographic information of subjects from COBRE dataset.</p></caption>
<table frame="hsides" rules="groups">
<thead><tr>
<th valign="top" align="left"><bold>Groups</bold></th>
<th valign="top" align="center"><bold>Normal control</bold></th>
<th valign="top" align="center"><bold>Schizophrenia</bold></th>
</tr>
</thead>
<tbody>
<tr>
<td valign="top" align="left">No of Subjects</td>
<td valign="top" align="center">72</td>
<td valign="top" align="center">72</td>
</tr>
<tr>
<td valign="top" align="left">Age (mean &#x000B1; STD)</td>
<td valign="top" align="center">35.875 &#x000B1; 11.74</td>
<td valign="top" align="center">38.167 &#x000B1; 13.894</td>
</tr>
<tr>
<td valign="top" align="left">Sex (Male/Female)</td>
<td valign="top" align="center">58/14</td>
<td valign="top" align="center">52/20</td>
</tr>
<tr>
<td valign="top" align="left">Handedness (Left/Right)</td>
<td valign="top" align="center">10/62</td>
<td valign="top" align="center">3/69</td>
</tr>
<tr>
<td valign="top" align="left">Age of onsets (years)</td>
<td valign="top" align="center">&#x02013;</td>
<td valign="top" align="center">21.17 &#x000B1; 7.51</td>
</tr>
<tr>
<td valign="top" align="left">Illness duration (years)</td>
<td valign="top" align="center">&#x02013;</td>
<td valign="top" align="center">9.03 &#x000B1; 9.88</td>
</tr>
<tr>
<td valign="top" align="left">PANSS positive</td>
<td valign="top" align="center">&#x02013;</td>
<td valign="top" align="center">14.96 &#x000B1; 4.83</td>
</tr>
<tr>
<td valign="top" align="left">PANSS negative</td>
<td valign="top" align="center">&#x02013;</td>
<td valign="top" align="center">14.53 &#x000B1; 4.83</td>
</tr>
<tr>
<td valign="top" align="left">PANSS general</td>
<td valign="top" align="center">&#x02013;</td>
<td valign="top" align="center">29.22 &#x000B1; 8.34</td>
</tr>
<tr>
<td valign="top" align="left">PANSS total</td>
<td valign="top" align="center">&#x02013;</td>
<td valign="top" align="center">58.71 &#x000B1; 13.75</td>
</tr>
</tbody>
</table>
<table-wrap-foot>
<p><italic>STD, Standard deviation; PANSS, positive and negative syndrome scale</italic>.</p>
</table-wrap-foot>
</table-wrap>
</sec>
<sec>
<title>Dataset and preprocessing</title>
<sec>
<title>Structural data acquisition</title>
<p>All the participants were scanned with a Siemens TIM 3.0-Tesla scanner. A multi-echo MPRAGE (MEMPR) sequence was used with the following parameters: TR/TE/TI &#x0003D; 2530/[1.64, 3.5, 5.36, 7.22, 9.08]/900 ms, flip angle &#x0003D; 7&#x000B0;, FOV &#x0003D; 256 &#x000D7; 256 mm, Slab thickness &#x0003D; 176 mm, Matrix &#x0003D; 256 &#x000D7; 256 &#x000D7; 176, Voxel size &#x0003D; 1 &#x000D7; 1 &#x000D7; 1 mm, Number of echoes &#x0003D; 5, Pixel bandwidth &#x0003D; 650 Hz, Total scan time &#x0003D; 6 min. Using 5 echoes, the TR, TI, and time to encode partitions for the MEMPR are similar to that of a conventional MPRAGE, resulting in similar GM/WM/CSF contrast.</p>
</sec>
<sec>
<title>Structural data preprocessing</title>
<p>Cortical reconstruction and volumetric segmentation were performed with the FreeSurfer v. 5.3.0 image analysis suite, which is documented and freely available for download online at <ext-link ext-link-type="uri" xlink:href="http://surfer.nmr.mgh.harvard.edu/">http://surfer.nmr.mgh.harvard.edu/</ext-link>. The technical details of these procedures have been described previously (Dale and Sereno, <xref ref-type="bibr" rid="B15">1993</xref>; Dale, <xref ref-type="bibr" rid="B14">1999</xref>; Fischl et al., <xref ref-type="bibr" rid="B29">1999a</xref>,<xref ref-type="bibr" rid="B30">b</xref>, <xref ref-type="bibr" rid="B26">2001</xref>, <xref ref-type="bibr" rid="B27">2002</xref>, <xref ref-type="bibr" rid="B28">2004</xref>; Fischl and Dale, <xref ref-type="bibr" rid="B25">2000</xref>; S&#x000E9;gonne et al., <xref ref-type="bibr" rid="B65">2004</xref>; Han et al., <xref ref-type="bibr" rid="B33">2006</xref>; Reuter et al., <xref ref-type="bibr" rid="B60">2012</xref>). Briefly, the preprocessing procedure includes motion correction of volumetric T1-weighted images, removal of non-brain tissue using a hybrid watershed/surface deformation procedure (S&#x000E9;gonne et al., <xref ref-type="bibr" rid="B65">2004</xref>), and automated Talairach transformation. Moreover, the procedure includes the segmentation of subcortical white matter and deep gray matter volumetric structures including the hippocampus, amygdala, caudate nucleus, putamen, ventricles (Fischl et al., <xref ref-type="bibr" rid="B27">2002</xref>, <xref ref-type="bibr" rid="B28">2004</xref>); intensity normalization, tessellation of the gray-white matter boundary, and automated topology correction (Fischl et al., <xref ref-type="bibr" rid="B26">2001</xref>; S&#x000E9;gonne et al., <xref ref-type="bibr" rid="B66">2007</xref>). In addition, surface deformation following intensity gradients was performed to optimally place the gray/white and gray/cerebrospinal fluid (CSF) borders at the location where the greatest shift in intensity defines the transition to the other tissue class (Dale and Sereno, <xref ref-type="bibr" rid="B15">1993</xref>; Dale, <xref ref-type="bibr" rid="B14">1999</xref>; Fischl and Dale, <xref ref-type="bibr" rid="B25">2000</xref>). Once the cortical models are complete, several deformable procedures can be performed for further data processing and analysis. These procedures include surface inflation (Fischl et al., <xref ref-type="bibr" rid="B29">1999a</xref>)and registration to a spherical atlas that is based on individual cortical folding patterns to match the cortical geometry across subjects (Fischl et al., <xref ref-type="bibr" rid="B30">1999b</xref>). Moreover, parcellation of the cerebral cortex into units with respect to gyral and sulcal structure (Fischl et al., <xref ref-type="bibr" rid="B28">2004</xref>; Desikan et al., <xref ref-type="bibr" rid="B18">2006</xref>) and creation of a variety of surface based data including maps of curvature and sulcal depth are part of the procedure. This method uses both intensity and continuity information from the entire three-dimensional MR volume in segmentation and deformation procedures to produce representations of cortical thickness, calculated as the closest distance from the gray/white boundary to the gray/CSF boundary at each vertex on the tessellated surface (Fischl and Dale, <xref ref-type="bibr" rid="B25">2000</xref>). The maps are created using spatial intensity gradients across tissue classes rather than simply relying on absolute signal intensity. The maps produced are not restricted to the voxel resolution of the original data, and are thus capable of detecting submillimeter differences between groups. Procedures for the measurement of cortical thickness have been validated against histological analysis (Rosas et al., <xref ref-type="bibr" rid="B61">2002</xref>) and manual measurements (Kuperberg et al., <xref ref-type="bibr" rid="B40">2003</xref>; Salat et al., <xref ref-type="bibr" rid="B63">2004</xref>). FreeSurfer morphometric procedures have shown good test-retest reliability across scanner manufacturers and across field strengths (Han et al., <xref ref-type="bibr" rid="B33">2006</xref>; Reuter et al., <xref ref-type="bibr" rid="B60">2012</xref>). A cortical surface-based Desikan-Killiany-Tourville (DKT) atlas (Klein and Tourville, <xref ref-type="bibr" rid="B38">2012</xref>) was mapped to a sphere aligning the cortical folding patterns, which provided accurate matching of the morphologically homologous cortical locations across subjects. For each of the DKT31 protocol-based segments, FreeSurfer calculated 9 different measures, including the number of vertices, surface area, gray matter volume, mean cortical thickness, cortical thickness standard deviation, mean cortical curvature, Gaussian cortical curvature, cortical folding index, and cortical curvature indices (Colby et al., <xref ref-type="bibr" rid="B11">2012</xref>). For the subcortical regions, FreeSurfer calculated the area and volume of the whole segment, white matter volume, and intensity and overall volume of the whole brain divisions, including of the cerebrospinal fluid (CSF), intracranial volume (ICV), gray matter (GM), and white matter (WM). Two of the selected measures are the most common features in structural studies (Arbabshirani et al., <xref ref-type="bibr" rid="B2">2017</xref>). The surface area was calculated by computing the area of every triangle in a standardized spherical surface tessellation. The local curvature was computed using the registration surface based on the folding patterns (Qureshi et al., <xref ref-type="bibr" rid="B58">2016</xref>, <xref ref-type="bibr" rid="B59">2017b</xref>).</p>
</sec>
<sec>
<title>Cortical and subcortical features</title>
<p>We used measures including the mean cortical thickness, cortical thickness standard deviation, surface area, volume, mean curvature, white matter volume, subcortical segment volume, subcortical intensity, and overall brain volume and intensity as the structural features. All features were acquired as morphological statistics for each subject during the FreeSurfer based preprocessing of the structural MRI data. In addition, after the preprocessing, FreeSurfer&#x00027;s QA Tools were used for the detection and removal of outliers and negative features (Qureshi et al., <xref ref-type="bibr" rid="B59">2017b</xref>).</p>
</sec>
<sec>
<title>Functional data acquisition</title>
<p>Resting state functional MRI data were collected with single-shot full k-space echo-planar imaging (EPI) with ramp sampling correction using the intercomissural line (AC-PC) as a reference (TR: 2 s, TE: 29 ms, matrix size: 64 &#x000D7; 64, 32 slices, voxel size: 3 &#x000D7; 3 &#x000D7; 4 mm<sup>3</sup>). During the acquisition process, all subjects were instructed to keep their eyes open and stare at the fixation cross.</p>
</sec>
<sec>
<title>Functional data preprocessing</title>
<p>Preprocessing of functional MRI data was based on Analysis of Functional Neuroimages AFNI software; <ext-link ext-link-type="uri" xlink:href="http://afni.nimh.nih.gov/afni/">http://afni.nimh.nih.gov/afni/</ext-link> (Cox, <xref ref-type="bibr" rid="B13">1996</xref>). Every single EPI volume was coregistered to the corresponding anatomical image of the subject and mapped to the Talairach coordinates space with the TT_N27&#x0002B;tlrc template. We excluded the first six images from each EPI volume to achieve the MR steady state. In addition, slice-timing correction was performed. We censored and excluded time-points based on the number of outliers and head motion magnitude. The same number of slices was excluded for all subjects. Slice alignment was applied using the local Pearson&#x00027;s correlation (LPC) cost function. Correction of head motion along with averaging of EPI volumes was performed to obtain a mean functional image. Each EPI volume underwent linear multiple regression to regress the motion derivatives and effects of the white matter and cerebrospinal fluid. Spatial smoothing was performed using a Gaussian kernel with a blur size of 6-mm full width at half maximum (FWHM). A polynomial detrending was applied (Qureshi et al., <xref ref-type="bibr" rid="B59">2017b</xref>).</p>
</sec>
<sec>
<title>Global connectivity features</title>
<p>A global connectivity measure was used to calculate the average brain-wise correlation coefficients (GCOR) of all the possible combinations of voxel time series. The GCOR estimation of cortical regions is a computationally expensive process. It involves the calculation of M (M&#x02212;1)/2 correlation estimates for an M voxel volume (Saad et al., <xref ref-type="bibr" rid="B62">2013</xref>). AFNI simplifies the process by taking reduced time series of each voxel and scaling it by its Euclidean norm. In addition, it averages the scaled time series over the whole brain mask and finally the length <italic>l</italic><sub>2</sub> &#x02212; <italic>norm</italic> of this averaged series represents the GCOR. We used the AFNI program 3dTcorrMap to get the global functional connectivity maps. This involved the calculation of correlation (Pearson&#x00027;s <italic>r</italic>) of the residual time series in each voxel with every other voxel in their brain mask and recording the mean correlation back in the voxel. These connectedness values were further transformed with Fisher&#x00027;s z-transformation to yield normally distributed measures (Gotts et al., <xref ref-type="bibr" rid="B32">2012</xref>). We used the DKD_ Desai_PM Atlas (Desikan et al., <xref ref-type="bibr" rid="B18">2006</xref>) of the AFNI package to acquire the ROI measures from 102 cortical and subcortical regions across the whole brain. Figure <xref ref-type="fig" rid="F1">1</xref> depicts a typical global functional connectivity map for one of the control subjects from the COBRE dataset.</p>
<fig id="F1" position="float">
<label>Figure 1</label>
<caption><p>An example of a typical global functional connectivity map for one of the control subject of COBRE dataset. First row depicts the coronal, second row depicts the axial and the third row depicts the sagittal view.</p></caption>
<graphic xlink:href="fninf-11-00059-g0001.tif"/>
</fig>
</sec>
<sec>
<title>Group independent component analysis features</title>
<p>Resting state functional MRI data were preprocessed again with FSL (FMRIB Software Library, <ext-link ext-link-type="uri" xlink:href="http://www.fmrib.ox.ac.uk/fsl">www.fmrib.ox.ac.uk/fsl</ext-link>) 6.0 for acquiring the group independent component analysis (gICA) based connectivity measures. FSL Multivariate Exploratory Linear Optimized Decomposition into Independent Components (MELODIC) version 3.14 was utilized to perform a single-session ICA. The number of independent components was set as 30, as a model with an order higher than 20 components is usually required for the detection of some components (e.g., S1, S2, striatum), whereas a higher order model shows a decrease in ICA repeatability (Abou-Elseoud et al., <xref ref-type="bibr" rid="B1">2010</xref>). We used variance normalization and thresholded the independent component maps with an alternative hypothesis test that was based on the fitting of a Gaussian/gamma mixture model to the distributions of the voxel intensities within the spatial maps and controlling the local false-discovery rate at <italic>p</italic> &#x0003C; 0.5 (Smith et al., <xref ref-type="bibr" rid="B68">2004</xref>; Beckmann et al., <xref ref-type="bibr" rid="B4">2005</xref>). Among the 30 independent components, 11 were discarded as noise and/or artifacts upon visual inspection by an experienced clinical psychiatrist. Finally, 19 components were identified as functional networks or subnetworks. Similar networks were used in our previous study (Oh et al., <xref ref-type="bibr" rid="B53">2017</xref>). Figure <xref ref-type="fig" rid="F2">2</xref> depicts some of the 19 selected components; they represent the well-known resting state functional networks: the default mode network, sensorimotor network, visual network, and frontal network.</p>
<fig id="F2" position="float">
<label>Figure 2</label>
<caption><p>Well-known resting state functional networks acquired from the selected independent component of group ICA result. First image in each network figure depicts the sagittal, second depicts the coronal and third depicts the axial view. The bright yellow color represents the IC&#x00027;s while the pale background colors depicts the MNI152 atlas ROI&#x00027;s. IC, Independent component; ICA, Independent component analysis; MNI, Montreal neuroimaging institute.</p></caption>
<graphic xlink:href="fninf-11-00059-g0002.tif"/>
</fig>
<p>Using these components, we applied the FSL dual regression technique (Filippini et al., <xref ref-type="bibr" rid="B24">2009</xref>; Littow et al., <xref ref-type="bibr" rid="B44">2010</xref>; Veer et al., <xref ref-type="bibr" rid="B72">2010</xref>), to acquire subject-specific spatial maps. During this process, we utilized the group spatial maps in a linear model that was fit against the separate fMRI datasets and then acquired the time-course matrices depicting the temporal dynamics for each subject and component. After that, we could acquire the subject-specific spatial maps with the time-course matrices. Next, we calculated the correlation coefficient between the selected 19 components. It resulted in a 19 &#x000D7; 19 correlation based connectivity matrix. We isolated the upper diagonal elements of the connectivity matrix and after vectorizing them, we used them as the gICA features for each subject. Figure <xref ref-type="fig" rid="F3">3</xref> depicts a typical ICA connectivity matrix for one of the control subjects from the COBRE dataset.</p>
<fig id="F3" position="float">
<label>Figure 3</label>
<caption><p>An example of a typical group independent component analysis based connectivity matrix for one of the control subject of COBRE dataset.</p></caption>
<graphic xlink:href="fninf-11-00059-g0003.tif"/>
</fig>
<p>Finally, we gathered four types of functional connectivity based features, namely: cortical ROI-based global connectivity, subcortical ROI-based global connectivity, whole brain-based global connectivity, and gICA connectivity features. The first three types of features were based on the DKD_ Desai_PM Atlas ROI&#x00027;s (Desikan et al., <xref ref-type="bibr" rid="B18">2006</xref>) and all of them were acquired by considering the average measure of global connectivity of the atlas ROIs.</p>
<p>Table <xref ref-type="table" rid="T2">2</xref> presents a brief overview of the structural and functional measures and number of features used in this study.</p>
<table-wrap position="float" id="T2">
<label>Table 2</label>
<caption><p>Summary of the features used in this study.</p></caption>
<table frame="hsides" rules="groups">
<thead><tr>
<th valign="top" align="left"><bold>Label</bold></th>
<th valign="top" align="center" colspan="2" style="border-bottom: thin solid #000000;"><bold>Structural</bold></th>
<th valign="top" align="center"><bold>Label</bold></th>
<th valign="top" align="center" colspan="2" style="border-bottom: thin solid #000000;"><bold>Functional</bold></th>
<th valign="top" align="center" colspan="2" style="border-bottom: thin solid #000000;"><bold>Multi-measure and Multi-modal</bold></th>
</tr>
<tr>
<th/>
<th valign="top" align="left"><bold>Measure</bold></th>
<th valign="top" align="center"><bold>Count</bold></th>
<th/>
<th valign="top" align="left"><bold>Measure</bold></th>
<th valign="top" align="center"><bold>Count</bold></th>
<th valign="top" align="left"><bold>Measure</bold></th>
<th valign="top" align="center"><bold>Count</bold></th>
</tr>
</thead>
<tbody>
<tr>
<td valign="top" align="left" colspan="8" style="background-color:#bbbdc0"><bold>CORTICAL FEATURES</bold></td>
</tr>
<tr>
<td valign="top" align="left">M1</td>
<td valign="top" align="left">CT</td>
<td valign="top" align="center">64</td>
<td valign="top" align="left">M7</td>
<td valign="top" align="left">Global connectivity</td>
<td valign="top" align="center">67</td>
<td valign="top" align="left">All cortical</td>
<td valign="top" align="center">449</td>
</tr>
<tr>
<td valign="top" align="left">M2</td>
<td valign="top" align="left">CT STD</td>
<td valign="top" align="center">62</td>
<td/>
<td/>
<td/>
<td/>
<td/>
</tr>
<tr>
<td valign="top" align="left">M3</td>
<td valign="top" align="left">Surface Area</td>
<td valign="top" align="center">64</td>
<td/>
<td/>
<td/>
<td/>
<td/>
</tr>
<tr>
<td valign="top" align="left">M4</td>
<td valign="top" align="left">Volume</td>
<td valign="top" align="center">62</td>
<td/>
<td/>
<td/>
<td/>
<td/>
</tr>
<tr>
<td valign="top" align="left">M5</td>
<td valign="top" align="left">Mean Curvature</td>
<td valign="top" align="center">62</td>
<td/>
<td/>
<td/>
<td/>
<td/>
</tr>
<tr>
<td valign="top" align="left">M6</td>
<td valign="top" align="left">WM Volume</td>
<td valign="top" align="center">68</td>
<td/>
<td/>
<td/>
<td/>
<td/>
</tr>
<tr style="border-top: thin solid #000000;">
<td valign="top" align="left">Total</td>
<td valign="top" align="center" colspan="2">382</td>
<td valign="top" align="center" colspan="3">67</td>
<td valign="top" align="center" colspan="2">449</td>
</tr>
<tr>
<td valign="top" align="left" colspan="8" style="background-color:#bbbdc0"><bold>SUBCORTICAL FEATURES</bold></td>
</tr>
<tr>
<td valign="top" align="left">M8</td>
<td valign="top" align="left">Volume</td>
<td valign="top" align="center">26</td>
<td valign="top" align="left">M10</td>
<td valign="top" align="left">Global connectivity</td>
<td valign="top" align="center">24</td>
<td valign="top" align="left">All subcortical</td>
<td valign="top" align="center">76</td>
</tr>
<tr>
<td valign="top" align="left">M9</td>
<td valign="top" align="left">Intensity</td>
<td valign="top" align="center">26</td>
<td/>
<td/>
<td/>
<td/>
<td/>
</tr>
<tr style="border-top: thin solid #000000;">
<td valign="top" align="left">Total</td>
<td valign="top" align="center" colspan="2">52</td>
<td valign="top" align="center" colspan="3">24</td>
<td valign="top" align="center" colspan="2">76</td>
</tr>
<tr>
<td valign="top" align="left" colspan="8" style="background-color:#bbbdc0"><bold>WHOLE BRAIN FEATURES</bold></td>
</tr>
<tr>
<td valign="top" align="left">M11</td>
<td valign="top" align="left">Volume</td>
<td valign="top" align="center">27</td>
<td valign="top" align="left">M10</td>
<td valign="top" align="left">Global connectivity</td>
<td valign="top" align="center">11</td>
<td valign="top" align="left">All whole brain</td>
<td valign="top" align="center">223</td>
</tr>
<tr>
<td valign="top" align="left">M9</td>
<td valign="top" align="left">Intensity</td>
<td valign="top" align="center">14</td>
<td valign="top" align="left">M12</td>
<td valign="top" align="left">gICA connectivity</td>
<td valign="top" align="center">171</td>
<td/>
<td/>
</tr>
<tr style="border-top: thin solid #000000;">
<td valign="top" align="left">Total</td>
<td valign="top" align="center" colspan="2">41</td>
<td valign="top" align="center" colspan="3">182</td>
<td valign="top" align="center" colspan="2">223</td>
</tr>
<tr style="border-top: thin solid #000000;">
<td valign="top" align="left">Grand Total</td>
<td valign="top" align="center" colspan="7">748</td>
</tr>
</tbody>
</table>
<table-wrap-foot>
<p><italic>CT, Cortical thickness; STD, standard deviation; WM, white matter; gICA, group independent component analysis</italic>.</p>
</table-wrap-foot>
</table-wrap>
</sec>
</sec>
<sec>
<title>Hybrid weighted feature concatenation</title>
<p>Combining features of multiple modalities is a very effective approach for boosting the performance of a machine learning setup. It has been used for machine learning in many research domains, including computer vision for face recognition (Qureshi, <xref ref-type="bibr" rid="B56">2013</xref>), robotics (Naveed, <xref ref-type="bibr" rid="B50">2014</xref>), and in neuroimaging classification (Calhoun and Sui, <xref ref-type="bibr" rid="B8">2016</xref>). We investigated a common feature combination method of simple concatenation in this study, in contrast to the proposed simple and hybrid weighted feature concatenation (WC) method. The proposed method enhanced accuracy while maintaining the number of features similar to that of simple concatenation.</p>
<p>In the proposed method, we calculated the weights of each of 12 (nine structural and three functional) measures <italic>M</italic><sub>1</sub>, <italic>M</italic><sub>2</sub>, &#x02026;, <italic>M</italic><sub>12</sub> of the dataset. We want to have the important features to get a high weight <italic>W</italic><sub><italic>i</italic></sub> such that (0 &#x0003C;&#x0003D; <italic>W</italic><sub>i</sub> &#x0003C;&#x0003D; 1), and hence we choose <italic>W</italic><sub>i</sub> &#x0003D; 1 &#x02212; <italic>A</italic><sub><italic>max</italic></sub> &#x0002B; <italic>A</italic><sub>i</sub>. Briefly, for calculating the weights <italic>W</italic><sub>1</sub>, <italic>W</italic><sub>2</sub>, &#x02026;, <italic>W</italic><sub>12</sub>, we first calculated the accuracy of each of the12 measures and then subtracted the individual accuracy <italic>A</italic><sub>1</sub>, <italic>A</italic><sub>2</sub>, &#x02026;, <italic>A</italic><sub>12</sub> from the highest accuracy <inline-formula><mml:math id="M1"><mml:msub><mml:mrow><mml:mstyle class="text"><mml:mtext mathvariant="bold">A</mml:mtext></mml:mstyle></mml:mrow><mml:mrow><mml:mstyle class="text"><mml:mtext mathvariant="bold">max</mml:mtext></mml:mstyle></mml:mrow></mml:msub><mml:mo>=</mml:mo><mml:msup><mml:mrow><mml:mrow><mml:mo>[</mml:mo><mml:mrow><mml:msub><mml:mrow><mml:mstyle class="text"><mml:mtext class="textit" mathvariant="italic">A</mml:mtext></mml:mstyle></mml:mrow><mml:mrow><mml:mstyle class="text"><mml:mtext class="textit" mathvariant="italic">max</mml:mtext></mml:mstyle></mml:mrow></mml:msub></mml:mrow><mml:mo>]</mml:mo></mml:mrow></mml:mrow><mml:mrow><mml:mstyle class="text"><mml:mtext mathvariant="bold">T</mml:mtext></mml:mstyle></mml:mrow></mml:msup></mml:math></inline-formula> value among them. Finally, we subtracted the resulting value <italic>R</italic><sub>1</sub>, <italic>R</italic><sub>2</sub>, &#x02026;, <italic>R</italic><sub>12</sub> of each measure from vector <inline-formula><mml:math id="M2"><mml:mstyle class="text"><mml:mtext mathvariant="bold">1</mml:mtext></mml:mstyle><mml:mo>=</mml:mo><mml:mtext>&#x000A0;</mml:mtext><mml:msup><mml:mrow><mml:mrow><mml:mo>[</mml:mo><mml:mrow><mml:msub><mml:mrow><mml:mn>1</mml:mn></mml:mrow><mml:mrow><mml:mn>1</mml:mn></mml:mrow></mml:msub><mml:mo>,</mml:mo><mml:msub><mml:mrow><mml:mn>1</mml:mn></mml:mrow><mml:mrow><mml:mn>2</mml:mn></mml:mrow></mml:msub><mml:mo>,</mml:mo><mml:mtext>&#x000A0;</mml:mtext><mml:mo>.</mml:mo><mml:mo>.</mml:mo><mml:mo>,</mml:mo><mml:mtext>&#x000A0;</mml:mtext><mml:msub><mml:mrow><mml:mn>1</mml:mn></mml:mrow><mml:mrow><mml:mn>12</mml:mn></mml:mrow></mml:msub></mml:mrow><mml:mo>]</mml:mo></mml:mrow></mml:mrow><mml:mrow><mml:mstyle class="text"><mml:mtext class="textrm" mathvariant="normal">T</mml:mtext></mml:mstyle></mml:mrow></mml:msup></mml:math></inline-formula> (that corresponds to perfect classification) to acquire the weights. Mathematically,</p>
<disp-formula id="E1"><mml:math id="M3"><mml:mtable columnalign='left'><mml:mtr><mml:mtd><mml:mrow><mml:msup><mml:mrow><mml:mo stretchy='false'>[</mml:mo><mml:msub><mml:mi>R</mml:mi><mml:mn>1</mml:mn></mml:msub><mml:mo>,</mml:mo><mml:msub><mml:mi>R</mml:mi><mml:mn>2</mml:mn></mml:msub><mml:mo>,</mml:mo><mml:mo>&#x02026;</mml:mo><mml:mtext>&#x000A0;</mml:mtext><mml:mo>,</mml:mo><mml:mtext>&#x000A0;</mml:mtext><mml:msub><mml:mi>R</mml:mi><mml:mrow><mml:mn>12</mml:mn></mml:mrow></mml:msub><mml:mo stretchy='false'>]</mml:mo></mml:mrow><mml:mi>T</mml:mi></mml:msup><mml:mo>=</mml:mo><mml:msub><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>A</mml:mi></mml:mstyle><mml:mrow><mml:mstyle mathvariant='bold' mathsize='normal'><mml:mi>m</mml:mi><mml:mi>a</mml:mi><mml:mi>x</mml:mi></mml:mstyle><mml:mo>&#x02212;</mml:mo></mml:mrow></mml:msub><mml:mtext>&#x000A0;</mml:mtext><mml:msup><mml:mrow><mml:msub><mml:mo stretchy='false'>[</mml:mo><mml:mtext>&#x000A0;</mml:mtext></mml:msub><mml:msub><mml:mi>A</mml:mi><mml:mn>1</mml:mn></mml:msub><mml:mo>,</mml:mo><mml:msub><mml:mi>A</mml:mi><mml:mn>2</mml:mn></mml:msub><mml:mo>,</mml:mo><mml:mo>&#x02026;</mml:mo><mml:mtext>&#x000A0;</mml:mtext><mml:mo>,</mml:mo><mml:mtext>&#x000A0;</mml:mtext><mml:msub><mml:mi>A</mml:mi><mml:mrow><mml:mn>12</mml:mn></mml:mrow></mml:msub><mml:mo stretchy='false'>]</mml:mo></mml:mrow><mml:mi>T</mml:mi></mml:msup></mml:mrow></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mrow><mml:msup><mml:mrow><mml:mo stretchy='false'>[</mml:mo><mml:msub><mml:mi>W</mml:mi><mml:mn>1</mml:mn></mml:msub><mml:mo>,</mml:mo><mml:msub><mml:mi>W</mml:mi><mml:mn>2</mml:mn></mml:msub><mml:mo>,</mml:mo><mml:mtext>&#x000A0;</mml:mtext><mml:mo>&#x02026;</mml:mo><mml:mtext>&#x000A0;</mml:mtext><mml:mo>,</mml:mo><mml:msub><mml:mi>W</mml:mi><mml:mrow><mml:mn>12</mml:mn></mml:mrow></mml:msub><mml:mo stretchy='false'>]</mml:mo></mml:mrow><mml:mi>T</mml:mi></mml:msup><mml:mo>=</mml:mo><mml:mn>1</mml:mn><mml:mo>&#x02212;</mml:mo><mml:msup><mml:mrow><mml:mo stretchy='false'>[</mml:mo><mml:msub><mml:mi>R</mml:mi><mml:mn>1</mml:mn></mml:msub><mml:mo>,</mml:mo><mml:mtext>&#x000A0;</mml:mtext><mml:msub><mml:mi>R</mml:mi><mml:mn>2</mml:mn></mml:msub><mml:mo>,</mml:mo><mml:mo>&#x02026;</mml:mo><mml:mtext>&#x000A0;</mml:mtext><mml:mo>,</mml:mo><mml:msub><mml:mi>R</mml:mi><mml:mrow><mml:mn>12</mml:mn></mml:mrow></mml:msub><mml:mo stretchy='false'>]</mml:mo></mml:mrow><mml:mi>T</mml:mi></mml:msup></mml:mrow></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula>
<p>After acquiring the weights [<italic>W</italic><sub>1</sub>, <italic>W</italic><sub>2</sub>, &#x02026; , <italic>W</italic><sub>12</sub>]<sup><italic>T</italic></sup>, in the simple weighted concatenation, the weights of each feature type were multiplied with the corresponding measure data and were concatenated.</p>
<disp-formula id="E2"><mml:math id="M4"><mml:mtable columnalign='left'><mml:mtr><mml:mtd><mml:mi>S</mml:mi><mml:mi>i</mml:mi><mml:mi>m</mml:mi><mml:mi>p</mml:mi><mml:mi>l</mml:mi><mml:mi>e</mml:mi><mml:mtext>&#x000A0;</mml:mtext><mml:mi>W</mml:mi><mml:mi>e</mml:mi><mml:mi>i</mml:mi><mml:mi>g</mml:mi><mml:mi>h</mml:mi><mml:mi>t</mml:mi><mml:mi>e</mml:mi><mml:mi>d</mml:mi><mml:mtext>&#x000A0;</mml:mtext><mml:mi>C</mml:mi><mml:mi>o</mml:mi><mml:mi>n</mml:mi><mml:mi>a</mml:mi><mml:mi>t</mml:mi><mml:mi>e</mml:mi><mml:mi>n</mml:mi><mml:mi>a</mml:mi><mml:mi>t</mml:mi><mml:mi>i</mml:mi><mml:mi>o</mml:mi><mml:mi>n</mml:mi><mml:mo>=</mml:mo><mml:mo stretchy='false'>[</mml:mo><mml:msub><mml:mi>W</mml:mi><mml:mn>1</mml:mn></mml:msub><mml:mtext>&#x000A0;</mml:mtext><mml:mo>&#x02217;</mml:mo><mml:mtext>&#x000A0;</mml:mtext><mml:msub><mml:mi>M</mml:mi><mml:mn>1</mml:mn></mml:msub><mml:mo>&#x0007C;</mml:mo><mml:mo>&#x0007C;</mml:mo><mml:mtext>&#x000A0;</mml:mtext><mml:msub><mml:mi>W</mml:mi><mml:mn>2</mml:mn></mml:msub><mml:mo>&#x02217;</mml:mo><mml:msub><mml:mi>M</mml:mi><mml:mn>2</mml:mn></mml:msub><mml:mtext>&#x000A0;</mml:mtext><mml:mo>&#x0007C;</mml:mo><mml:mo>&#x0007C;</mml:mo><mml:mtext>&#x000A0;</mml:mtext><mml:mo>,</mml:mo><mml:mo>&#x02026;</mml:mo><mml:mtext>&#x000A0;</mml:mtext><mml:mo>,</mml:mo><mml:mtext>&#x000A0;</mml:mtext></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mtext>&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;</mml:mtext><mml:mo>&#x0007C;</mml:mo><mml:mo>&#x0007C;</mml:mo><mml:mtext>&#x000A0;</mml:mtext><mml:msub><mml:mi>W</mml:mi><mml:mrow><mml:mn>12</mml:mn></mml:mrow></mml:msub><mml:mtext>&#x000A0;&#x000A0;</mml:mtext><mml:mo>&#x02217;</mml:mo><mml:mtext>&#x000A0;&#x000A0;</mml:mtext><mml:msub><mml:mi>M</mml:mi><mml:mrow><mml:mn>12</mml:mn></mml:mrow></mml:msub><mml:mo stretchy='false'>]</mml:mo></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula>
<p>Hybrid weighted concatenation was performed by multiplying the corresponding weight with its measure data and dividing the long-concatenated matrix with the sum of the corresponding weights. This step was performed separately for the seven cortical and four subcortical data types regardless of the modality. This division of cortical and subcortical measures was based on the anatomical ROIs acquired from the DKT and DKD_ Desai_PM atlases.</p>
<disp-formula id="E3"><mml:math id="M5"><mml:mtable columnalign='left'><mml:mtr><mml:mtd><mml:mtext>&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;</mml:mtext><mml:mi>C</mml:mi><mml:mi>o</mml:mi><mml:mi>r</mml:mi><mml:mi>t</mml:mi><mml:mi>i</mml:mi><mml:mi>c</mml:mi><mml:mi>a</mml:mi><mml:mi>l</mml:mi><mml:mtext>&#x000A0;</mml:mtext><mml:mi>W</mml:mi><mml:mi>C</mml:mi><mml:mtext>&#x000A0;</mml:mtext><mml:mo>=</mml:mo><mml:mo stretchy='false'>[</mml:mo><mml:msub><mml:mi>W</mml:mi><mml:mn>1</mml:mn></mml:msub><mml:mo>&#x02217;</mml:mo><mml:msub><mml:mi>M</mml:mi><mml:mn>1</mml:mn></mml:msub><mml:mo>&#x0007C;</mml:mo><mml:mo>&#x0007C;</mml:mo><mml:mtext>&#x000A0;</mml:mtext><mml:msub><mml:mi>W</mml:mi><mml:mn>2</mml:mn></mml:msub><mml:mo>&#x02217;</mml:mo><mml:msub><mml:mi>M</mml:mi><mml:mn>2</mml:mn></mml:msub><mml:mtext>&#x000A0;</mml:mtext><mml:mo>&#x0007C;</mml:mo><mml:mo>&#x0007C;</mml:mo><mml:mtext>&#x000A0;</mml:mtext><mml:mo>,</mml:mo><mml:mn>...</mml:mn><mml:mtext>&#x000A0;</mml:mtext><mml:mo>,</mml:mo><mml:mtext>&#x000A0;</mml:mtext><mml:mo>&#x0007C;</mml:mo><mml:mo>&#x0007C;</mml:mo></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mtext>&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;</mml:mtext><mml:msub><mml:mi>W</mml:mi><mml:mn>7</mml:mn></mml:msub><mml:mo>&#x02217;</mml:mo><mml:msub><mml:mi>M</mml:mi><mml:mn>7</mml:mn></mml:msub><mml:mo stretchy='false'>]</mml:mo><mml:mtext>&#x000A0;</mml:mtext><mml:mo>/</mml:mo><mml:mstyle displaystyle='true'><mml:munderover><mml:mo>&#x02211;</mml:mo><mml:mrow><mml:mi>i</mml:mi><mml:mo>=</mml:mo><mml:mn>1</mml:mn></mml:mrow><mml:mn>7</mml:mn></mml:munderover><mml:mrow><mml:msub><mml:mi>W</mml:mi><mml:mi>i</mml:mi></mml:msub></mml:mrow></mml:mstyle></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mi>S</mml:mi><mml:mi>u</mml:mi><mml:mi>b</mml:mi><mml:mi>c</mml:mi><mml:mi>o</mml:mi><mml:mi>r</mml:mi><mml:mi>t</mml:mi><mml:mi>i</mml:mi><mml:mi>c</mml:mi><mml:mi>a</mml:mi><mml:mi>l</mml:mi><mml:mtext>&#x000A0;</mml:mtext><mml:mi>W</mml:mi><mml:mi>C</mml:mi><mml:mtext>&#x000A0;</mml:mtext><mml:mo>=</mml:mo><mml:mo stretchy='false'>[</mml:mo><mml:msub><mml:mi>W</mml:mi><mml:mn>8</mml:mn></mml:msub><mml:mtext>&#x000A0;</mml:mtext><mml:mo>&#x02217;</mml:mo><mml:mtext>&#x000A0;</mml:mtext><mml:msub><mml:mi>M</mml:mi><mml:mn>8</mml:mn></mml:msub><mml:mo>&#x0007C;</mml:mo><mml:mo>&#x0007C;</mml:mo><mml:mtext>&#x000A0;</mml:mtext><mml:msub><mml:mi>W</mml:mi><mml:mn>9</mml:mn></mml:msub><mml:mtext>&#x000A0;</mml:mtext><mml:mo>&#x02217;</mml:mo><mml:msub><mml:mi>M</mml:mi><mml:mn>9</mml:mn></mml:msub><mml:mtext>&#x000A0;</mml:mtext><mml:mo>&#x0007C;</mml:mo><mml:mo>&#x0007C;</mml:mo><mml:mtext>&#x000A0;</mml:mtext><mml:mo>,</mml:mo><mml:mn>...</mml:mn><mml:mtext>&#x000A0;</mml:mtext><mml:mo>,</mml:mo><mml:mtext>&#x000A0;</mml:mtext><mml:mo>&#x0007C;</mml:mo><mml:mo>&#x0007C;</mml:mo><mml:mtext>&#x000A0;</mml:mtext></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mtext>&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;</mml:mtext><mml:msub><mml:mi>W</mml:mi><mml:mrow><mml:mn>11</mml:mn></mml:mrow></mml:msub><mml:mo>&#x02217;</mml:mo><mml:msub><mml:mi>M</mml:mi><mml:mrow><mml:mn>11</mml:mn></mml:mrow></mml:msub><mml:mo stretchy='false'>]</mml:mo><mml:mo>/</mml:mo><mml:mstyle displaystyle='true'><mml:munderover><mml:mo>&#x02211;</mml:mo><mml:mrow><mml:mi>i</mml:mi><mml:mo>=</mml:mo><mml:mn>8</mml:mn></mml:mrow><mml:mrow><mml:mn>11</mml:mn></mml:mrow></mml:munderover><mml:mrow><mml:msub><mml:mi>W</mml:mi><mml:mi>i</mml:mi></mml:msub></mml:mrow></mml:mstyle></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mtext>&#x000A0;&#x000A0;</mml:mtext><mml:mi>H</mml:mi><mml:mi>y</mml:mi><mml:mi>b</mml:mi><mml:mi>r</mml:mi><mml:mi>i</mml:mi><mml:mi>d</mml:mi><mml:mtext>&#x000A0;</mml:mtext><mml:mi>W</mml:mi><mml:mi>C</mml:mi><mml:mtext>&#x000A0;&#x000A0;</mml:mtext><mml:mo>=</mml:mo><mml:mtext>&#x000A0;</mml:mtext><mml:mo stretchy='false'>[</mml:mo><mml:mi>C</mml:mi><mml:mi>o</mml:mi><mml:mi>r</mml:mi><mml:mi>t</mml:mi><mml:mi>i</mml:mi><mml:mi>c</mml:mi><mml:mi>a</mml:mi><mml:mi>l</mml:mi><mml:mtext>&#x000A0;</mml:mtext><mml:mi>W</mml:mi><mml:mi>C</mml:mi><mml:mo>&#x0007C;</mml:mo><mml:mo>&#x0007C;</mml:mo><mml:mi>S</mml:mi><mml:mi>u</mml:mi><mml:mi>b</mml:mi><mml:mi>c</mml:mi><mml:mi>o</mml:mi><mml:mi>r</mml:mi><mml:mi>t</mml:mi><mml:mi>i</mml:mi><mml:mi>c</mml:mi><mml:mi>a</mml:mi><mml:mi>l</mml:mi><mml:mi>W</mml:mi><mml:mi>C</mml:mi><mml:mo>&#x0007C;</mml:mo><mml:mo>&#x0007C;</mml:mo></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mtext>&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;</mml:mtext><mml:mi>g</mml:mi><mml:mi>I</mml:mi><mml:mi>C</mml:mi><mml:mi>A</mml:mi><mml:mtext>&#x000A0;</mml:mtext><mml:mi>f</mml:mi><mml:mi>e</mml:mi><mml:mi>a</mml:mi><mml:mi>t</mml:mi><mml:mi>u</mml:mi><mml:mi>r</mml:mi><mml:mi>e</mml:mi><mml:mi>s</mml:mi><mml:mo stretchy='false'>]</mml:mo></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula>
<p>Where the symbol &#x0201C;||&#x0201D; corresponds to the concatenation. These simple and hybrid weighted feature concatenation methods were robust and outperformed simple concatenation.</p>
</sec>
<sec>
<title>Classification</title>
<p>Five classifiers were used in this study, namely, ELM, linear and non-linear (radial basis function) SVM, LDA, and random forest ensemble classifiers. In addition, we reported the classification results without applying any feature selection to validate the significance of the proposed hybrid weighted feature concatenation framework. A brief description of all the classifiers used in this study is as follows.</p>
<sec>
<title>Extreme learning machine classifier</title>
<p>The extreme learning machine originally proposed by Huang et al. (<xref ref-type="bibr" rid="B36">2006</xref>) has been adopted in many previous neuroimaging studies (Termenon et al., <xref ref-type="bibr" rid="B71">2013</xref>; Zhang and Zhang, <xref ref-type="bibr" rid="B75">2015</xref>; Qureshi et al., <xref ref-type="bibr" rid="B58">2016</xref>), and (Qureshi et al., <xref ref-type="bibr" rid="B59">2017b</xref>) in the binary and multiclass settings. The ELM randomly assigns the weights and bias to the input data to compute the output weight matrix. This random assignment of weights makes the ELM algorithm very fast compared to other gradient-based classifiers such as the SVM. A more detailed discussion of the classifier can be found elsewhere (Huang et al., <xref ref-type="bibr" rid="B36">2006</xref>; Qureshi et al., <xref ref-type="bibr" rid="B58">2016</xref>). The ELM classifier requires the &#x0201C;number of nodes&#x0201D; as the only hyper-parameter that has to be tuned for achieving maximum performance in terms of accuracy. In this study, we used the Matlab implementation of the ELM. A greedy search method was used to tune this parameter for achieving maximum test accuracy. In this study, the search scale for selecting this parameter was set to <italic>N</italic> &#x0003D; [90, 91, &#x02026;, 220]. A careful selection of the number of nodes is necessary to prevent the classifier from overfitting. All the features were normalized and scaled to values between &#x02013;1 and 1 for improving the performance of the ELM classifier (Qureshi et al., <xref ref-type="bibr" rid="B58">2016</xref>, <xref ref-type="bibr" rid="B59">2017b</xref>).</p>
</sec>
<sec>
<title>Support vector machine (SVM) classifier</title>
<p>The SVM classifier, which was originally proposed by Cortes and Vapnik (<xref ref-type="bibr" rid="B12">1995</xref>), has been one of the most popular machine-learning tools in the neuroscience domain in the last decade. It is a supervised classification algorithm. It maps features in higher dimensional space using linear and non-linear functions known as kernels. In this study, we used both linear and non-linear (RBF) SVM. In case of SVM-RBF, value of scaling parameter &#x0201C;&#x003C3;&#x0201D; was tuned in the range &#x003C3; &#x0003D; [0.000001, 0.00001, &#x02026;, 100000, 1000000] and box-constraint parameter &#x0201C;C&#x0201D; was tuned in range of <italic>C</italic> &#x0003D; [0.005, 0.01, 0.1, &#x02026;, 100000, 1000000] using logarithmic grid search method.</p>
</sec>
<sec>
<title>Linear discriminant analysis (LDA) classifier</title>
<p>The LDA classifier, originally proposed by Duda and Hart (<xref ref-type="bibr" rid="B21">1973</xref>), is a linear classifier that utilizes hyperplanes to discriminate the data. These hyperplanes maximize the interclass mean while keeping the interclass variance minimized.</p>
</sec>
<sec>
<title>Random forest ensemble classifier</title>
<p>Random forest trees, originally proposed by Breiman (<xref ref-type="bibr" rid="B6">1996</xref>), are a method of building a forest of uncorrelated trees with randomized node optimization and bagging. Out-of-bag errors are used as an estimate of the generalization error. The random forest (RF) measures variable importance through permutation (Liaw and Wiener, <xref ref-type="bibr" rid="B43">2002</xref>). The general bootstrap aggregation algorithm was used for training. In RF implementation, with decision trees as the learner type, only the number of trees and the number of splits for prediction need to be defined. In this study, we used 30 trees and 20 splits.</p>
</sec>
</sec>
<sec>
<title>Cross-validation, performance evaluation, and significance testing methods</title>
<p>The classifier performance was measured in terms of nested 10-by-10-fold cross-validated classification accuracy. In the nested cross-validation we repeated each of the 10-fold cross-validation 10 times and reported the average accuracy of the ten 10-fold cross-validation trials. In addition, sensitivity, specificity, negative predictive value, positive predictive value, and F1-Score were calculated as supporting measures for performance evaluation. These measures were calculated through the true positive, true negative, false positive, and false negative values acquired from the confusion matrix by computing the predicted labels with the true labels of the test data. Formally these measures are defined as follows:</p>
<disp-formula id="E4"><mml:math id="M6"><mml:mtable columnalign='left'><mml:mtr><mml:mtd><mml:mtext>&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;</mml:mtext><mml:mi>A</mml:mi><mml:mi>c</mml:mi><mml:mi>c</mml:mi><mml:mi>u</mml:mi><mml:mi>r</mml:mi><mml:mi>a</mml:mi><mml:mi>c</mml:mi><mml:mi>y</mml:mi><mml:mtext>&#x000A0;&#x000A0;</mml:mtext><mml:mo>=</mml:mo><mml:mtext>&#x000A0;&#x000A0;</mml:mtext><mml:mo stretchy='false'>(</mml:mo><mml:mi>T</mml:mi><mml:mi>P</mml:mi><mml:mo>+</mml:mo><mml:mi>T</mml:mi><mml:mi>N</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mo>/</mml:mo><mml:mo stretchy='false'>(</mml:mo><mml:mi>T</mml:mi><mml:mi>P</mml:mi><mml:mo>+</mml:mo><mml:mi>T</mml:mi><mml:mi>N</mml:mi></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mtext>&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;</mml:mtext><mml:mo>+</mml:mo><mml:mi>F</mml:mi><mml:mi>P</mml:mi><mml:mo>+</mml:mo><mml:mi>F</mml:mi><mml:mi>N</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mtext>&#x000A0;</mml:mtext></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mtext>&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;</mml:mtext><mml:mi>S</mml:mi><mml:mi>e</mml:mi><mml:mi>n</mml:mi><mml:mi>s</mml:mi><mml:mi>i</mml:mi><mml:mi>t</mml:mi><mml:mi>i</mml:mi><mml:mi>v</mml:mi><mml:mi>i</mml:mi><mml:mi>t</mml:mi><mml:mi>y</mml:mi><mml:mtext>&#x000A0;</mml:mtext><mml:mo>=</mml:mo><mml:mtext>&#x000A0;</mml:mtext><mml:mi>T</mml:mi><mml:mi>P</mml:mi><mml:mo>/</mml:mo><mml:mo stretchy='false'>(</mml:mo><mml:mi>T</mml:mi><mml:mi>P</mml:mi><mml:mo>+</mml:mo><mml:mi>F</mml:mi><mml:mi>N</mml:mi><mml:mo stretchy='false'>)</mml:mo></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mtext>&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;</mml:mtext><mml:mi>S</mml:mi><mml:mi>p</mml:mi><mml:mi>e</mml:mi><mml:mi>c</mml:mi><mml:mi>i</mml:mi><mml:mi>f</mml:mi><mml:mi>i</mml:mi><mml:mi>c</mml:mi><mml:mi>i</mml:mi><mml:mi>t</mml:mi><mml:mi>y</mml:mi><mml:mtext>&#x000A0;</mml:mtext><mml:mo>=</mml:mo><mml:mtext>&#x000A0;</mml:mtext><mml:mi>T</mml:mi><mml:mi>N</mml:mi><mml:mo>/</mml:mo><mml:mo stretchy='false'>(</mml:mo><mml:mi>F</mml:mi><mml:mi>P</mml:mi><mml:mo>+</mml:mo><mml:mi>T</mml:mi><mml:mi>N</mml:mi><mml:mo stretchy='false'>)</mml:mo></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mi>N</mml:mi><mml:mi>e</mml:mi><mml:mi>g</mml:mi><mml:mi>a</mml:mi><mml:mi>t</mml:mi><mml:mi>i</mml:mi><mml:mi>v</mml:mi><mml:mi>e</mml:mi><mml:mtext>&#x000A0;</mml:mtext><mml:mi>P</mml:mi><mml:mi>r</mml:mi><mml:mi>e</mml:mi><mml:mi>d</mml:mi><mml:mi>i</mml:mi><mml:mi>c</mml:mi><mml:mi>t</mml:mi><mml:mi>i</mml:mi><mml:mi>v</mml:mi><mml:mi>e</mml:mi><mml:mi>V</mml:mi><mml:mi>a</mml:mi><mml:mi>l</mml:mi><mml:mi>u</mml:mi><mml:mi>e</mml:mi><mml:mtext>&#x000A0;</mml:mtext><mml:mo stretchy='false'>(</mml:mo><mml:mi>N</mml:mi><mml:mi>P</mml:mi><mml:mi>V</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mtext>&#x000A0;</mml:mtext><mml:mo>=</mml:mo><mml:mtext>&#x000A0;</mml:mtext><mml:mi>T</mml:mi><mml:mi>N</mml:mi><mml:mo>/</mml:mo><mml:mo stretchy='false'>(</mml:mo><mml:mi>F</mml:mi><mml:mi>N</mml:mi><mml:mo>+</mml:mo><mml:mi>T</mml:mi><mml:mi>N</mml:mi><mml:mo stretchy='false'>)</mml:mo></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mtext>&#x000A0;&#x000A0;</mml:mtext><mml:mi>P</mml:mi><mml:mi>o</mml:mi><mml:mi>s</mml:mi><mml:mi>i</mml:mi><mml:mi>t</mml:mi><mml:mi>i</mml:mi><mml:mi>v</mml:mi><mml:mi>e</mml:mi><mml:mtext>&#x000A0;</mml:mtext><mml:mi>P</mml:mi><mml:mi>r</mml:mi><mml:mi>e</mml:mi><mml:mi>d</mml:mi><mml:mi>i</mml:mi><mml:mi>c</mml:mi><mml:mi>t</mml:mi><mml:mi>i</mml:mi><mml:mi>v</mml:mi><mml:mi>e</mml:mi><mml:mi>V</mml:mi><mml:mi>a</mml:mi><mml:mi>l</mml:mi><mml:mi>u</mml:mi><mml:mi>e</mml:mi><mml:mtext>&#x000A0;</mml:mtext><mml:mo stretchy='false'>(</mml:mo><mml:mi>P</mml:mi><mml:mi>P</mml:mi><mml:mi>V</mml:mi><mml:mo stretchy='false'>)</mml:mo><mml:mtext>&#x000A0;</mml:mtext><mml:mo>=</mml:mo><mml:mtext>&#x000A0;</mml:mtext><mml:mi>T</mml:mi><mml:mi>P</mml:mi><mml:mo>/</mml:mo><mml:mo stretchy='false'>(</mml:mo><mml:mi>T</mml:mi><mml:mi>P</mml:mi><mml:mo>+</mml:mo><mml:mi>F</mml:mi><mml:mi>P</mml:mi><mml:mo stretchy='false'>)</mml:mo></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mtext>&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;&#x000A0;</mml:mtext><mml:mi>F</mml:mi><mml:mn>1</mml:mn><mml:mo>&#x02212;</mml:mo><mml:mi>S</mml:mi><mml:mi>c</mml:mi><mml:mi>o</mml:mi><mml:mi>r</mml:mi><mml:mi>e</mml:mi><mml:mtext>&#x000A0;</mml:mtext><mml:mo>=</mml:mo><mml:mtext>&#x000A0;</mml:mtext><mml:mn>2</mml:mn><mml:mi>T</mml:mi><mml:mi>P</mml:mi><mml:mo>/</mml:mo><mml:mo stretchy='false'>(</mml:mo><mml:mn>2</mml:mn><mml:mi>T</mml:mi><mml:mi>P</mml:mi><mml:mo>+</mml:mo><mml:mi>F</mml:mi><mml:mi>P</mml:mi><mml:mo>+</mml:mo><mml:mi>F</mml:mi><mml:mi>N</mml:mi><mml:mo stretchy='false'>)</mml:mo></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula>
<p>A Permutation test was used to assess the statistical significance of ELM classifier performance (Golland and Fischl, <xref ref-type="bibr" rid="B31">2003</xref>). Briefly, it works as follows. First, we chose the actual test accuracy as the test statistic of the classifier, the class labels for testing the dataset permuted randomly and were given to the classifier and checked for cross-validation. Generally, the lower the <italic>p</italic>-value of the permuted prediction rate against the prediction rate of the original data labels the higher the significance of the classifier performance. There is no fixed rule for setting the number of permutations. We permuted the data 10,000 times in the current study similar to previous studies (Qureshi et al., <xref ref-type="bibr" rid="B58">2016</xref>, <xref ref-type="bibr" rid="B57">2017a</xref>,<xref ref-type="bibr" rid="B59">b</xref>)</p>
<p>Figure <xref ref-type="fig" rid="F4">4</xref> shows the overall framework of the study including preprocessing, features extraction and hybrid weighted concatenation, and finally classification.</p>
<fig id="F4" position="float">
<label>Figure 4</label>
<caption><p>This figure depicts the overall classification feature weighting and concatenation framework.</p></caption>
<graphic xlink:href="fninf-11-00059-g0004.tif"/>
</fig>
</sec>
</sec>
<sec sec-type="results" id="s3">
<title>Results</title>
<p>Results are reported for all the unimodal and multimodal measures for both the simple and hybrid weighted feature concatenation methods in addition to the results of each standalone functional connectivity measure and modality from both the cortical and subcortical regions. The most important result of this study was the highest 10-by-10-fold nested cross-validated predictive accuracy of 99.29% (<italic>p</italic> &#x0003C; 0.0001) by using the hybrid weighted feature concatenation method. In addition, we reported the classification performance of simple concatenation of multimodal cortical and subcortical features. Stand-alone measure accuracies for all the structural and functional features were also reported.</p>
<p>The weight of each measure along with the rank is mentioned in the Table <xref ref-type="table" rid="T3">3</xref>.</p>
<table-wrap position="float" id="T3">
<label>Table 3</label>
<caption><p>Weight and rank of each individual measure.</p></caption>
<table frame="hsides" rules="groups">
<thead><tr>
<th valign="top" align="left"><bold>Rank</bold></th>
<th valign="top" align="left"><bold>Measure</bold></th>
<th valign="top" align="center"><bold>Weight</bold></th>
</tr>
</thead>
<tbody>
<tr>
<td valign="top" align="left">1</td>
<td valign="top" align="left">Group ICA</td>
<td valign="top" align="center">1.0000</td>
</tr>
<tr>
<td valign="top" align="left">2</td>
<td valign="top" align="left">Curvature</td>
<td valign="top" align="center">0.9943</td>
</tr>
<tr>
<td valign="top" align="left">3</td>
<td valign="top" align="left">SC GCOR</td>
<td valign="top" align="center">0.9829</td>
</tr>
<tr>
<td valign="top" align="left">4</td>
<td valign="top" align="left">Thickness</td>
<td valign="top" align="center">0.9819</td>
</tr>
<tr>
<td valign="top" align="left">5</td>
<td valign="top" align="left">Overall Volume</td>
<td valign="top" align="center">0.9810</td>
</tr>
<tr>
<td valign="top" align="left">6</td>
<td valign="top" align="left">Volume</td>
<td valign="top" align="center">0.9768</td>
</tr>
<tr>
<td valign="top" align="left">7</td>
<td valign="top" align="left">Cortical GCOR</td>
<td valign="top" align="center">0.9762</td>
</tr>
<tr>
<td valign="top" align="left">8</td>
<td valign="top" align="left">Thickness STD</td>
<td valign="top" align="center">0.9753</td>
</tr>
<tr>
<td valign="top" align="left">9</td>
<td valign="top" align="left">SC Intensity</td>
<td valign="top" align="center">0.9695</td>
</tr>
<tr>
<td valign="top" align="left">10</td>
<td valign="top" align="left">SC Volume</td>
<td valign="top" align="center">0.9694</td>
</tr>
<tr>
<td valign="top" align="left">11</td>
<td valign="top" align="left">WM</td>
<td valign="top" align="center">0.9661</td>
</tr>
<tr>
<td valign="top" align="left">12</td>
<td valign="top" align="left">Surface Area</td>
<td valign="top" align="center">0.9518</td>
</tr>
</tbody>
</table>
<table-wrap-foot>
<p><italic>ICA, Independent component analysis; SC, subcortical; GCOR, global average functional connectivity; STD, standard deviation; WM, white matter</italic>.</p>
</table-wrap-foot>
</table-wrap>
<p>Table <xref ref-type="table" rid="T4">4</xref> summarizes the comparative results of all five ELM, SVM-L, SVM-RBF, LDA, and RF classifiers. Mean testing and training classification scores after 10-by-10-fold nested cross-validation of all the feature groups along with sensitivity, specificity, F1-score, negative predictive value, and positive predictive value measures for all the unimodal and multimodal features are reported for ELM.</p>
<table-wrap position="float" id="T4">
<label>Table 4</label>
<caption><p>Mean classification performance of multimodal features.</p></caption>
<table frame="hsides" rules="groups">
<thead><tr>
<th valign="top" align="left"><bold>Classifier</bold></th>
<th valign="top" align="center" colspan="8"><bold>Extreme learning machine</bold></th>
<th valign="top" align="center"><bold>SVM-L</bold></th>
<th valign="top" align="center"><bold>SVM-RBF</bold></th>
<th valign="top" align="center"><bold>LDA</bold></th>
<th valign="top" align="center"><bold>RF</bold></th>
</tr>
</thead>
<tbody>
<tr>
<td valign="top" align="left"><bold>Feature type</bold></td>
<td valign="top" align="center"><bold>Train Acc</bold></td>
<td valign="top" align="center"><bold>Test Acc</bold></td>
<td valign="top" align="center"><italic><bold>p</bold></italic><bold>-value</bold></td>
<td valign="top" align="center"><bold>Sn</bold></td>
<td valign="top" align="center"><bold>Sp</bold></td>
<td valign="top" align="center"><bold>F1-Score</bold></td>
<td valign="top" align="center"><bold>PPV</bold></td>
<td valign="top" align="center"><bold>NPV</bold></td>
<td valign="top" align="center"><bold>Test Acc</bold></td>
<td valign="top" align="center"><bold>Test Acc</bold></td>
<td valign="top" align="center"><bold>Test Acc</bold></td>
<td valign="top" align="center"><bold>Test Acc</bold></td>
</tr>
<tr style="border-top: thin solid #000000;">
<td valign="top" align="left">Hybrid WC</td>
<td valign="top" align="center">0.9954</td>
<td valign="top" align="center">0.9929</td>
<td valign="top" align="center">0.0001</td>
<td valign="top" align="center">1.000</td>
<td valign="top" align="center">0.9857</td>
<td valign="top" align="center">0.9933</td>
<td valign="top" align="center">1.000</td>
<td valign="top" align="center">0.9875</td>
<td valign="top" align="center">0.7780</td>
<td valign="top" align="center">0.763</td>
<td valign="top" align="center">0.6810</td>
<td valign="top" align="center">0.7080</td>
</tr>
<tr>
<td valign="top" align="left">Simple WC</td>
<td valign="top" align="center">0.9977</td>
<td valign="top" align="center">0.9804</td>
<td valign="top" align="center">0.0001</td>
<td valign="top" align="center">0.9732</td>
<td valign="top" align="center">0.9875</td>
<td valign="top" align="center">0.9790</td>
<td valign="top" align="center">0.9778</td>
<td valign="top" align="center">0.9875</td>
<td valign="top" align="center">0.7500</td>
<td valign="top" align="center">0.750</td>
<td valign="top" align="center">0.6880</td>
<td valign="top" align="center">0.7010</td>
</tr>
<tr>
<td valign="top" align="left">Simple concatenation</td>
<td valign="top" align="center">0.9931</td>
<td valign="top" align="center">0.9724</td>
<td valign="top" align="center">0.0001</td>
<td valign="top" align="center">0.9607</td>
<td valign="top" align="center">0.9857</td>
<td valign="top" align="center">0.9723</td>
<td valign="top" align="center">0.9625</td>
<td valign="top" align="center">0.9875</td>
<td valign="top" align="center">0.7710</td>
<td valign="top" align="center">0.750</td>
<td valign="top" align="center">0.6880</td>
<td valign="top" align="center">0.7500</td>
</tr>
<tr>
<td valign="top" align="left">Cortical WC</td>
<td valign="top" align="center">0.9899</td>
<td valign="top" align="center">0.9367</td>
<td valign="top" align="center">0.0001</td>
<td valign="top" align="center">0.9429</td>
<td valign="top" align="center">0.9321</td>
<td valign="top" align="center">0.9359</td>
<td valign="top" align="center">0.9500</td>
<td valign="top" align="center">0.9375</td>
<td valign="top" align="center">0.5960</td>
<td valign="top" align="center">0.611</td>
<td valign="top" align="center">0.5830</td>
<td valign="top" align="center">0.5490</td>
</tr>
<tr>
<td valign="top" align="left">Subcortical WC</td>
<td valign="top" align="center">0.9931</td>
<td valign="top" align="center">0.9248</td>
<td valign="top" align="center">0.0001</td>
<td valign="top" align="center">0.9571</td>
<td valign="top" align="center">0.8857</td>
<td valign="top" align="center">0.9296</td>
<td valign="top" align="center">0.9639</td>
<td valign="top" align="center">0.9128</td>
<td valign="top" align="center">0.6670</td>
<td valign="top" align="center">0.667</td>
<td valign="top" align="center">0.6250</td>
<td valign="top" align="center">0.6040</td>
</tr>
<tr>
<td valign="top" align="left">Cortical concatenation</td>
<td valign="top" align="center">0.9938</td>
<td valign="top" align="center">0.9381</td>
<td valign="top" align="center">0.0001</td>
<td valign="top" align="center">0.9303</td>
<td valign="top" align="center">0.9464</td>
<td valign="top" align="center">0.9367</td>
<td valign="top" align="center">0.9403</td>
<td valign="top" align="center">0.9528</td>
<td valign="top" align="center">0.6320</td>
<td valign="top" align="center">0.638</td>
<td valign="top" align="center">0.6110</td>
<td valign="top" align="center">0.6110</td>
</tr>
<tr>
<td valign="top" align="left">Subcortical concatenation</td>
<td valign="top" align="center">0.9908</td>
<td valign="top" align="center">0.9314</td>
<td valign="top" align="center">0.0001</td>
<td valign="top" align="center">0.9196</td>
<td valign="top" align="center">0.9446</td>
<td valign="top" align="center">0.9309</td>
<td valign="top" align="center">0.9260</td>
<td valign="top" align="center">0.9514</td>
<td valign="top" align="center">0.6390</td>
<td valign="top" align="center">0.681</td>
<td valign="top" align="center">0.6180</td>
<td valign="top" align="center">0.6460</td>
</tr>
<tr>
<td valign="top" align="left">Functional concatenation</td>
<td valign="top" align="center">0.9961</td>
<td valign="top" align="center">0.9452</td>
<td valign="top" align="center">0.0001</td>
<td valign="top" align="center">0.9304</td>
<td valign="top" align="center">0.9625</td>
<td valign="top" align="center">0.9426</td>
<td valign="top" align="center">0.9375</td>
<td valign="top" align="center">0.9607</td>
<td valign="top" align="center">0.6740</td>
<td valign="top" align="center">0.729</td>
<td valign="top" align="center">0.6880</td>
<td valign="top" align="center">0.6740</td>
</tr>
<tr>
<td valign="top" align="left">Structural concatenation</td>
<td valign="top" align="center">0.9907</td>
<td valign="top" align="center">0.9319</td>
<td valign="top" align="center">0.0001</td>
<td valign="top" align="center">0.9304</td>
<td valign="top" align="center">0.9357</td>
<td valign="top" align="center">0.9308</td>
<td valign="top" align="center">0.9389</td>
<td valign="top" align="center">0.9417</td>
<td valign="top" align="center">0.6460</td>
<td valign="top" align="center">0.625</td>
<td valign="top" align="center">0.6600</td>
<td valign="top" align="center">0.6460</td>
</tr>
</tbody>
</table>
<table-wrap-foot>
<p><italic>Sn, Sensitivity; Sp, Specificity; SVM-L, linear support vector machine; SVM-RBF, support vector machine with radial basis function kernel, LDA, linear discriminant analysis; WC, Weighted Concatenation; Acc, Accuracy; PPV, positive predictive value; NPV, Negative predictive value; RF, Random Forest</italic>.</p>
</table-wrap-foot>
</table-wrap>
<p>Figure <xref ref-type="fig" rid="F5">5</xref> depicts all the mean measures reported in Table <xref ref-type="table" rid="T3">3</xref> along with the standard deviation for the ELM results.</p>
<fig id="F5" position="float">
<label>Figure 5</label>
<caption><p>Mean and standard deviation of the classification performance parameters for ELM.</p></caption>
<graphic xlink:href="fninf-11-00059-g0005.tif"/>
</fig>
<p>In addition, we calculated the accuracy of all the individual measures for each modality separately. Table <xref ref-type="table" rid="T5">5</xref> summarizes the mean classification scores after 10-by-10-fold nested cross-validation for each individual measure type of structural and functional data.</p>
<table-wrap position="float" id="T5">
<label>Table 5</label>
<caption><p>Mean classification results of each measure of the data from each modality.</p></caption>
<table frame="hsides" rules="groups">
<thead><tr>
<th valign="top" align="left"><bold>Classifier</bold></th>
<th valign="top" align="center" colspan="8"><bold>Extreme learning machine</bold></th>
<th valign="top" align="center"><bold>SVM-L</bold></th>
<th valign="top" align="center"><bold>SVM-RBF</bold></th>
<th valign="top" align="center"><bold>LDA</bold></th>
<th valign="top" align="center"><bold>RF</bold></th>
</tr>
</thead>
<tbody>
<tr>
<td valign="top" align="left"><bold>Feature type</bold></td>
<td valign="top" align="center"><bold>Train Acc</bold></td>
<td valign="top" align="center"><bold>Test Acc</bold></td>
<td valign="top" align="center"><italic><bold>p</bold></italic><bold>-value</bold></td>
<td valign="top" align="center"><bold>Sn</bold></td>
<td valign="top" align="center"><bold>Sp</bold></td>
<td valign="top" align="center"><bold>F1-Score</bold></td>
<td valign="top" align="center"><bold>PPV</bold></td>
<td valign="top" align="center"><bold>NPV</bold></td>
<td valign="top" align="center"><bold>Test Acc</bold></td>
<td valign="top" align="center"><bold>Test Acc</bold></td>
<td valign="top" align="center"><bold>Test Acc</bold></td>
<td valign="top" align="center"><bold>Test Acc</bold></td>
</tr>
<tr style="border-top: thin solid #000000;">
<td valign="top" align="left">Thickness</td>
<td valign="top" align="center">0.9908</td>
<td valign="top" align="center">0.9114</td>
<td valign="top" align="center">0.0001</td>
<td valign="top" align="center">0.9339</td>
<td valign="top" align="center">0.8929</td>
<td valign="top" align="center">0.9120</td>
<td valign="top" align="center">0.9403</td>
<td valign="top" align="center">0.9024</td>
<td valign="top" align="center">0.5630</td>
<td valign="top" align="center">0.598</td>
<td valign="top" align="center">0.5900</td>
<td valign="top" align="center">0.6250</td>
</tr>
<tr>
<td valign="top" align="left">Thickness STD</td>
<td valign="top" align="center">1.000</td>
<td valign="top" align="center">0.9048</td>
<td valign="top" align="center">0.0001</td>
<td valign="top" align="center">0.9000</td>
<td valign="top" align="center">0.9089</td>
<td valign="top" align="center">0.8993</td>
<td valign="top" align="center">0.9175</td>
<td valign="top" align="center">0.9099</td>
<td valign="top" align="center">0.6040</td>
<td valign="top" align="center">0.614</td>
<td valign="top" align="center">0.5900</td>
<td valign="top" align="center">0.5070</td>
</tr>
<tr>
<td valign="top" align="left">Surface Area</td>
<td valign="top" align="center">0.9900</td>
<td valign="top" align="center">0.8813</td>
<td valign="top" align="center">0.0001</td>
<td valign="top" align="center">0.8339</td>
<td valign="top" align="center">0.9304</td>
<td valign="top" align="center">0.8736</td>
<td valign="top" align="center">0.8575</td>
<td valign="top" align="center">0.9339</td>
<td valign="top" align="center">0.5420</td>
<td valign="top" align="center">0.500</td>
<td valign="top" align="center">0.4720</td>
<td valign="top" align="center">0.5690</td>
</tr>
<tr>
<td valign="top" align="left">Volume</td>
<td valign="top" align="center">0.9930</td>
<td valign="top" align="center">0.9063</td>
<td valign="top" align="center">0.0001</td>
<td valign="top" align="center">0.8786</td>
<td valign="top" align="center">0.9321</td>
<td valign="top" align="center">0.9010</td>
<td valign="top" align="center">0.8975</td>
<td valign="top" align="center">0.9375</td>
<td valign="top" align="center">0.4580</td>
<td valign="top" align="center">0.576</td>
<td valign="top" align="center">0.5140</td>
<td valign="top" align="center">0.4240</td>
</tr>
<tr>
<td valign="top" align="left">Curvature</td>
<td valign="top" align="center">0.9961</td>
<td valign="top" align="center">0.9238</td>
<td valign="top" align="center">0.0001</td>
<td valign="top" align="center">0.9339</td>
<td valign="top" align="center">0.9143</td>
<td valign="top" align="center">0.9259</td>
<td valign="top" align="center">0.9403</td>
<td valign="top" align="center">0.9300</td>
<td valign="top" align="center">0.5830</td>
<td valign="top" align="center">0.601</td>
<td valign="top" align="center">0.5830</td>
<td valign="top" align="center">0.5560</td>
</tr>
<tr>
<td valign="top" align="left">WM Volume</td>
<td valign="top" align="center">0.9954</td>
<td valign="top" align="center">0.8956</td>
<td valign="top" align="center">0.0001</td>
<td valign="top" align="center">0.8643</td>
<td valign="top" align="center">0.9286</td>
<td valign="top" align="center">0.8916</td>
<td valign="top" align="center">0.8820</td>
<td valign="top" align="center">0.9367</td>
<td valign="top" align="center">0.4720</td>
<td valign="top" align="center">0.542</td>
<td valign="top" align="center">0.5070</td>
<td valign="top" align="center">0.4930</td>
</tr>
<tr>
<td valign="top" align="left">Cortical GCOR</td>
<td valign="top" align="center">0.9891</td>
<td valign="top" align="center">0.9057</td>
<td valign="top" align="center">0.0001</td>
<td valign="top" align="center">0.9304</td>
<td valign="top" align="center">0.8839</td>
<td valign="top" align="center">0.9051</td>
<td valign="top" align="center">0.9385</td>
<td valign="top" align="center">0.8913</td>
<td valign="top" align="center">0.5490</td>
<td valign="top" align="center">0.591</td>
<td valign="top" align="center">0.5970</td>
<td valign="top" align="center">0.6250</td>
</tr>
<tr>
<td valign="top" align="left">SC Volume</td>
<td valign="top" align="center">0.9745</td>
<td valign="top" align="center">0.8989</td>
<td valign="top" align="center">0.0001</td>
<td valign="top" align="center">0.9018</td>
<td valign="top" align="center">0.8946</td>
<td valign="top" align="center">0.8958</td>
<td valign="top" align="center">0.9157</td>
<td valign="top" align="center">0.9014</td>
<td valign="top" align="center">0.7010</td>
<td valign="top" align="center">0.597</td>
<td valign="top" align="center">0.6180</td>
<td valign="top" align="center">0.6250</td>
</tr>
<tr>
<td valign="top" align="left">SC Intensity</td>
<td valign="top" align="center">0.9930</td>
<td valign="top" align="center">0.8990</td>
<td valign="top" align="center">0.0001</td>
<td valign="top" align="center">0.8589</td>
<td valign="top" align="center">0.9339</td>
<td valign="top" align="center">0.8906</td>
<td valign="top" align="center">0.8864</td>
<td valign="top" align="center">0.9389</td>
<td valign="top" align="center">0.6740</td>
<td valign="top" align="center">0.625</td>
<td valign="top" align="center">0.6180</td>
<td valign="top" align="center">0.6320</td>
</tr>
<tr>
<td valign="top" align="left">SC GCOR</td>
<td valign="top" align="center">0.9884</td>
<td valign="top" align="center">0.9124</td>
<td valign="top" align="center">0.0001</td>
<td valign="top" align="center">0.8750</td>
<td valign="top" align="center">0.9482</td>
<td valign="top" align="center">0.9048</td>
<td valign="top" align="center">0.9008</td>
<td valign="top" align="center">0.9528</td>
<td valign="top" align="center">0.5280</td>
<td valign="top" align="center">0.583</td>
<td valign="top" align="center">0.5970</td>
<td valign="top" align="center">0.5760</td>
</tr>
<tr>
<td valign="top" align="left">Overall Volume</td>
<td valign="top" align="center">0.9861</td>
<td valign="top" align="center">0.9105</td>
<td valign="top" align="center">0.0001</td>
<td valign="top" align="center">0.9161</td>
<td valign="top" align="center">0.9036</td>
<td valign="top" align="center">0.9100</td>
<td valign="top" align="center">0.9228</td>
<td valign="top" align="center">0.9103</td>
<td valign="top" align="center">0.6320</td>
<td valign="top" align="center">0.631</td>
<td valign="top" align="center">0.5490</td>
<td valign="top" align="center">0.5630</td>
</tr>
<tr>
<td valign="top" align="left">Group ICA</td>
<td valign="top" align="center">0.9915</td>
<td valign="top" align="center">0.9295</td>
<td valign="top" align="center">0.0001</td>
<td valign="top" align="center">0.9000</td>
<td valign="top" align="center">0.9571</td>
<td valign="top" align="center">0.9269</td>
<td valign="top" align="center">0.9139</td>
<td valign="top" align="center">0.9639</td>
<td valign="top" align="center">0.7150</td>
<td valign="top" align="center">0.743</td>
<td valign="top" align="center">0.6940</td>
<td valign="top" align="center">0.6390</td>
</tr>
</tbody>
</table>
<table-wrap-foot>
<p><italic>Sn, Sensitivity; Sp, Specificity; SVM-L, linear support vector machine; SVM-RBF, support vector machine with radial basis function kernel, LDA, linear discriminant analysis; WC, Weighted Concatenation; Acc, Accuracy; PPV, positive predictive value; NPV, Negative predictive value; GCOR, global average functional connectivity; ICA, independent component analysis; WM, white matter; STD, standard deviation; SC, subcortical; RF, random forest</italic>.</p>
</table-wrap-foot>
</table-wrap>
<p>Figure <xref ref-type="fig" rid="F6">6</xref> depicts all the mean measures reported in Table <xref ref-type="table" rid="T4">4</xref> along with the standard deviation for the ELM results.</p>
<fig id="F6" position="float">
<label>Figure 6</label>
<caption><p>Mean and standard deviation of the classification performance parameters for ELM.</p></caption>
<graphic xlink:href="fninf-11-00059-g0006.tif"/>
</fig>
</sec>
<sec sec-type="discussion" id="s4">
<title>Discussion</title>
<p>This study reports the multimodal hybrid weighted feature concatenation of neuroimaging data for the purpose of classification. We proposed a simple and powerful hybrid weighted feature concatenation method. Two types of functional connectivity based features were utilized in this study to acquire the highest classification accuracy. Group independent component analysis (gICA) of functional connectivity features is a very robust stand-alone measure for achieving high classification accuracy and provides an accuracy as high as 92.95% (<italic>p</italic> &#x0003C; 0.0001) when used with an extreme learning machine. In addition, our highest prediction result of 99.29% (<italic>p</italic> &#x0003C; 0.0001) accuracy shows that the proposed hybrid weighted features concatenation method is very effective for boosting the accuracy of the ELM classifier.</p>
<p>The ELM classifier in combination with the proposed hybrid weighted feature concatenation method is effective for the classification of neuroimaging data. A few recent studies such as Qureshi et al. (<xref ref-type="bibr" rid="B58">2016</xref>), Qureshi et al. (<xref ref-type="bibr" rid="B57">2017a</xref>), and Qureshi et al. (<xref ref-type="bibr" rid="B59">2017b</xref>) were based on an extreme learning machine classification framework for neuroimaging data. However, the current study proposed a very simplified hybrid weighted feature concatenation method that allows for the acquisition of higher classification accuracies without requiring any kind of feature selection approach. To wit, this advanced concatenation method is time-effective while boosting classification performance.</p>
<sec>
<title>Comparison to previous studies</title>
<p>For the purpose of comparing the efficiency of the method proposed in the current study, we enlisted all previous studies on multimodal classification of Schizophrenia in Table <xref ref-type="table" rid="T6">6</xref>, to the best of our knowledge.</p>
<table-wrap position="float" id="T6">
<label>Table 6</label>
<caption><p>Comparison with the classification scores of previous studies using multimodal data.</p></caption>
<table frame="hsides" rules="groups">
<thead><tr>
<th valign="top" align="left"><bold>Study reference</bold></th>
<th valign="top" align="left"><bold>Modality</bold></th>
<th valign="top" align="center"><bold>Sample size</bold></th>
<th valign="top" align="left"><bold>Features</bold></th>
<th valign="top" align="center"><bold>Feature count</bold></th>
<th valign="top" align="left"><bold>Classifier</bold></th>
<th valign="top" align="center"><bold>Accuracy (%)</bold></th>
</tr>
</thead>
<tbody>
<tr>
<td valign="top" align="left">Proposed Method</td>
<td valign="top" align="left">sMRI &#x0002B; fMRI</td>
<td valign="top" align="center">144</td>
<td valign="top" align="left">Structural ROI measures, Global functional connectivity, Group ICA</td>
<td valign="top" align="center">748</td>
<td valign="top" align="left">ELM</td>
<td valign="top" align="center">99.29</td>
</tr>
<tr>
<td valign="top" align="left">Du et al., <xref ref-type="bibr" rid="B20">2012</xref></td>
<td valign="top" align="left">rs-fMRI &#x0002B; task-related fMRI (auditory oddball task)</td>
<td valign="top" align="center">56</td>
<td valign="top" align="left">Kernel PCA with spatial ICA maps</td>
<td valign="top" align="center">53</td>
<td valign="top" align="left">Fisher&#x00027;s linear discriminant</td>
<td valign="top" align="center">98</td>
</tr>
<tr>
<td valign="top" align="left">Silva et al., <xref ref-type="bibr" rid="B67">2014</xref></td>
<td valign="top" align="left">sMRI &#x0002B; fMRI</td>
<td valign="top" align="center">144</td>
<td valign="top" align="left">Gray matter density based ICA features (sMRI), group ICA features (fMRI)</td>
<td valign="top" align="center">410</td>
<td valign="top" align="left">Gaussian process classifier</td>
<td valign="top" align="center">94</td>
</tr>
<tr>
<td valign="top" align="left">Cetin et al., <xref ref-type="bibr" rid="B10">2016</xref></td>
<td valign="top" align="left">rs-fMRI &#x0002B; task-related fMRI (auditory oddball task) &#x0002B; MEG</td>
<td valign="top" align="center">55</td>
<td valign="top" align="left">Group ICA based functional connectivity scores, MEG data for each frequency</td>
<td valign="top" align="center">103</td>
<td valign="top" align="left">LDA, Na&#x000EF;ve Bayes classifier, and non-linear SVM</td>
<td valign="top" align="center">90</td>
</tr>
<tr>
<td valign="top" align="left">Ota et al., <xref ref-type="bibr" rid="B54">2013</xref></td>
<td valign="top" align="left">sMRI &#x0002B; diffusion MRI</td>
<td valign="top" align="center">50</td>
<td valign="top" align="left">Volume and fractional anisotropy (FA) of certain regions (insula, thalamus, ACC, ventricles, and corpus callosum)</td>
<td valign="top" align="center">31</td>
<td valign="top" align="left">Linear discriminant analysis</td>
<td valign="top" align="center">88</td>
</tr>
<tr>
<td valign="top" align="left">Yang et al., <xref ref-type="bibr" rid="B73">2010</xref></td>
<td valign="top" align="left">task-related fMRI (auditory oddball task) &#x0002B; SNP (genetic data)</td>
<td valign="top" align="center">40</td>
<td valign="top" align="left">Voxels in the fMRI map, ICA components, SNPs</td>
<td valign="top" align="center">411</td>
<td valign="top" align="left">Support vector machine</td>
<td valign="top" align="center">87</td>
</tr>
<tr>
<td valign="top" align="left">Sui et al., <xref ref-type="bibr" rid="B70">2013</xref></td>
<td valign="top" align="left">sMRI &#x0002B; rs-fMRI &#x0002B; diffusion MRI</td>
<td valign="top" align="center">63</td>
<td valign="top" align="left">GM density, ALFF (amplitude of low-frequency fluctuation), FA</td>
<td valign="top" align="center">1,863</td>
<td valign="top" align="left">Support vector machine</td>
<td valign="top" align="center">79</td>
</tr>
</tbody>
</table>
</table-wrap>
<p>To date, numerous classification studies have been conducted using only one neuroimaging modality to discriminate patients with schizophrenia from healthy controls (Fan et al., <xref ref-type="bibr" rid="B23">2005</xref>, <xref ref-type="bibr" rid="B22">2008</xref>; Bassett et al., <xref ref-type="bibr" rid="B3">2012</xref>; Zanetti et al., <xref ref-type="bibr" rid="B74">2013</xref>). As shown in Table <xref ref-type="table" rid="T6">6</xref>, we listed all previous classification studies that used schizophrenia vs. normal control multimodal data, and some of these studies achieved high classification accuracy. However, a relatively small sample size can boost accuracy by chance, therefore, more than 130 subjects were reportedly required to perform such a study (Nieuwenhuis et al., <xref ref-type="bibr" rid="B51">2012</xref>). In fact, only one of these studies fulfilled this criterion (Silva et al., <xref ref-type="bibr" rid="B67">2014</xref>). The result of this study was good (94% accuracy); however, the number of included feature types was smaller than that of the present study. For example, we utilized global and group ICA-based connectivity features from the fMRI data as well as other structural measures including surface area, curvature, and white matter information. Considering that additional features, such as surface area, can be related to disease progress, the use of more features might have contributed to the higher accuracy of our classification (Li et al., <xref ref-type="bibr" rid="B42">2016</xref>). White matter information, in particular, should not be ignored as there is evidence that schizophrenia is a disorder involving both gray and white matter abnormalities (Davis et al., <xref ref-type="bibr" rid="B17">2003</xref>; Kubicki et al., <xref ref-type="bibr" rid="B39">2005</xref>).</p>
</sec>
<sec>
<title>Clinical significance of the results</title>
<p>To summarize, our results showed that the highest level of classification accuracy may be obtained when integrating all structural information including gray and white matter, with functional connectivity information such as ICA and global connectivity. It may mean that, several parts of the brain experience deterioration, both structurally and functionally, as a result of schizophrenia, thus, classification using machine learning should include the maximum amount of information computational capacity permits. The DSM criteria, by which schizophrenia is diagnosed, is still not perfect. The population of patients with schizophrenia is heterogeneous, as the diagnosis is based on observable symptoms, and that presents a challenge for diagnosing or representing all implicated characteristics by a single type of feature. In fact, since we did not adopt a feature selection process during the analysis, it is not known which features influenced the classification. A gross estimation may be performed by observing feature measure weight information as provided in Table <xref ref-type="table" rid="T3">3</xref>, where group ICA features appear as the most significant contributor to the classification, although all other features also exhibited &#x0007E;90% accuracy. These results may support the well-known findings that schizophrenia is associated with comprehensive abnormalities in both structural and functional domains. However, in terms of ranking importance, functional connectivity information showed slightly superior performance than structural information, and with regard to structural information, cortical information was superior to subcortical information in classification performance. Taken together, it appears that aberrant functional connectivity and structural abnormalities in the cortex should be given more weight, which is in line with recent research findings (Dong et al., <xref ref-type="bibr" rid="B19">2017</xref>; Larivi&#x000E8;re et al., <xref ref-type="bibr" rid="B41">2017</xref>; Massey et al., <xref ref-type="bibr" rid="B46">2017</xref>; M&#x000F8;rch-Johnsen et al., <xref ref-type="bibr" rid="B49">2017</xref>; Peters et al., <xref ref-type="bibr" rid="B55">2017</xref>).</p>
<p>In real clinical practice, the misdiagnosis related to schizophrenia usually happens due to many other psychiatric diseases which can show the similar psychotic symptoms such as mood disorders and personality disorders. In addition, the person in the premorbid or prodromal stage of psychosis who can almost fulfill the diagnostic criteria of schizophrenia confuses the clinicians about whether they should be diagnosed with schizophrenia or not. In this study, the high-risk groups or groups with other psychiatric diseases were not included, therefore our classifier has some limitations to be used directly in clinical settings because it can only differentiate healthy people and schizophrenia patients. Therefore, other classifiers (bipolar disorder vs. healthy control, bipolar disorder vs. schizophrenia, or multiclass classifier) should be developed and ameliorated to give more information to the clinicians. Considering the machine learning based diagnosis is not perfect yet, it should be used only as a supportive tool in conjunction with a standard diagnosis. However, it can be really helpful when the diagnosis is difficult.</p>
</sec>
</sec>
<sec id="s5">
<title>Limitations</title>
<p>First, our classification process underwent nested cross-validation without novel data for testing. Although we acknowledge this issue, we sought to use as much training data as possible to overcome the sample size limitations in the classification. Second, the symptom severity score (PANSS) showed that our sample&#x00027;s symptoms were relatively mild, however, the patients had already experienced active symptoms, therefore, it was challenging to use our classifier for purpose of prediction. If we had included a high-risk group, we could have utilized our classifier for early prediction.</p>
<p>In addition, it appears it is relatively easier to distinguish patients from healthy controls than to distinguish patients from high-risk groups. Considering our classifier was only learned to distinguish non-healthy brains from healthy brains, it has limited role in real clinical field. In other words, other classifiers using the data from other groups, such as bipolar disorder and high-risk groups for schizophrenia, would be more helpful. Those classifiers would enable the identification of schizophrenia-specific features and could perform the differential diagnosis among many disorders that involve structural or functional brain abnormalities. However, despite of those limitations, we believe that high accuracy of our classifier would be useful in some limited clinical situations.</p>
</sec>
<sec sec-type="conclusions" id="s6">
<title>Conclusion</title>
<p>The proposed multimodal hybrid weighted feature concatenation method is robust, simple, and straightforward, and provides promising results. The extreme learning machine is an excellent choice for classifying neuroimaging data. Multimodal and multi-measure features are robust for providing higher accuracy. The group ICA based functional connectivity features method is appropriate and exhibits the highest discriminatory power as a stand-alone measure type. We did not utilize any feature selection and optimization algorithm in this study, but as the chosen features had extremely highly discriminatory power, the present study produced clinically acceptable diagnostic accuracies.</p>
</sec>
<sec id="s7">
<title>Author contributions</title>
<p>MQ and JO have contributed equally to this work. MQ and JO selected the subjects for balanced experiment design from the publicly available COBRE database. MQ pre-processed the data, developed, and applied the proposed feature concatenation methods and classified the data. JO have drafted the introduction and discussion sections. DC, HJ, and BL helped in drafting the manuscript with MQ and JO. BL supervised the entire research process and revised the manuscript for publication. All authors contributed to the research design, results interpretation, and proofreading of the final manuscript.</p>
<sec>
<title>Conflict of interest statement</title>
<p>The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.</p>
</sec>
</sec>
</body>
<back>
<ack>
<p>Authors thankfully acknowledge The Center for Biomedical Research Excellence (COBRE) for publicly releasing the dataset.</p>
</ack>
<ref-list>
<title>References</title>
<ref id="B1">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Abou-Elseoud</surname> <given-names>A.</given-names></name> <name><surname>Starck</surname> <given-names>T.</given-names></name> <name><surname>Remes</surname> <given-names>J.</given-names></name> <name><surname>Nikkinen</surname> <given-names>J.</given-names></name> <name><surname>Tervonen</surname> <given-names>O.</given-names></name> <name><surname>Kiviniemi</surname> <given-names>V.</given-names></name></person-group> (<year>2010</year>). <article-title>The effect of model order selection in group PICA</article-title>. <source>Hum. Brain Mapp.</source> <volume>31</volume>, <fpage>1207</fpage>&#x02013;<lpage>1216</lpage>. <pub-id pub-id-type="doi">10.1002/hbm.20929</pub-id><pub-id pub-id-type="pmid">20063361</pub-id></citation></ref>
<ref id="B2">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Arbabshirani</surname> <given-names>M. R.</given-names></name> <name><surname>Plis</surname> <given-names>S.</given-names></name> <name><surname>Sui</surname> <given-names>J.</given-names></name> <name><surname>Calhoun</surname> <given-names>V. D.</given-names></name></person-group> (<year>2017</year>). <article-title>Single subject prediction of brain disorders in neuroimaging: promises and pitfalls</article-title>. <source>Neuroimage</source> <volume>145</volume>, <fpage>137</fpage>&#x02013;<lpage>165</lpage>. <pub-id pub-id-type="doi">10.1016/j.neuroimage.2016.02.079</pub-id><pub-id pub-id-type="pmid">27012503</pub-id></citation></ref>
<ref id="B3">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Bassett</surname> <given-names>D. S.</given-names></name> <name><surname>Nelson</surname> <given-names>B. G.</given-names></name> <name><surname>Mueller</surname> <given-names>B. A.</given-names></name> <name><surname>Camchong</surname> <given-names>J.</given-names></name> <name><surname>Lim</surname> <given-names>K. O.</given-names></name></person-group> (<year>2012</year>). <article-title>Altered resting state complexity in schizophrenia</article-title>. <source>Neuroimage</source> <volume>59</volume>, <fpage>2196</fpage>&#x02013;<lpage>2207</lpage>. <pub-id pub-id-type="doi">10.1016/j.neuroimage.2011.10.002</pub-id><pub-id pub-id-type="pmid">22008374</pub-id></citation></ref>
<ref id="B4">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Beckmann</surname> <given-names>C. F.</given-names></name> <name><surname>Deluca</surname> <given-names>M.</given-names></name> <name><surname>Devlin</surname> <given-names>J. T.</given-names></name> <name><surname>Smith</surname> <given-names>S. M.</given-names></name></person-group> (<year>2005</year>). <article-title>Investigations into resting-state connectivity using independent component analysis</article-title>. <source>Philos. Trans. R. Soc. Lond. B Biol. Sci.</source> <volume>360</volume>, <fpage>1001</fpage>&#x02013;<lpage>1013</lpage>. <pub-id pub-id-type="doi">10.1098/rstb.2005.1634</pub-id><pub-id pub-id-type="pmid">16087444</pub-id></citation></ref>
<ref id="B5">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Bhugra</surname> <given-names>D.</given-names></name></person-group> (<year>2005</year>). <article-title>The global prevalence of schizophrenia</article-title>. <source>PLoS Med.</source> <volume>2</volume>:<fpage>e151</fpage>. <pub-id pub-id-type="doi">10.1371/journal.pmed.0020151</pub-id><pub-id pub-id-type="pmid">15916460</pub-id></citation></ref>
<ref id="B6">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Breiman</surname> <given-names>L.</given-names></name></person-group> (<year>1996</year>). <article-title>Bagging predictors</article-title>. <source>Mach. Learn.</source> <volume>24</volume>, <fpage>123</fpage>&#x02013;<lpage>140</lpage>. <pub-id pub-id-type="doi">10.1007/BF00058655</pub-id></citation></ref>
<ref id="B7">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Cabral</surname> <given-names>C.</given-names></name> <name><surname>Kambeitz-Ilankovic</surname> <given-names>L.</given-names></name> <name><surname>Kambeitz</surname> <given-names>J.</given-names></name> <name><surname>Calhoun</surname> <given-names>V. D.</given-names></name> <name><surname>Dwyer</surname> <given-names>D. B.</given-names></name> <name><surname>Von Saldern</surname> <given-names>S.</given-names></name> <etal/></person-group>. (<year>2016</year>). <article-title>Classifying schizophrenia using multimodal multivariate pattern recognition analysis: evaluating the impact of individual clinical profiles on the neurodiagnostic performance</article-title>. <source>Schizophr. Bull.</source> <volume>42</volume>(<supplement>Suppl. 1</supplement>), <fpage>S110</fpage>&#x02013;<lpage>S117</lpage>. <pub-id pub-id-type="doi">10.1093/schbul/sbw053</pub-id><pub-id pub-id-type="pmid">27460614</pub-id></citation></ref>
<ref id="B8">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Calhoun</surname> <given-names>V. D.</given-names></name> <name><surname>Sui</surname> <given-names>J.</given-names></name></person-group> (<year>2016</year>). <article-title>Multimodal fusion of brain imaging data: a key to finding the missing link (s) in complex mental illness</article-title>. <source>Biol. Psychiatry. Cogn. Neurosci. Neuroimaging</source> <volume>1</volume>, <fpage>230</fpage>&#x02013;<lpage>244</lpage>. <pub-id pub-id-type="doi">10.1016/j.bpsc.2015.12.005</pub-id><pub-id pub-id-type="pmid">27347565</pub-id></citation></ref>
<ref id="B9">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Calhoun</surname> <given-names>V. D.</given-names></name> <name><surname>Sui</surname> <given-names>J.</given-names></name> <name><surname>Kiehl</surname> <given-names>K.</given-names></name> <name><surname>Turner</surname> <given-names>J.</given-names></name> <name><surname>Allen</surname> <given-names>E.</given-names></name> <name><surname>Pearlson</surname> <given-names>G.</given-names></name></person-group> (<year>2011</year>). <article-title>Exploring the psychosis functional connectome: aberrant intrinsic networks in schizophrenia and bipolar disorder</article-title>. <source>Front. Psychiatry</source> <volume>2</volume>:<fpage>75</fpage>. <pub-id pub-id-type="doi">10.3389/fpsyt.2011.00075</pub-id><pub-id pub-id-type="pmid">22291663</pub-id></citation></ref>
<ref id="B10">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Cetin</surname> <given-names>M. S.</given-names></name> <name><surname>Houck</surname> <given-names>J. M.</given-names></name> <name><surname>Rashid</surname> <given-names>B.</given-names></name> <name><surname>Agacoglu</surname> <given-names>O.</given-names></name> <name><surname>Stephen</surname> <given-names>J. M.</given-names></name> <name><surname>Sui</surname> <given-names>J.</given-names></name> <etal/></person-group>. (<year>2016</year>). <article-title>Multimodal classification of schizophrenia patients with MEG and fMRI data using static and dynamic connectivity measures</article-title>. <source>Front. Neurosci</source>. <volume>10</volume>:<fpage>466</fpage>. <pub-id pub-id-type="doi">10.3389/fnins.2016.00466</pub-id><pub-id pub-id-type="pmid">27807403</pub-id></citation></ref>
<ref id="B11">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Colby</surname> <given-names>J. B.</given-names></name> <name><surname>Rudie</surname> <given-names>J. D.</given-names></name> <name><surname>Brown</surname> <given-names>J. A.</given-names></name> <name><surname>Douglas</surname> <given-names>P. K.</given-names></name> <name><surname>Cohen</surname> <given-names>M. S.</given-names></name> <name><surname>Shehzad</surname> <given-names>Z.</given-names></name></person-group> (<year>2012</year>). <article-title>Insights into multimodal imaging classification of ADHD</article-title>. <source>Front. Syst. Neurosci.</source> <volume>6</volume>:<fpage>59</fpage>. <pub-id pub-id-type="doi">10.3389/fnsys.2012.00059</pub-id><pub-id pub-id-type="pmid">22912605</pub-id></citation></ref>
<ref id="B12">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Cortes</surname> <given-names>C.</given-names></name> <name><surname>Vapnik</surname> <given-names>V.</given-names></name></person-group> (<year>1995</year>). <article-title>Support-vector networks</article-title>. <source>Mach. Learn.</source> <volume>20</volume>, <fpage>273</fpage>&#x02013;<lpage>297</lpage>. <pub-id pub-id-type="doi">10.1007/BF00994018</pub-id></citation></ref>
<ref id="B13">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Cox</surname> <given-names>R. W.</given-names></name></person-group> (<year>1996</year>). <article-title>AFNI: software for analysis and visualization of functional magnetic resonance neuroimages</article-title>. <source>Comput. Biomed. Res.</source> <volume>29</volume>, <fpage>162</fpage>&#x02013;<lpage>173</lpage>. <pub-id pub-id-type="doi">10.1006/cbmr.1996.0014</pub-id><pub-id pub-id-type="pmid">8812068</pub-id></citation></ref>
<ref id="B14">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Dale</surname> <given-names>A. M.</given-names></name></person-group> (<year>1999</year>). <article-title>Optimal experimental design for event-related fMRI</article-title>. <source>Hum. Brain Mapp.</source> <volume>8</volume>, <fpage>109</fpage>&#x02013;<lpage>114</lpage>. <pub-id pub-id-type="pmid">10524601</pub-id></citation></ref>
<ref id="B15">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Dale</surname> <given-names>A. M.</given-names></name> <name><surname>Sereno</surname> <given-names>M. I.</given-names></name></person-group> (<year>1993</year>). <article-title>Improved localizadon of cortical activity by combining EEG and MEG with MRI cortical surface reconstruction: a linear approach</article-title>. <source>J. Cogn. Neurosci.</source> <volume>5</volume>, <fpage>162</fpage>&#x02013;<lpage>176</lpage>. <pub-id pub-id-type="doi">10.1162/jocn.1993.5.2.162</pub-id><pub-id pub-id-type="pmid">23972151</pub-id></citation></ref>
<ref id="B16">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Davatzikos</surname> <given-names>C.</given-names></name> <name><surname>Shen</surname> <given-names>D.</given-names></name> <name><surname>Gur</surname> <given-names>R. C.</given-names></name> <name><surname>Wu</surname> <given-names>X.</given-names></name> <name><surname>Liu</surname> <given-names>D.</given-names></name> <name><surname>Fan</surname> <given-names>Y.</given-names></name> <etal/></person-group>. (<year>2005</year>). <article-title>Whole-brain morphometric study of schizophrenia revealing a spatially complex set of focal abnormalities</article-title>. <source>Arch. Gen. Psychiatry</source> <volume>62</volume>, <fpage>1218</fpage>&#x02013;<lpage>1227</lpage>. <pub-id pub-id-type="doi">10.1001/archpsyc.62.11.1218</pub-id><pub-id pub-id-type="pmid">16275809</pub-id></citation></ref>
<ref id="B17">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Davis</surname> <given-names>K. L.</given-names></name> <name><surname>Stewart</surname> <given-names>D. G.</given-names></name> <name><surname>Friedman</surname> <given-names>J. I.</given-names></name> <name><surname>Buchsbaum</surname> <given-names>M.</given-names></name> <name><surname>Harvey</surname> <given-names>P. D.</given-names></name> <name><surname>Hof</surname> <given-names>P. R.</given-names></name> <etal/></person-group>. (<year>2003</year>). <article-title>White matter changes in schizophrenia: evidence for myelin-related dysfunction</article-title>. <source>Arch. Gen. Psychiatry</source> <volume>60</volume>, <fpage>443</fpage>&#x02013;<lpage>456</lpage>. <pub-id pub-id-type="doi">10.1001/archpsyc.60.5.443</pub-id><pub-id pub-id-type="pmid">12742865</pub-id></citation></ref>
<ref id="B18">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Desikan</surname> <given-names>R. S.</given-names></name> <name><surname>S&#x000E9;gonne</surname> <given-names>F.</given-names></name> <name><surname>Fischl</surname> <given-names>B.</given-names></name> <name><surname>Quinn</surname> <given-names>B. T.</given-names></name> <name><surname>Dickerson</surname> <given-names>B. C.</given-names></name> <name><surname>Blacker</surname> <given-names>D.</given-names></name> <etal/></person-group>. (<year>2006</year>). <article-title>An automated labeling system for subdividing the human cerebral cortex on MRI scans into gyral based regions of interest</article-title>. <source>Neuroimage</source> <volume>31</volume>, <fpage>968</fpage>&#x02013;<lpage>980</lpage>. <pub-id pub-id-type="doi">10.1016/j.neuroimage.2006.01.021</pub-id><pub-id pub-id-type="pmid">16530430</pub-id></citation></ref>
<ref id="B19">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Dong</surname> <given-names>D.</given-names></name> <name><surname>Wang</surname> <given-names>Y.</given-names></name> <name><surname>Chang</surname> <given-names>X.</given-names></name> <name><surname>Luo</surname> <given-names>C.</given-names></name> <name><surname>Yao</surname> <given-names>D.</given-names></name></person-group> (<year>2017</year>). <article-title>Dysfunction of large-scale brain networks in schizophrenia: a meta-analysis of resting-state functional connectivity</article-title>. <source>Schizophr. Bull.</source> <pub-id pub-id-type="doi">10.1093/schbul/sbx034</pub-id><pub-id pub-id-type="pmid">28338943</pub-id></citation></ref>
<ref id="B20">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Du</surname> <given-names>W.</given-names></name> <name><surname>Calhoun</surname> <given-names>V. D.</given-names></name> <name><surname>Li</surname> <given-names>H.</given-names></name> <name><surname>Ma</surname> <given-names>S.</given-names></name> <name><surname>Eichele</surname> <given-names>T.</given-names></name> <name><surname>Kiehl</surname> <given-names>K. A.</given-names></name> <etal/></person-group>. (<year>2012</year>). <article-title>High Classification Accuracy for Schizophrenia with Rest and Task fMRI Data</article-title>. <source>Front. Hum. Neurosci.</source> <volume>6</volume>:<fpage>145</fpage>. <pub-id pub-id-type="doi">10.3389/fnhum.2012.00145</pub-id><pub-id pub-id-type="pmid">22675292</pub-id></citation></ref>
<ref id="B21">
<citation citation-type="book"><person-group person-group-type="author"><name><surname>Duda</surname> <given-names>R.</given-names></name> <name><surname>Hart</surname> <given-names>P.</given-names></name></person-group> (<year>1973</year>). <source>Pattern Classification and Scene Analysis</source>. <publisher-loc>New York, NY</publisher-loc>: <publisher-name>Wiley</publisher-name>.</citation></ref>
<ref id="B22">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Fan</surname> <given-names>Y.</given-names></name> <name><surname>Gur</surname> <given-names>R. E.</given-names></name> <name><surname>Gur</surname> <given-names>R. C.</given-names></name> <name><surname>Wu</surname> <given-names>X.</given-names></name> <name><surname>Shen</surname> <given-names>D.</given-names></name> <name><surname>Calkins</surname> <given-names>M. E.</given-names></name> <etal/></person-group>. (<year>2008</year>). <article-title>Unaffected family members and schizophrenia patients share brain structure patterns: a high-dimensional pattern classification study</article-title>. <source>Biol. Psychiatry</source> <volume>63</volume>, <fpage>118</fpage>&#x02013;<lpage>124</lpage>. <pub-id pub-id-type="doi">10.1016/j.biopsych.2007.03.015</pub-id><pub-id pub-id-type="pmid">17555720</pub-id></citation></ref>
<ref id="B23">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Fan</surname> <given-names>Y.</given-names></name> <name><surname>Shen</surname> <given-names>D.</given-names></name> <name><surname>Davatzikos</surname> <given-names>C.</given-names></name></person-group> (<year>2005</year>). <article-title>Classification of structural images via high-dimensional image warping, robust feature extraction, and SVM</article-title>. <source>Med. Image Comput. Comput. Assist. Interv.</source> <volume>8</volume>, <fpage>1</fpage>&#x02013;<lpage>8</lpage>. <pub-id pub-id-type="doi">10.1007/11566465_1</pub-id><pub-id pub-id-type="pmid">16685822</pub-id></citation></ref>
<ref id="B24">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Filippini</surname> <given-names>N.</given-names></name> <name><surname>Macintosh</surname> <given-names>B. J.</given-names></name> <name><surname>Hough</surname> <given-names>M. G.</given-names></name> <name><surname>Goodwin</surname> <given-names>G. M.</given-names></name> <name><surname>Frisoni</surname> <given-names>G. B.</given-names></name> <name><surname>Smith</surname> <given-names>S. M.</given-names></name> <etal/></person-group>. (<year>2009</year>). <article-title>Distinct patterns of brain activity in young carriers of the APOE-&#x003B5;4 allele</article-title>. <source>Proc. Natl. Acad. Sci. U.S.A</source> <volume>106</volume>, <fpage>7209</fpage>&#x02013;<lpage>7214</lpage>. <pub-id pub-id-type="doi">10.1073/pnas.0811879106</pub-id><pub-id pub-id-type="pmid">19357304</pub-id></citation></ref>
<ref id="B25">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Fischl</surname> <given-names>B.</given-names></name> <name><surname>Dale</surname> <given-names>A. M.</given-names></name></person-group> (<year>2000</year>). <article-title>Measuring the thickness of the human cerebral cortex from magnetic resonance images</article-title>. <source>Proc. Natl. Acad. Sci. U.S.A.</source> <volume>97</volume>, <fpage>11050</fpage>&#x02013;<lpage>11055</lpage>. <pub-id pub-id-type="doi">10.1073/pnas.200033797</pub-id><pub-id pub-id-type="pmid">10984517</pub-id></citation></ref>
<ref id="B26">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Fischl</surname> <given-names>B.</given-names></name> <name><surname>Liu</surname> <given-names>A.</given-names></name> <name><surname>Dale</surname> <given-names>A. M.</given-names></name></person-group> (<year>2001</year>). <article-title>Automated manifold surgery: constructing geometrically accurate and topologically correct models of the human cerebral cortex</article-title>. <source>IEEE Trans. Med. Imaging</source> <volume>20</volume>, <fpage>70</fpage>&#x02013;<lpage>80</lpage>. <pub-id pub-id-type="doi">10.1109/42.906426</pub-id><pub-id pub-id-type="pmid">11293693</pub-id></citation></ref>
<ref id="B27">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Fischl</surname> <given-names>B.</given-names></name> <name><surname>Salat</surname> <given-names>D. H.</given-names></name> <name><surname>Busa</surname> <given-names>E.</given-names></name> <name><surname>Albert</surname> <given-names>M.</given-names></name> <name><surname>Dieterich</surname> <given-names>M.</given-names></name> <name><surname>Haselgrove</surname> <given-names>C.</given-names></name> <etal/></person-group>. (<year>2002</year>). <article-title>Whole brain segmentation: automated labeling of neuroanatomical structures in the human brain</article-title>. <source>Neuron</source> <volume>33</volume>, <fpage>341</fpage>&#x02013;<lpage>355</lpage>. <pub-id pub-id-type="doi">10.1016/S0896-6273(02)00569-X</pub-id><pub-id pub-id-type="pmid">11832223</pub-id></citation></ref>
<ref id="B28">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Fischl</surname> <given-names>B.</given-names></name> <name><surname>Salat</surname> <given-names>D. H.</given-names></name> <name><surname>Van Der Kouwe</surname> <given-names>A. J.</given-names></name> <name><surname>Makris</surname> <given-names>N.</given-names></name> <name><surname>S&#x000E9;gonne</surname> <given-names>F.</given-names></name> <name><surname>Quinn</surname> <given-names>B. T.</given-names></name> <etal/></person-group>. (<year>2004</year>). <article-title>Sequence-independent segmentation of magnetic resonance images</article-title>. <source>Neuroimage</source> <volume>23</volume>, <fpage>S69</fpage>&#x02013;<lpage>S84</lpage>. <pub-id pub-id-type="doi">10.1016/j.neuroimage.2004.07.016</pub-id><pub-id pub-id-type="pmid">15501102</pub-id></citation></ref>
<ref id="B29">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Fischl</surname> <given-names>B.</given-names></name> <name><surname>Sereno</surname> <given-names>M. I.</given-names></name> <name><surname>Dale</surname> <given-names>A. M.</given-names></name></person-group> (<year>1999a</year>). <article-title>Cortical surface-based analysis: II: inflation, flattening, and a surface-based coordinate system</article-title>. <source>Neuroimage</source> <volume>9</volume>, <fpage>195</fpage>&#x02013;<lpage>207</lpage>. <pub-id pub-id-type="pmid">9931269</pub-id></citation></ref>
<ref id="B30">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Fischl</surname> <given-names>B.</given-names></name> <name><surname>Sereno</surname> <given-names>M. I.</given-names></name> <name><surname>Tootell</surname> <given-names>R. B.</given-names></name> <name><surname>Dale</surname> <given-names>A. M.</given-names></name></person-group> (<year>1999b</year>). <article-title>High-resolution intersubject averaging and a coordinate system for the cortical surface</article-title>. <source>Hum. Brain Mapp.</source> <volume>8</volume>, <fpage>272</fpage>&#x02013;<lpage>284</lpage>. <pub-id pub-id-type="pmid">10619420</pub-id></citation></ref>
<ref id="B31">
<citation citation-type="book"><person-group person-group-type="author"><name><surname>Golland</surname> <given-names>P.</given-names></name> <name><surname>Fischl</surname> <given-names>B.</given-names></name></person-group> (<year>2003</year>). <article-title>Permutation tests for classification: towards statistical significance in image-based studies</article-title>, in <source>Information Processing in Medical Imaging: 18th International Conference, IPMI 2003</source>, eds <person-group person-group-type="editor"><name><surname>Taylor</surname> <given-names>C.</given-names></name> <name><surname>Noble</surname> <given-names>J. A.</given-names></name></person-group> (<publisher-loc>Berlin; Heidelberg</publisher-loc>: <publisher-name>Springer</publisher-name>), <fpage>330</fpage>&#x02013;<lpage>341</lpage>. <pub-id pub-id-type="pmid">15344469</pub-id></citation></ref>
<ref id="B32">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Gotts</surname> <given-names>S. J.</given-names></name> <name><surname>Simmons</surname> <given-names>W. K.</given-names></name> <name><surname>Milbury</surname> <given-names>L. A.</given-names></name> <name><surname>Wallace</surname> <given-names>G. L.</given-names></name> <name><surname>Cox</surname> <given-names>R. W.</given-names></name> <name><surname>Martin</surname> <given-names>A.</given-names></name></person-group> (<year>2012</year>). <article-title>Fractionation of social brain circuits in autism spectrum disorders</article-title>. <source>Brain</source> <volume>135</volume>(<issue>Pt. 9</issue>), <fpage>2711</fpage>&#x02013;<lpage>2725</lpage>. <pub-id pub-id-type="doi">10.1093/brain/aws160</pub-id><pub-id pub-id-type="pmid">22791801</pub-id></citation></ref>
<ref id="B33">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Han</surname> <given-names>X.</given-names></name> <name><surname>Jovicich</surname> <given-names>J.</given-names></name> <name><surname>Salat</surname> <given-names>D.</given-names></name> <name><surname>Van Der Kouwe</surname> <given-names>A.</given-names></name> <name><surname>Quinn</surname> <given-names>B.</given-names></name> <name><surname>Czanner</surname> <given-names>S.</given-names></name> <etal/></person-group>. (<year>2006</year>). <article-title>Reliability of MRI-derived measurements of human cerebral cortical thickness: the effects of field strength, scanner upgrade and manufacturer</article-title>. <source>Neuroimage</source> <volume>32</volume>, <fpage>180</fpage>&#x02013;<lpage>194</lpage>. <pub-id pub-id-type="doi">10.1016/j.neuroimage.2006.02.051</pub-id><pub-id pub-id-type="pmid">16651008</pub-id></citation></ref>
<ref id="B34">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Hanlon</surname> <given-names>F. M.</given-names></name> <name><surname>Houck</surname> <given-names>J. M.</given-names></name> <name><surname>Pyeatt</surname> <given-names>C. J.</given-names></name> <name><surname>Lundy</surname> <given-names>S. L.</given-names></name> <name><surname>Euler</surname> <given-names>M. J.</given-names></name> <name><surname>Weisend</surname> <given-names>M. P.</given-names></name> <etal/></person-group>. (<year>2011</year>). <article-title>Bilateral hippocampal dysfunction in schizophrenia</article-title>. <source>Neuroimage</source> <volume>58</volume>, <fpage>1158</fpage>&#x02013;<lpage>1168</lpage>. <pub-id pub-id-type="doi">10.1016/j.neuroimage.2011.06.091</pub-id><pub-id pub-id-type="pmid">21763438</pub-id></citation></ref>
<ref id="B35">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Ho</surname> <given-names>B.</given-names></name> <name><surname>Andreasen</surname> <given-names>N. C.</given-names></name> <name><surname>Nopoulos</surname> <given-names>P.</given-names></name> <name><surname>Arndt</surname> <given-names>S.</given-names></name> <name><surname>Magnotta</surname> <given-names>V.</given-names></name> <name><surname>Flaum</surname> <given-names>M.</given-names></name></person-group> (<year>2003</year>). <article-title>Progressive structural brain abnormalities and their relationship to clinical outcome: a longitudinal magnetic resonance imaging study early in schizophrenia</article-title>. <source>Arch. Gen. Psychiatry</source> <volume>60</volume>, <fpage>585</fpage>&#x02013;<lpage>594</lpage>. <pub-id pub-id-type="doi">10.1001/archpsyc.60.6.585</pub-id><pub-id pub-id-type="pmid">12796222</pub-id></citation></ref>
<ref id="B36">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Huang</surname> <given-names>G.-B.</given-names></name> <name><surname>Zhu</surname> <given-names>Q.-Y.</given-names></name> <name><surname>Siew</surname> <given-names>C.-K.</given-names></name></person-group> (<year>2006</year>). <article-title>Extreme learning machine: theory and applications</article-title>. <source>Neurocomputing</source> <volume>70</volume>, <fpage>489</fpage>&#x02013;<lpage>501</lpage>. <pub-id pub-id-type="doi">10.1016/j.neucom.2005.12.126</pub-id></citation></ref>
<ref id="B37">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Karlsgodt</surname> <given-names>K. H.</given-names></name> <name><surname>Sun</surname> <given-names>D.</given-names></name> <name><surname>Cannon</surname> <given-names>T. D.</given-names></name></person-group> (<year>2010</year>). <article-title>Structural and functional brain abnormalities in schizophrenia</article-title>. <source>Curr. Dir. Psychol. Sci.</source> <volume>19</volume>, <fpage>226</fpage>&#x02013;<lpage>231</lpage>. <pub-id pub-id-type="doi">10.1177/0963721410377601</pub-id><pub-id pub-id-type="pmid">25414548</pub-id></citation></ref>
<ref id="B38">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Klein</surname> <given-names>A.</given-names></name> <name><surname>Tourville</surname> <given-names>J.</given-names></name></person-group> (<year>2012</year>). <article-title>101 Labeled brain images and a consistent human cortical labeling protocol</article-title>. <source>Front. Neurosci.</source> <volume>6</volume>:<fpage>171</fpage>. <pub-id pub-id-type="doi">10.3389/fnins.2012.00171</pub-id><pub-id pub-id-type="pmid">23227001</pub-id></citation></ref>
<ref id="B39">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Kubicki</surname> <given-names>M.</given-names></name> <name><surname>Park</surname> <given-names>H.</given-names></name> <name><surname>Westin</surname> <given-names>C.-F.</given-names></name> <name><surname>Nestor</surname> <given-names>P. G.</given-names></name> <name><surname>Mulkern</surname> <given-names>R. V.</given-names></name> <name><surname>Maier</surname> <given-names>S. E.</given-names></name> <etal/></person-group>. (<year>2005</year>). <article-title>DTI and MTR abnormalities in schizophrenia: analysis of white matter integrity</article-title>. <source>Neuroimage</source> <volume>26</volume>, <fpage>1109</fpage>&#x02013;<lpage>1118</lpage>. <pub-id pub-id-type="doi">10.1016/j.neuroimage.2005.03.026</pub-id><pub-id pub-id-type="pmid">15878290</pub-id></citation></ref>
<ref id="B40">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Kuperberg</surname> <given-names>G. R.</given-names></name> <name><surname>Broome</surname> <given-names>M. R.</given-names></name> <name><surname>Mcguire</surname> <given-names>P. K.</given-names></name> <name><surname>David</surname> <given-names>A. S.</given-names></name> <name><surname>Eddy</surname> <given-names>M.</given-names></name> <name><surname>Ozawa</surname> <given-names>F.</given-names></name> <etal/></person-group>. (<year>2003</year>). <article-title>Regionally localized thinning of the cerebral cortex in schizophrenia</article-title>. <source>Arch. Gen. Psychiatry</source> <volume>60</volume>, <fpage>878</fpage>&#x02013;<lpage>888</lpage>. <pub-id pub-id-type="doi">10.1001/archpsyc.60.9.878</pub-id><pub-id pub-id-type="pmid">12963669</pub-id></citation></ref>
<ref id="B41">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Larivi&#x000E8;re</surname> <given-names>S.</given-names></name> <name><surname>Lavigne</surname> <given-names>K. M.</given-names></name> <name><surname>Woodward</surname> <given-names>T. S.</given-names></name> <name><surname>Gerretsen</surname> <given-names>P.</given-names></name> <name><surname>Graff-Guerrero</surname> <given-names>A.</given-names></name> <name><surname>Menon</surname> <given-names>M.</given-names></name></person-group> (<year>2017</year>). <article-title>Altered functional connectivity in brain networks underlying self-referential processing in delusions of reference in schizophrenia</article-title>. <source>Psychiatry Res. Neuroimaging</source> <volume>263</volume>, <fpage>32</fpage>&#x02013;<lpage>43</lpage>. <pub-id pub-id-type="doi">10.1016/j.pscychresns.2017.03.005</pub-id><pub-id pub-id-type="pmid">28315577</pub-id></citation></ref>
<ref id="B42">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Li</surname> <given-names>G.</given-names></name> <name><surname>Wang</surname> <given-names>L.</given-names></name> <name><surname>Shi</surname> <given-names>F.</given-names></name> <name><surname>Lyall</surname> <given-names>A. E.</given-names></name> <name><surname>Ahn</surname> <given-names>M.</given-names></name> <name><surname>Peng</surname> <given-names>Z.</given-names></name> <etal/></person-group>. (<year>2016</year>). <article-title>Cortical thickness and surface area in neonates at high risk for schizophrenia</article-title>. <source>Brain Struct. Funct.</source> <volume>221</volume>, <fpage>447</fpage>&#x02013;<lpage>461</lpage>. <pub-id pub-id-type="doi">10.1007/s00429-014-0917-3</pub-id><pub-id pub-id-type="pmid">25362539</pub-id></citation></ref>
<ref id="B43">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Liaw</surname> <given-names>A.</given-names></name> <name><surname>Wiener</surname> <given-names>M.</given-names></name></person-group> (<year>2002</year>). <article-title>Classification and regression by randomForest</article-title>. <source>R News</source> <volume>2</volume>, <fpage>18</fpage>&#x02013;<lpage>22</lpage>.</citation></ref>
<ref id="B44">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Littow</surname> <given-names>H.</given-names></name> <name><surname>Elseoud</surname> <given-names>A. A.</given-names></name> <name><surname>Haapea</surname> <given-names>M.</given-names></name> <name><surname>Isohanni</surname> <given-names>M.</given-names></name> <name><surname>Moilanen</surname> <given-names>I.</given-names></name> <name><surname>Mankinen</surname> <given-names>K.</given-names></name> <etal/></person-group>. (<year>2010</year>). <article-title>Age-related differences in functional nodes of the brain cortex&#x02013;a high model order group ICA study</article-title>. <source>Front. Syst. Neurosci.</source> <volume>4</volume>:<fpage>32</fpage>. <pub-id pub-id-type="doi">10.3389/fnsys.2010.00032</pub-id><pub-id pub-id-type="pmid">20953235</pub-id></citation></ref>
<ref id="B45">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Mao</surname> <given-names>Y.</given-names></name> <name><surname>Ge</surname> <given-names>X.</given-names></name> <name><surname>Frank</surname> <given-names>C. L.</given-names></name> <name><surname>Madison</surname> <given-names>J. M.</given-names></name> <name><surname>Koehler</surname> <given-names>A. N.</given-names></name> <name><surname>Doud</surname> <given-names>M. K.</given-names></name> <etal/></person-group>. (<year>2009</year>). <article-title>Disrupted in schizophrenia 1 regulates neuronal progenitor proliferation via modulation of gsk3&#x003B2;/&#x003B2;-catenin signaling</article-title>. <source>Cell</source> <volume>136</volume>, <fpage>1017</fpage>&#x02013;<lpage>1031</lpage>. <pub-id pub-id-type="doi">10.1016/j.cell.2008.12.044</pub-id><pub-id pub-id-type="pmid">19303846</pub-id></citation></ref>
<ref id="B46">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Massey</surname> <given-names>S. H.</given-names></name> <name><surname>Stern</surname> <given-names>D.</given-names></name> <name><surname>Alden</surname> <given-names>E. C.</given-names></name> <name><surname>Petersen</surname> <given-names>J. E.</given-names></name> <name><surname>Cobia</surname> <given-names>D. J.</given-names></name> <name><surname>Wang</surname> <given-names>L.</given-names></name> <etal/></person-group>. (<year>2017</year>). <article-title>Cortical thickness of neural substrates supporting cognitive empathy in individuals with schizophrenia</article-title>. <source>Schizophr. Res.</source> <volume>179</volume>, <fpage>119</fpage>&#x02013;<lpage>124</lpage>. <pub-id pub-id-type="doi">10.1016/j.schres.2016.09.025</pub-id><pub-id pub-id-type="pmid">27665257</pub-id></citation></ref>
<ref id="B47">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Mayer</surname> <given-names>A. R.</given-names></name> <name><surname>Ruhl</surname> <given-names>D.</given-names></name> <name><surname>Merideth</surname> <given-names>F.</given-names></name> <name><surname>Ling</surname> <given-names>J.</given-names></name> <name><surname>Hanlon</surname> <given-names>F. M.</given-names></name> <name><surname>Bustillo</surname> <given-names>J.</given-names></name> <etal/></person-group>. (<year>2013</year>). <article-title>Functional imaging of the hemodynamic sensory gating response in schizophrenia</article-title>. <source>Hum. Brain Mapp.</source> <volume>34</volume>, <fpage>2302</fpage>&#x02013;<lpage>2312</lpage>. <pub-id pub-id-type="doi">10.1002/hbm.22065</pub-id><pub-id pub-id-type="pmid">22461278</pub-id></citation></ref>
<ref id="B48">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Mirzaei</surname> <given-names>G.</given-names></name> <name><surname>Adeli</surname> <given-names>A.</given-names></name> <name><surname>Adeli</surname> <given-names>H.</given-names></name></person-group> (<year>2016</year>). <article-title>Imaging and machine learning techniques for diagnosis of Alzheimer&#x00027;s disease</article-title>. <source>Rev. Neurosci.</source> <volume>27</volume>, <fpage>857</fpage>&#x02013;<lpage>870</lpage>. <pub-id pub-id-type="doi">10.1515/revneuro-2016-0029</pub-id><pub-id pub-id-type="pmid">27518905</pub-id></citation></ref>
<ref id="B49">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>M&#x000F8;rch-Johnsen</surname> <given-names>L.</given-names></name> <name><surname>Nesv&#x000E5;g</surname> <given-names>R.</given-names></name> <name><surname>J&#x000F8;rgensen</surname> <given-names>K. N.</given-names></name> <name><surname>Lange</surname> <given-names>E. H.</given-names></name> <name><surname>Hartberg</surname> <given-names>C. B.</given-names></name> <name><surname>Haukvik</surname> <given-names>U. K.</given-names></name> <etal/></person-group>. (<year>2017</year>). <article-title>Auditory cortex characteristics in schizophrenia: associations with auditory hallucinations</article-title>. <source>Schizophr. Bull.</source> <volume>43</volume>, <fpage>75</fpage>&#x02013;<lpage>83</lpage>. <pub-id pub-id-type="doi">10.1093/schbul/sbw130</pub-id><pub-id pub-id-type="pmid">27605526</pub-id></citation></ref>
<ref id="B50">
<citation citation-type="book"><person-group person-group-type="author"><name><surname>Naveed</surname> <given-names>S.</given-names></name></person-group> (<year>2014</year>). <source>Kalman Filter Approach for Mobile Robot Localization Fusing Ultrasonic Sensor and Laser Range Finder Measurements</source>. Master in Engineering, <publisher-name>Chosun University</publisher-name>.</citation></ref>
<ref id="B51">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Nieuwenhuis</surname> <given-names>M.</given-names></name> <name><surname>Van Haren</surname> <given-names>N. E.</given-names></name> <name><surname>Hulshoff Pol</surname> <given-names>H. E.</given-names></name> <name><surname>Cahn</surname> <given-names>W.</given-names></name> <name><surname>Kahn</surname> <given-names>R. S.</given-names></name> <name><surname>Schnack</surname> <given-names>H. G.</given-names></name></person-group> (<year>2012</year>). <article-title>Classification of schizophrenia patients and healthy controls from structural MRI scans in two large independent samples</article-title>. <source>Neuroimage</source> <volume>61</volume>, <fpage>606</fpage>&#x02013;<lpage>612</lpage>. <pub-id pub-id-type="doi">10.1016/j.neuroimage.2012.03.079</pub-id><pub-id pub-id-type="pmid">22507227</pub-id></citation></ref>
<ref id="B52">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Oh</surname> <given-names>J.</given-names></name> <name><surname>Chun</surname> <given-names>J.-W.</given-names></name> <name><surname>Joon Jo</surname> <given-names>H.</given-names></name> <name><surname>Kim</surname> <given-names>E.</given-names></name> <name><surname>Park</surname> <given-names>H.-J.</given-names></name> <name><surname>Lee</surname> <given-names>B.</given-names></name> <etal/></person-group>. (<year>2015</year>). <article-title>The neural basis of a deficit in abstract thinking in patients with schizophrenia</article-title>. <source>Psychiatry Res. Neuroimaging</source> <volume>234</volume>, <fpage>66</fpage>&#x02013;<lpage>73</lpage>. <pub-id pub-id-type="doi">10.1016/j.pscychresns.2015.08.007</pub-id><pub-id pub-id-type="pmid">26329118</pub-id></citation></ref>
<ref id="B53">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Oh</surname> <given-names>J.</given-names></name> <name><surname>Chun</surname> <given-names>J. W.</given-names></name> <name><surname>Kim</surname> <given-names>E.</given-names></name> <name><surname>Park</surname> <given-names>H. J.</given-names></name> <name><surname>Lee</surname> <given-names>B.</given-names></name> <name><surname>Kim</surname> <given-names>J. J.</given-names></name></person-group> (<year>2017</year>). <article-title>Aberrant neural networks for the recognition memory of socially relevant information in patients with schizophrenia</article-title>. <source>Brain Behav.</source> <volume>7</volume>:<fpage>e00602</fpage>. <pub-id pub-id-type="doi">10.1002/brb3.602</pub-id><pub-id pub-id-type="pmid">28127520</pub-id></citation></ref>
<ref id="B54">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Ota</surname> <given-names>M.</given-names></name> <name><surname>Ishikawa</surname> <given-names>M.</given-names></name> <name><surname>Sato</surname> <given-names>N.</given-names></name> <name><surname>Hori</surname> <given-names>H.</given-names></name> <name><surname>Sasayama</surname> <given-names>D.</given-names></name> <name><surname>Hattori</surname> <given-names>K.</given-names></name> <etal/></person-group>. (<year>2013</year>). <article-title>Discrimination between schizophrenia and major depressive disorder by magnetic resonance imaging of the female brain</article-title>. <source>J. Psychiatr. Res.</source> <volume>47</volume>, <fpage>1383</fpage>&#x02013;<lpage>1388</lpage>. <pub-id pub-id-type="doi">10.1016/j.jpsychires.2013.06.010</pub-id><pub-id pub-id-type="pmid">23830450</pub-id></citation></ref>
<ref id="B55">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Peters</surname> <given-names>H.</given-names></name> <name><surname>Riedl</surname> <given-names>V.</given-names></name> <name><surname>Manoliu</surname> <given-names>A.</given-names></name> <name><surname>Scherr</surname> <given-names>M.</given-names></name> <name><surname>Schwerth&#x000F6;ffer</surname> <given-names>D.</given-names></name> <name><surname>Zimmer</surname> <given-names>C.</given-names></name> <etal/></person-group>. (<year>2017</year>). <article-title>Changes in extra-striatal functional connectivity in patients with schizophrenia in a psychotic episode</article-title>. <source>Br. J. Psychiatry</source> <volume>210</volume>, <fpage>75</fpage>&#x02013;<lpage>82</lpage>. <pub-id pub-id-type="doi">10.1192/bjp.bp.114.151928</pub-id><pub-id pub-id-type="pmid">26892851</pub-id></citation></ref>
<ref id="B56">
<citation citation-type="book"><person-group person-group-type="author"><name><surname>Qureshi</surname> <given-names>M. N. I.</given-names></name></person-group> (<year>2013</year>). <source>RGB-D Fusion Based on Facial Normal Vectors Using 3D Point Cloud Master in Engineering.</source> <publisher-name>Chosun Univeristy</publisher-name>, <publisher-loc>South Korea</publisher-loc>.</citation></ref>
<ref id="B57">
<citation citation-type="book"><person-group person-group-type="author"><name><surname>Qureshi</surname> <given-names>M. N. I.</given-names></name> <name><surname>Jo</surname> <given-names>H. J.</given-names></name> <name><surname>Lee</surname> <given-names>B.</given-names></name></person-group> (<year>2017a</year>). <article-title>ADHD subgroup discrimination with global connectivity features using hierarchical extreme learning machine: resting-state FMRI study</article-title>, in <source>2017 IEEE 14th International Symposium on Biomedical Imaging (ISBI 2017)</source> (<publisher-loc>Melbourne, VIC</publisher-loc>), <fpage>529</fpage>&#x02013;<lpage>532</lpage>.</citation></ref>
<ref id="B58">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Qureshi</surname> <given-names>M. N. I.</given-names></name> <name><surname>Min</surname> <given-names>B.</given-names></name> <name><surname>Jo</surname> <given-names>H. J.</given-names></name> <name><surname>Lee</surname> <given-names>B.</given-names></name></person-group> (<year>2016</year>). <article-title>Multiclass classification for the differential diagnosis on the adhd subtypes using recursive feature elimination and hierarchical extreme learning machine: structural MRI Study</article-title>. <source>PLoS ONE</source> <volume>11</volume>:<fpage>e0160697</fpage>. <pub-id pub-id-type="doi">10.1371/journal.pone.0160697</pub-id><pub-id pub-id-type="pmid">27500640</pub-id></citation></ref>
<ref id="B59">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Qureshi</surname> <given-names>M. N. I.</given-names></name> <name><surname>Oh</surname> <given-names>J.</given-names></name> <name><surname>Min</surname> <given-names>B.</given-names></name> <name><surname>Jo</surname> <given-names>H. J.</given-names></name> <name><surname>Lee</surname> <given-names>B.</given-names></name></person-group> (<year>2017b</year>). <article-title>Multi-modal, multi-measure, and multi-class discrimination of ADHD with hierarchical feature extraction and extreme learning machine using structural and functional brain MRI</article-title>. <source>Front. Hum. Neurosci.</source> <volume>11</volume>:<fpage>157</fpage>. <pub-id pub-id-type="doi">10.3389/fnhum.2017.00157</pub-id><pub-id pub-id-type="pmid">28420972</pub-id></citation></ref>
<ref id="B60">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Reuter</surname> <given-names>M.</given-names></name> <name><surname>Schmansky</surname> <given-names>N. J.</given-names></name> <name><surname>Rosas</surname> <given-names>H. D.</given-names></name> <name><surname>Fischl</surname> <given-names>B.</given-names></name></person-group> (<year>2012</year>). <article-title>Within-subject template estimation for unbiased longitudinal image analysis</article-title>. <source>Neuroimage</source> <volume>61</volume>, <fpage>1402</fpage>&#x02013;<lpage>1418</lpage>. <pub-id pub-id-type="doi">10.1016/j.neuroimage.2012.02.084</pub-id><pub-id pub-id-type="pmid">22430496</pub-id></citation></ref>
<ref id="B61">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Rosas</surname> <given-names>H.</given-names></name> <name><surname>Liu</surname> <given-names>A.</given-names></name> <name><surname>Hersch</surname> <given-names>S.</given-names></name> <name><surname>Glessner</surname> <given-names>M.</given-names></name> <name><surname>Ferrante</surname> <given-names>R.</given-names></name> <name><surname>Salat</surname> <given-names>D.</given-names></name> <etal/></person-group>. (<year>2002</year>). <article-title>Regional and progressive thinning of the cortical ribbon in huntington&#x00027;s disease</article-title>. <source>Neurology</source> <volume>58</volume>, <fpage>695</fpage>&#x02013;<lpage>701</lpage>. <pub-id pub-id-type="doi">10.1212/WNL.58.5.695</pub-id><pub-id pub-id-type="pmid">11889230</pub-id></citation></ref>
<ref id="B62">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Saad</surname> <given-names>Z. S.</given-names></name> <name><surname>Reynolds</surname> <given-names>R. C.</given-names></name> <name><surname>Jo</surname> <given-names>H. J.</given-names></name> <name><surname>Gotts</surname> <given-names>S. J.</given-names></name> <name><surname>Chen</surname> <given-names>G.</given-names></name> <name><surname>Martin</surname> <given-names>A.</given-names></name> <etal/></person-group>. (<year>2013</year>). <article-title>Correcting brain-wide correlation differences in resting-state FMRI</article-title>. <source>Brain Connect.</source> <volume>3</volume>, <fpage>339</fpage>&#x02013;<lpage>352</lpage>. <pub-id pub-id-type="doi">10.1089/brain.2013.0156</pub-id><pub-id pub-id-type="pmid">23705677</pub-id></citation></ref>
<ref id="B63">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Salat</surname> <given-names>D. H.</given-names></name> <name><surname>Buckner</surname> <given-names>R. L.</given-names></name> <name><surname>Snyder</surname> <given-names>A. Z.</given-names></name> <name><surname>Greve</surname> <given-names>D. N.</given-names></name> <name><surname>Desikan</surname> <given-names>R. S.</given-names></name> <name><surname>Busa</surname> <given-names>E.</given-names></name> <etal/></person-group>. (<year>2004</year>). <article-title>Thinning of the cerebral cortex in aging</article-title>. <source>Cereb. Cortex</source> <volume>14</volume>, <fpage>721</fpage>&#x02013;<lpage>730</lpage>. <pub-id pub-id-type="doi">10.1093/cercor/bhh032</pub-id><pub-id pub-id-type="pmid">15054051</pub-id></citation></ref>
<ref id="B64">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Schnack</surname> <given-names>H. G.</given-names></name> <name><surname>Nieuwenhuis</surname> <given-names>M.</given-names></name> <name><surname>Van Haren</surname> <given-names>N. E.</given-names></name> <name><surname>Abramovic</surname> <given-names>L.</given-names></name> <name><surname>Scheewe</surname> <given-names>T. W.</given-names></name> <name><surname>Brouwer</surname> <given-names>R. M.</given-names></name> <etal/></person-group>. (<year>2014</year>). <article-title>Can structural MRI aid in clinical classification? A machine learning study in two <italic>independent samples of patients with schizophrenia, bipolar disorder and healthy subjects</italic></article-title>. <source>Neuroimage</source> <volume>84</volume>, <fpage>299</fpage>&#x02013;<lpage>306</lpage>. <pub-id pub-id-type="doi">10.1016/j.neuroimage.2013.08.053</pub-id><pub-id pub-id-type="pmid">24004694</pub-id></citation></ref>
<ref id="B65">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>S&#x000E9;gonne</surname> <given-names>F.</given-names></name> <name><surname>Dale</surname> <given-names>A. M.</given-names></name> <name><surname>Busa</surname> <given-names>E.</given-names></name> <name><surname>Glessner</surname> <given-names>M.</given-names></name> <name><surname>Salat</surname> <given-names>D.</given-names></name> <name><surname>Hahn</surname> <given-names>H. K.</given-names></name> <etal/></person-group>. (<year>2004</year>). <article-title>A hybrid approach to the skull stripping problem in MRI</article-title>. <source>Neuroimage</source> <volume>22</volume>, <fpage>1060</fpage>&#x02013;<lpage>1075</lpage>. <pub-id pub-id-type="doi">10.1016/j.neuroimage.2004.03.032</pub-id><pub-id pub-id-type="pmid">15219578</pub-id></citation></ref>
<ref id="B66">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>S&#x000E9;gonne</surname> <given-names>F.</given-names></name> <name><surname>Pacheco</surname> <given-names>J.</given-names></name> <name><surname>Fischl</surname> <given-names>B.</given-names></name></person-group> (<year>2007</year>). <article-title>Geometrically accurate topology-correction of cortical surfaces using nonseparating loops</article-title>. <source>IEEE Trans. Med. Imaging</source> <volume>26</volume>, <fpage>518</fpage>&#x02013;<lpage>529</lpage>. <pub-id pub-id-type="doi">10.1109/TMI.2006.887364</pub-id><pub-id pub-id-type="pmid">17427739</pub-id></citation></ref>
<ref id="B67">
<citation citation-type="book"><person-group person-group-type="author"><name><surname>Silva</surname> <given-names>R. F.</given-names></name> <name><surname>Castro</surname> <given-names>E.</given-names></name> <name><surname>Gupta</surname> <given-names>C. N.</given-names></name> <name><surname>Cetin</surname> <given-names>M.</given-names></name> <name><surname>Arbabshirani</surname> <given-names>M.</given-names></name> <name><surname>Potluru</surname> <given-names>V. K.</given-names></name> <etal/></person-group>. (<year>2014</year>). <article-title>The tenth annual MLSP competition: schizophrenia classification challenge</article-title>, in <source>2014 IEEE International Workshop on Machine Learning for Signal Processing, MLSP</source> (<publisher-loc>Reims</publisher-loc>), <fpage>1</fpage>&#x02013;<lpage>6</lpage>.</citation></ref>
<ref id="B68">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Smith</surname> <given-names>S. M.</given-names></name> <name><surname>Jenkinson</surname> <given-names>M.</given-names></name> <name><surname>Woolrich</surname> <given-names>M. W.</given-names></name> <name><surname>Beckmann</surname> <given-names>C. F.</given-names></name> <name><surname>Behrens</surname> <given-names>T. E.</given-names></name> <name><surname>Johansen-Berg</surname> <given-names>H.</given-names></name> <etal/></person-group>. (<year>2004</year>). <article-title>Advances in functional and structural MR image analysis and implementation as FSL</article-title>. <source>Neuroimage</source> <volume>23</volume>, <fpage>S208</fpage>&#x02013;<lpage>S219</lpage>. <pub-id pub-id-type="doi">10.1016/j.neuroimage.2004.07.051</pub-id><pub-id pub-id-type="pmid">15501092</pub-id></citation></ref>
<ref id="B69">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Stephen</surname> <given-names>J. M.</given-names></name> <name><surname>Coffman</surname> <given-names>B. A.</given-names></name> <name><surname>Jung</surname> <given-names>R. E.</given-names></name> <name><surname>Bustillo</surname> <given-names>J. R.</given-names></name> <name><surname>Aine</surname> <given-names>C. J.</given-names></name> <name><surname>Calhoun</surname> <given-names>V. D.</given-names></name></person-group> (<year>2013</year>). <article-title>Using joint ICA to link function and structure using MEG and DTI in schizophrenia</article-title>. <source>NeuroImage</source> <volume>83</volume>, <fpage>418</fpage>&#x02013;<lpage>430</lpage>. <pub-id pub-id-type="doi">10.1016/j.neuroimage.2013.06.038</pub-id><pub-id pub-id-type="pmid">23777757</pub-id></citation></ref>
<ref id="B70">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Sui</surname> <given-names>J.</given-names></name> <name><surname>He</surname> <given-names>H.</given-names></name> <name><surname>Yu</surname> <given-names>Q.</given-names></name> <name><surname>Chen</surname> <given-names>J.</given-names></name> <name><surname>Rogers</surname> <given-names>J.</given-names></name> <name><surname>Pearlson</surname> <given-names>G. D.</given-names></name> <etal/></person-group>. (<year>2013</year>). <article-title>Combination of resting state fMRI, DTI, and sMRI data to discriminate schizophrenia by N-way MCCA &#x0002B; jICA</article-title>. <source>Front. Hum. Neurosci.</source> <volume>7</volume>:<fpage>235</fpage>. <pub-id pub-id-type="doi">10.3389/fnhum.2013.00235</pub-id><pub-id pub-id-type="pmid">23755002</pub-id></citation></ref>
<ref id="B71">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Termenon</surname> <given-names>M.</given-names></name> <name><surname>Grana</surname> <given-names>M.</given-names></name> <name><surname>Besga</surname> <given-names>A.</given-names></name> <name><surname>Echeveste</surname> <given-names>J.</given-names></name> <name><surname>Gonzalez-Pinto</surname> <given-names>A.</given-names></name></person-group> (<year>2013</year>). <article-title>Lattice independent component analysis feature selection on diffusion weighted imaging for Alzheimer&#x00027;s disease classification</article-title>. <source>Neurocomputing</source> <volume>114</volume>, <fpage>132</fpage>&#x02013;<lpage>141</lpage>. <pub-id pub-id-type="doi">10.1016/j.neucom.2012.08.044</pub-id></citation></ref>
<ref id="B72">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Veer</surname> <given-names>I. M.</given-names></name> <name><surname>Beckmann</surname> <given-names>C.</given-names></name> <name><surname>Van Tol</surname> <given-names>M.-J.</given-names></name> <name><surname>Ferrarini</surname> <given-names>L.</given-names></name> <name><surname>Milles</surname> <given-names>J.</given-names></name> <name><surname>Veltman</surname> <given-names>D.</given-names></name> <etal/></person-group>. (<year>2010</year>). <article-title>Whole brain resting-state analysis reveals decreased functional connectivity in major depression</article-title>. <source>Front. Syst. Neurosci.</source> <volume>4</volume>:<fpage>41</fpage>. <pub-id pub-id-type="doi">10.3389/fnsys.2010.00041</pub-id><pub-id pub-id-type="pmid">20941370</pub-id></citation></ref>
<ref id="B73">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Yang</surname> <given-names>H.</given-names></name> <name><surname>Liu</surname> <given-names>J.</given-names></name> <name><surname>Sui</surname> <given-names>J.</given-names></name> <name><surname>Pearlson</surname> <given-names>G.</given-names></name> <name><surname>Calhoun</surname> <given-names>V. D.</given-names></name></person-group> (<year>2010</year>). <article-title>A hybrid machine learning method for fusing fMRI and genetic data: combining both improves classification of schizophrenia</article-title>. <source>Front. Hum. Neurosci</source>. <volume>4</volume>:<fpage>192</fpage>. <pub-id pub-id-type="doi">10.3389/fnhum.2010.00192</pub-id><pub-id pub-id-type="pmid">21119772</pub-id></citation></ref>
<ref id="B74">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Zanetti</surname> <given-names>M. V.</given-names></name> <name><surname>Schaufelberger</surname> <given-names>M. S.</given-names></name> <name><surname>Doshi</surname> <given-names>J.</given-names></name> <name><surname>Ou</surname> <given-names>Y.</given-names></name> <name><surname>Ferreira</surname> <given-names>L. K.</given-names></name> <name><surname>Menezes</surname> <given-names>P. R.</given-names></name> <etal/></person-group>. (<year>2013</year>). <article-title>Neuroanatomical pattern classification in a population-based sample of first-episode schizophrenia</article-title>. <source>Prog. Neuro Psychopharmacol. Biol. Psychiatry</source> <volume>43</volume>, <fpage>116</fpage>&#x02013;<lpage>125</lpage>. <pub-id pub-id-type="doi">10.1016/j.pnpbp.2012.12.005</pub-id><pub-id pub-id-type="pmid">23261522</pub-id></citation></ref>
<ref id="B75">
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Zhang</surname> <given-names>L.</given-names></name> <name><surname>Zhang</surname> <given-names>D.</given-names></name></person-group> (<year>2015</year>). <article-title>Domain adaptation extreme learning machines for drift compensation in E-nose systems</article-title>. <source>IEEE Trans. Instrum. Meas.</source> <volume>64</volume>, <fpage>1790</fpage>&#x02013;<lpage>1801</lpage> <pub-id pub-id-type="doi">10.1109/TIM.2014.2367775</pub-id></citation></ref>
</ref-list>
<fn-group>
<fn fn-type="financial-disclosure"><p><bold>Funding.</bold> This research was also supported by the Pioneer Research Center Program through the National Research Foundation of Korea funded by the Ministry of Science, ICT &#x00026; Future Planning (grant no. 2012-0009462). This work was also supported by the GIST Research Institute (GRI) in 2017.</p>
</fn>
</fn-group>
</back>
</article>
