A collection of resting-state fMRI data, called fmri-open-qc-rest, was used for demonstrating the QC procedure in DPABI. The fmri-open-qc-rest dataset includes participants pooled from 7 different datasets, each with about 20 subjects (total N = 139). It’s a demonstrating data of the fMRI Open QC Project and the anonymous samples were selected from widely-used open-available datasets such as the functional connectome project (FCP) (Biswal et al., 2010), the autism brain imaging data exchange (ABIDE) (Di Martino et al., 2014) and the OpenNeuro resource (Markiewicz et al., 2021). The sex and age of participants were not available in the fmri-open-qc-rest dataset.

Both a volume-based pipeline (DPARSF) and a surface-based pipeline (DPABISurf) in DPABI were used to preprocess the MRI data. Surface-based methods are increasingly common in the most recent studies and are superior to volume-based methods in terms of structure localization, spatial smoothing, and reproducibility (Coalson et al., 2018). However, the surface-based methods were time-consuming and omitted the analysis of subcortical and cerebellar areas. The volume-based approaches would be appropriate for conducting whole-brain analysis, preprocessing large datasets, etc. Additionally, the DPARSF pipelines reorient/QC module offered a user-friendly graphical user interface for visually assessing the image quality before the remaining laborious stages (e.g., structure segmentation).

In specific, surface-based preprocessing was performed by DPABISurf (Yan et al., 2021), a surface-based fMRI data analysis toolbox evolved from DPABI/DPARSF. DPABISurf used docker technology to wrap the whole computing environment for fMRIPrep (Esteban et al., 2019), FreeSurfer (Fischl, 2012), ANTs (Tustison et al., 2014), FSL (Jenkinson et al., 2012), AFNI (Cox, 1996), SPM (Ashburner, 2012), GNU Parallel (Tange, 2011), PALM (Winkler et al., 2014), MATLAB (The MathWorks Inc., Natick, MA, USA), Docker¹ and DPABI (Yan et al., 2016), etc. The pipelines mentioned above have their own preprocessing and QC procedures and an elaborate comparison among these pipelines could be found in the ENIGMA HALFpipe references (Waller et al., 2022). The resting-state functional images and T1-weighted images were preprocessed by the following steps. (1) Checking the BIDS JSON-format image meta-data; (2) intensity non-uniformity correction and skull-stripping; (3) tissue segmentation of cerebrospinal fluid (CSF), white matter (WM), and gray matter (GM); (4) brain surface reconstruction; (5) deleting initial 10 time points; (6) boundary-based registration of BOLD and T1 images; (7) BOLD image spatial normalization to fsaverage5 space; (8) head-motion, WM, and CSF signal and linear trend nuisance regression; (9) bandpass filtering (0.01–0.1 Hz); (10) spatial smoothing [full-width at half-maximum (FWHM) of 6 mm]. Detailed preprocessing procedures can be found in our previous research (Chen and Yan, 2021).

Of note, slice-timing corrections were not conducted because there were errors in the slice-timing information of some participants. Normally, DPABISurf/DPARSF would read the slice-timing information from DICOM header files (if the input images were in DICOM format) and metadata files in the BIDS format or the DPABI format (if the input images were in NIFTI format). As the demonstrating data in the fmri-open-qc-rest dataset were in NIFTI format, the slice-timing correction procedures would use the related metadata in the BIDS schema. The related information such as acquisition time for each slice and the scanning sequence (e.g., interleave or sequence while scanning different slices in a volume) were recorded in separated JSON files in the BIDS data-structure and could not be extracted from the NIFTI images themselves. In the fmri-open-qc-rest dataset, slice-timing-related information of some participants was missing or incorrect. The exact details were provided in see Section “3.2. Issues in MRI meta-data.” Therefore, we skipped the slice-timing correction while this procedure might be necessary for the images with a relatively long repetition time (Sladky et al., 2011) (e.g., TR = 2.5 for most of the participants in the dataset).

Volume-based data preprocessing in our study was carried out using the Data Processing Assistant for resting-state fMRI (DPARSF) (Yan and Zang, 2010), which was based on SPM (Friston et al., 1994) and had been integrated into Data Processing and Analysis of Brain Imaging (DPABI) (Yan et al., 2016). The first 10 time-points of the fMRI series were discarded. The head motion was corrected by a six-parameter (rigid body) linear transformation with a two-pass procedure (Yan et al., 2013). Reorient/QC was a module in DPARSF pipeline for both adjusting the orientation of the images and visually checking the image quality of each T1-weighted or BOLD image. We rated each image by a 5-point scale. The 5-point rating scales provided semiquantitative scores for the results of the visually evaluation in reorient/QC module. More points equaled better images. The derived reports would record both the rating scores and the comments for images. After the whole Reorient/QC procedures were finished, a QC-score-threshold of 3 was set in the following dialog box. The images with extremely bad quality were not be involved in the further preprocessing procedure to avoid contaminating other samples in the certain procedures (e.g., creating a group template). After coregistering the structural and functional images and unified segmentation (Ashburner and Friston, 2005) on T1 image, spatial normalization to MNI-152 space [a coordinate system created by Montreal Neurological Institute (Fonov et al., 2009)] was performed using the Diffeomorphic Anatomical Registration Through Exponentiated Lie algebra (DARTEL) tool (Goto et al., 2013). The Friston 24-parameter model (Friston et al., 1996) was applied to regress out head motion effects. White matter signal, cerebrospinal fluid signal and linear trends were regressed out from each voxel’s time course. Finally, all images were filtered by temporal bandpass filtering (0.01–0.1 Hz) to reduce the effect of low-frequency drift and high-frequency physiological noise.

In general, we adopted six DPABI-derived reports to exclude participants with insufficient quality. The detailed criteria according to the reports were listed in Table 1. The QC procedures were integrated into two pipelines with graphic user interfaces (GUI) for the volume-based methods and surface-based methods. A detailed introduction to these modules could be found in the related course at http://rfmri.org/Course. An intuitive exclusive tool for checking spatial normalization quality in the volume-based preprocessing was displayed in Figure 1. The detailed criteria for eliminating samples derived from these reports were listed in Table 1.

To preliminarily illustrate the effect of quality control in statistical analysis, we conducted two-sample t-tests to show the sex differences in some common fMRI metrics. Of note, a comprehensive evaluation of the QC-effect in group-level analysis (e.g., taking into account the site-effect and the reduced sample size after eliminating samples) would be a larger and separate topic. Importantly, the sex labels of the participants were not provided by the organizers of fmri-open-qc-rest dataset and we used a T1-weighted image-based classifier to predict the sex of each participant (Lu et al., 2022). Considering the sex classifier achieved about 95% accuracy, we supposed that the estimated classifier values would be close to the ground truth. Sex differences were tested in both the images with QC and the images without any QC. For the statistics without QC, thirteen estimated male participants and seven estimated female participants were excluded. The fMRI metrics included regional homogeneity (ReHo), (fractional) amplitude of low-frequency fluctuations (fALFF/ALFF) and degree centrality (DC). The sites and the mean FD-Jenkinson scores were included as covariates. The statistical maps of the two-sample t-tests were corrected for family-wise error rate (FWER) using Gaussian random field (GRF) correction. The vertex-wise threshold was 0.001 and the cluster-wise threshold for GRF correction was 0.017 (0.05/3, 3 for Bonferroni correction of two hemispheres and one subcortical area).

In sum, 12 participants were excluded after quality control in DPABI and 8 participants might be further excluded on a stricter standard, accounting for 8.6 and 5.8% of the whole fmri-open-qc-rest sample (please see a detailed excluding list with subject ID in Supplementary Table 1). The detailed QC criteria were described in the following sections. The orders of these sections were determined by the frequency of being triggered and the importance of the excluding criterion in each section (e.g., from high to low).

There were several potential issues in the meta-data of images that were identified before preprocessing.

Firstly, the functional images in site-2 and site-5 could not pass the BIDS metadata validation procedure in DPABI. The bids-validation tools reported that “slice-timing values contain invalid value as it is greater than the repetition time” for five participants (e.g., from sub-501 to sub-504, sub-509). Therefore, the five participants with the specific slice-timing errors were labeled as “uncertain,” as we suspected the acquisition sequences were thoroughly distorted. In addition, some of the participants did not have any slice-timing information in the BIDS schema. As we skipped the slice-timing correction in preprocessing, these participants were not excluded from the present study.

Secondly, the number of volumes (time points) was not consistent in site-1 and site-6. It may be acceptable for site-6 as we anticipate that site-6 were constructed by multiple sub-site. But the two participants (e.g., sub-114 and sub-115) with fewer volumes compared with the others in site-1 may suggest data loss in practice. We did not label these suspicious samples as “uncertain” or “excluded” as we did not know the actual scanning protocols for these participants. However, we still raised this frequently occurring issue (inconsistent number of volumes for the images with the same scanning protocol) to inform the beginner of MRI data processing.

Thirdly, sub-605 had two runs of the BOLD series in the raw data while the others only had one run in each session. No additional information was available to help determine which run was more appropriate for further processing. We arbitrarily used the latter one and did not exclude this participant. Because in the practice, the additional run of an MRI series was probably due to the unsatisfying quality of the previous run of the same series (e.g., head-motion exceeding the criteria).

The head-motion induced artifacts were the most frequently reported issue in the QC procedure. Seven out of twenty “uncertain” or “excluded” participants were potentially excluded due to unacceptable head-motion. Some of them were visually identified and the others were identified by the head-motion report generated by DPARSF/DPABISurf (Figure 2A). Of note, the criteria related to the head-motion should be determined according to the research topic (Nebel et al., 2022).

The core procedure of the surface-based methods was brain surface reconstruction. The surface reconstruction could fail due to a variety of quality problems (e.g., low brain coverage of field of view, low signal-to-noise ratio, abnormal brain structure and imaging artifacts, Figure 2B). In addition, the low quality of skull stripping may also hamper accurate surface reconstruction (Figure 2C).

There were two structural images of the participants that failed to achieve satisfying spatial normalization (Figure 2D). Spatial normalization (and related structural segmentation) could fail due to the low quality of images and local minimum in optimization induced by certain random seeds under extremely rare circumstances. Spatial normalization could be substantially improved by the reorientation procedure (e.g., manually rigid translation and rotation before spatial normalization) in DPARSF.

Besides head-motion, there are many MRI artifacts that could affect the image quality, including magnetic susceptibility artifacts, wraparound artifacts, coil-related artifacts, chemical artifacts and et al. (e.g., the T1-weighted images of sub-305 were blurred by unknow MRI artifacts, Figure 2E).

It’s very challenging for neuroscientists to distinguish abnormal brain structures from normal anatomy or tiny MRI artifacts (Figures 2F, G). For example, sub-509 was labeled as uncertain because of the large ventricle. The QC classifiers of the UKBiobank would also take “Bad registration: Structurally atypical: Big Ventricles” as a problem situation. However, large ventricles might be common in the aged population and may not relate to disorders. Therefore, the eliminating criteria could be changeable according to the aim of the studies.

The functional MRIs of two subjects (sub-518 and sub-519) were flipped along the z-axis (Figure 2H). These results underlined the importance of visually checking the images. Flipped images along z-axis (up-down) could be further reversed and are less destructive, but images flipping along the x-axis (left-right) would be harder to recognize and would significantly affect brain symmetry research.

As shown in Figure 3, both of the statistical maps of ReHo sex differences (with/without QC) showed significantly decreased spontaneous activity strength in the posterior cingulate cortex in the male group, which was consistent with the pre-existing literature (Chen et al., 2018). However, the maximal effect size values (Cohen’s f²) with QC (0.234 in the left hemisphere, 0.173 in the right hemisphere and 0.161 in the subcortical area) were higher than that without QC (0.221 in the left hemisphere, 0.152 in the right hemisphere and 0.153 in the subcortical area). Similarly, the maximal effect size values in the sex difference statistical maps of DC, fALFF, and ALFF with QC were higher than that without QC (Supplementary Figures 1–3).