The advent of open-access data repositories across various scientific fields has initiated new avenues with the potential for transformative discoveries. While poised for a rapid change in pace across many medical fields, particularly those related to the health and brain sciences, these new initiatives have also started to encourage novel exchange and reproducibility of results across labs worldwide. The field of autism research is among those beginning to greatly benefit from these new databanks. Scientists now have at their disposal the opportunity of uncovering new mechanisms and reporting new correlations in multi-modal data with high statistical power, owing this new possibility to the large number of available participants' data. Indeed, it is now possible to aggregate data from different sites and attain a very large number of subjects to build normative data sets from typical controls, as well as to examine pathologies of the nervous systems in relation to new standardized normative scales. Such new characterizations of mental illnesses respond to a recent paradigm shift in psychiatry neuroscience whereby neurodevelopmental disorders are now conceptualized as precursors of mental disorders (e.g., schizophrenia and related mental illnesses) emerging later in life (Paus et al., 2008; Insel, 2009, 2010; Casey et al., 2014)

One such repository is the Autism Brain Imaging Data Exchange (ABIDE) encompassing (in 2017) imaging and demographics data, including 17 sites in ABIDE I (http://fcon_1000.projects.nitrc.org/indi/abide/abide_I.html) and 19 sites in ABIDE II (http://fcon_1000.projects.nitrc.org/indi/abide/abide_II.html) (Di Martino et al., 2014). Indeed, data from ABIDE has been used to examine several new and important questions in autism. Recent studies have examined specific sex-based differences (Alaerts et al., 2016), differentiations in structural organization of the motor systems in light of repetitive behaviors (Supekar and Menon, 2015), cortical volume and gyrification (Schaer et al., 2015), among other analyses and characterizations of morphological parameters. Because of the large sample size, the new results have unprecedented statistical power (Torres and Denisova, 2016). Further, open access to these data has opened new avenues for replication and critical assessment regarding the reliability of clinical tests reported in the demographic data. Among such tests are the ADOS-2 and ADOS-G scores, IQ and medication status (Torres and Denisova, 2016; Torres et al., 2017).

One concern about image analyses has been the presence of motion artifacts distorting the images (Appendix Figure A1). Despite instructions to the participants to remain as still as possible, the human body is in constant motion (heartbeat, respiration, involuntary movements, etc.) Some excess motion may distort the image frame. For this reason, the head motion is tracked throughout the scanning session. Head motion can be extracted from the images time course using conventional methods (Friston et al., 1995; Worsley and Friston, 1995) and open-access software available to researchers (Cox, 1996). The head motion parameters are commonly used to determine the magnitude of the motion and set the threshold to eliminate frames contaminated by motion artifacts, a process coined in some circles “scrubbing.”

Scrubbing can create irregular gaps in the original time series of imaging data. Given individual differences in the amount of involuntary motions such as those of the head at rest, such gaps can vary by participant. As such, given a study with different demographics (e.g., autism and age-matched neurotypical controls in Appendix Figure A1) and owing to the excess motion in autism (Torres and Denisova, 2016; Torres et al., 2017), we would be comparing very disparate sizes of overall number of the clean image frames selected to be included in further analyses. In this way, one would be (unknowingly) skewing the statistical inference and further interpretation of the results. Moreover, because statistical inference may be affected by the non-uniform scrubbing across different sites, reproducibility of research may be compromised. Irregular gaps in the clean data could introduce different biases and give rise to very different outcomes even in cases when the two sites may have implemented an identical study, asked identical questions and recruited participants under identical inclusion/exclusion criteria.

Each person's cumulative involuntary head motion expends energy and in extreme cases (such as those in Appendix Figure A1) energy expenditure may incur in fatigue. We do not know if such energy expenditure would affect blood oxygenation and hemodynamic responses. Thus, we do not know what the cumulative consequences of such excess involuntary motions may be for energy expenditure, fatigue and the BOLD signal in general. At the very least, early in the pipeline of analyses, we can examine the stochastic properties of the original head motion time series data, before scrubbing takes place, and gain insights into the nature of the stochastic processes likely underlying the original time series.

The ABIDE data sets have not been scrubbed, so we have access to the original time series of the head motion data and can study the fluctuations in amplitude of the linear displacement and angular rotations of the head; i.e. as they were originally captured during the resting state of the fMRI sessions. These time series depend on the sampling resolution (SR) of the scanner, i.e. of the number of frames per unit time that the scanner captures, which are different in different sites of ABIDE. Since these differences in SR impact the variability of the speed-dependent data that is used to set thresholds for scrubbing, it is possible that different noise quality, denoting different types of underlying random processes (Stanley et al., 1999; Seely and Macklem, 2004; Perkiömäki et al., 2005) may be inherently present in the data (Appendix Figure A2). Characterizing the stochastic properties of the raw data is then important because the removal of motion artifacts depends on the threshold criteria derived from the head motion being tracked (Friston et al., 1996). In turn, as illustrated in Appendix Figure A2, different random processes underlying time series data may lead to diverse cumulative effects and give rise to inherent biases in thresholding the data to be eliminated from a given set.

One way to characterize the stochastic features inherent to the raw data is by empirically estimating the probability distributions underlying a parameter commonly used in the literature of motor control to investigate the nature of the variability in biophysical data. Such data include motion biorhythms extracted from signals harnessed from heart rate (Peng et al., 1995b) including local-scale of shorter time series (Castiglioni et al., 2007), gait (Hausdorff et al., 1997; Kaipust et al., 2013) including short time series (Qiu et al., 2016; Terrier, 2016), finger tapping (Botcharova et al., 2015), among others. The parameter of interest is the alpha exponent (explained in the methods) derived from Detrended Fluctuation Analyses (DFA) (Peng et al., 1995b), a popular method to examine stochastic processes and gaining insights on the self-affinity / stationarity (or lack thereof) of biophysical time series data (Stanley et al., 1999).

It is possible that the noise quality emerging from these analyses point to different types of random processes characterizing events captured by these time series under disparate sampling rates. For example, by capturing (or omitting) small rotations or displacements of the head with different frequencies per unit time (Appendix Figure A3), we may introduce different biases in the early scrubbing stage of the data processing pipeline. This stage cleans the images from motor artifacts. As such, to further proceed with data cleansing and statistical inference, one would need to take into consideration the different sampling resolutions of scanners in different ABIDE sites. ABIDE I and II comprise 26 sites with sampling resolution above 1 Hz; in contrast to 2 sites below 1 Hz. It suffices for us to examine two extreme cases to learn if variations in sampling resolution give rise to different ranges of noise. Having this knowledge could help researchers further design appropriate analyses for statistical inference, better standardize their methods and more generally increase the rates of research outcome reproducibility across labs.

The variable degree of skewness we have previously found in the empirical distributions of linear and angular speed peaks derived from these ABIDE data (Torres et al., 2017) sets motivated us to further explore the possibility that different random processes may underlie the time series data collected under different SR. As such, here we try to elucidate the quality of the noise inherently present in the variability of the speed-dependent raw data comprising all the original frames of the studies of ABIDE (i.e., without scrubbing the images).

Given prior results from other fields concerning differences in signals representing different kinds of random processes (Hausdorff et al., 1995; Havlin et al., 1995; Peng et al., 1995b), we here hypothesize that the differences in sampling rates of the scanners will affect the nature of the noise in the data. By noise, we specifically mean the noise to signal ratio, whereby to obtain the ratio, we empirically estimate the Probability Density Function (PDF), the mean and the variance characterizing the parameter of interest (the alpha index). This contrasts with assuming a given theoretical distribution (e.g., the Gaussian) or a given feature of the random process (e.g., stationarity). As a possible corollary of this proposition, we posit that within sites of similar SR, we may be able to use the noise range (empirically obtained from the raw data) to further characterize and differentiate neurodevelopmental disorders in the broad spectrum of autism relative to normative ranges. We report our results on combining and analyzing 2,154 participants from 28 sites of ABIDE I and II. Further, we provide evidence that different disorders can be well characterized by different noise qualities relative to age-matched typical controls.

This paper addressed the question of whether the disparity in sampling resolution across different sites in the ABIDE repository would affect the quality of the noise-to-signal ratios empirically derived from the fluctuations in the amplitude of head motion speed. The main motivation for this question is the common use of speed amplitude as criterion to threshold the scrubbing of the imaging slides early in the pipeline of fMRI analyses. The disparate SR resulting in statistically different alphas strongly suggests adhering to SR that are comparable when pooling data and thresholding motor artifacts before further statistical inferential analyses.

To address this question, we used DFA to estimate the scaling (alpha) exponent values for the time series of linear and angular speed peaks defining the envelope (amplitude) of this signal. These were obtained from the head motions extractable from the rs-fMRI data using traditional methods from the imaging community. More specifically, such methods are commonly employed to detect excess in head motion and remove motion artifacts from the set of images acquired in the imaging session. This pre-processing stage eliminates “contaminated” frames and selects the remaining frames for statistical inference in subsequent image analyses (e.g., functional connectivity, structure and morphology, among others). The methods presented in the paper also address the disparity in clinical group sizes within diagnoses of the ABIDE demographics, thus providing a way to enable appropriate statistical comparison of groups while using similar size and age composition according to the various DSM and phenotypical criteria listed in the demographics data of ABIDE.

Given the wealth of information of the ABIDE repository, many more comparisons and other parameters could have been explored. Here we focused primarily on ASD (DSM-IV and non-DSM-IV), AS, TD, FEMALES, MALES and MEDS, No MEDS reports. These groups served to illustrate that:

The statistical estimation procedures using comparable group size and age composition uncovered a gradient of alpha values toward the ranges of fractional Brownian motion (fBm) for the sites with sampling resolution below 1 Hz (denoted SR0) (Figure 9). Most values fell above 1.5 alpha-ranges indicating persistent fBm (Appendix Figure A2). This contrasted with lower ranges of alpha values tending instead in the opposite direction, toward the pink noise range for sites with sampling resolution above 1 Hz (denoted SR1) including 0.5 < α ≤ 1 in the range of persistent fractional Gaussian noise (fGn) (Appendix Figure A2). Within each range, the alpha values were distinct for each of the groups of interest with systematically large separation between TD and ASD; but also, separation between the ASD and AS groups (whenever data for comparison were available). Comparisons between MEDS and No MEDS groups also yielded differences in both types of sampling resolution, thus suggesting that future studies using repositories that report medication intake may benefit from such analyses to determine possible effects of medication on the person's biorhythms (see prior results here, Torres and Denisova, 2016). Further, males and females yielded differences and some degree of overlapping that may be refined using other methods for each of the diagnosis under consideration. Indeed, recent work using these ABIDE sets provided evidence to that end between the females with ASD and females with AS (Torres et al., 2017) that emerged using the micro-movements (fluctuations in peak speed amplitude) data type instead of the raw speed data used here. That work provides scaling and standardization methods that further address the problem of different anatomical sizes in the participants of these databases and the effects such allometric issues have on speed-dependent waveforms. However, discussion of such issues is beyond the scope of this paper. Here, our goal is far more modest than providing a standardized waveform addressing such age-dependent allometric effects. We merely aimed at questioning if different SR could lead to differences in variability with statistically significant distinctions in noise quality.

It is important to note that the biorhythms we extracted from head motion data obtainable from the rs-fMRI time series are merely one out of many possible read-outs from the nervous systems across the body (i.e., the peripheral output that includes motion as a form of kinesthetic reafference, such as breathing and heart rhythms, bodily kinematics and electromyography, among others). Other biorhythms are also reported in various data repositories. As such, the methods presented here may be of use when pooling data harnessed under different sampling resolutions. They include for example electroencephalography (EEG) and other morphological and structural parameters in cross-sectional and longitudinal data, expressible as time series of fluctuations. For a summary on various open access data repositories see (Eickhoff et al., 2016).

Given that analyses involving imaging undergoes a pre-processing stage whereby frames are eliminated based on a threshold derived from head motions, and head motion artifacts are characterized using fluctuations in the amplitude of the linear displacement and/or angular rotations of the head, frame by frame, it may be important for researchers of that community to not assume a theoretical type of random process or follow a “one size fits all” approach. Instead, the results from our analyses suggest that empirical estimation of different random processes likely underlying the time series under consideration may be more appropriate (as we underscore in Appendix Figure A4). In turn, such empirical estimation (rather than a priori assumption of a theoretical distribution family and/or random process type) particularly when done in a personalized manner, may help other steps preceding the selection of frames to eliminate before other analyses for statistical inference.

In summary, we present evidence that when using large data repositories and pooling data from different fMRI sites, we should be mindful of the underlying instrumentation used to gather the data in the first place. We should also consider the sample sizes and age-compositions of the various groups, and build methods amenable to design standardized scales that we can then map to physical phenomena. In this way, we can initiate the path toward the design of new phenotypic characterizations of the human spectrum to properly associate phenotypic data gathered with subjective and objective means with existing genotypic data, now shared in multiple open-access repositories (Eickhoff et al., 2016).

Acquiring more rigorous scientific practices in the fields that study disorders of the nervous systems may help us reproduce results in open access settings. In turn, this may stimulate the exchange of information worldwide across labs and accelerate the design of target treatments in the future. This work is merely an example (out of many possible methods) of beginning the steps to design research programs within the fields of Psychiatric and Psychology that follow empirical estimation procedures of the scientific method.

CC: Wrote the methods, analyzed all data and produced the Supplementary Materials; SM: Extracted all head motion data from ABIDE I and ABIDE II, processed all the demographics for exclusion-inclusion criteria and implemented randomizer for bootstrapping; JV: Developed the Python code for DFA; ET: Designed study, created/implemented analyses and wrote paper; CC, SM, JV, and ET: Edited paper. All authors read and approved the last version of the MS.