A Simulation Toolkit for Testing the Sensitivity and Accuracy of Corticometry Pipelines - OmidYeganeh, Mona; Khalili-Mahani, Najmeh; Bermudez, Patrick; Ross, Alison; Lepage, Claude; Vincent, Robert D.; Jeon, S.; Lewis, Lindsay B.; Das, S.; Zijdenbos, Alex P.; Rioux, Pierre; Adalat, Reza; et al. (Frontiers in Neuroinformatics, 2021)
Keywords: neuroimaging1, cortical thickness2, tissue deformation3, statistical analysis4, sensitivity5, specificity6, population simulation7, repeated tests8.In recent years, the replicability of neuroimaging findings has become an important concern to the research community. Neuroimaging pipelines consist of myriad numerical procedures, which can have a cumulative effect on the accuracy of findings. To address this problem, we propose a method for simulating artificial lesions in the brain in order to estimate the sensitivity and specificity of lesion detection, using different automated corticometry pipelines. We have applied this method to different versions of two widely used neuroimaging pipelines (CIVET and FreeSurfer), in terms of coefficients of variation; sensitivity and specificity of detecting lesions in 4 different regions of interest in the cortex, while introducing variations to the lesion size, the blurring kernel used prior to statistical analyses, and different thickness metrics (in CIVET). These variations are tested in a between-subject design (in two random groups, with and without lesions, using T1-weigted MRIs of 152 individuals from the International Consortium of Brain Mapping (ICBM) dataset, df=151) and in a within-subject pre-post-lesion design (using 21 T1-Weighted MRIs of a single adult individual, scanned in the Infant Brain Imaging Study (IBIS), df=19) The simulation method is sensitive to partial volume effect and the lesion size. Comparisons between pipelines, illustrates the ability of this method to uncover differences in sensitivity and specificity of lesion detection. In the absence of 'ground truth', such simulation tools help address reproducibility issues that may arise from spurious computational noise. We propose that this method be adopted in the workflow of software development and release.