A primary goal of neuroscience - and cognitive neuroscience in particular - is to understand the brain mechanisms that support behavior. Studies that attempt to empirically examine the relationship between brain and mind require direct examination of which sources of neural variability are actually related to behavioral variability in cognitive domains of interest. A common approach in task-based neuroimaging studies such as those using fMRI is to correlate the mean BOLD response in a certain task across participants with a behavioral measure taken from the same participants (for discussion, see Rousselet and Pernet, 2012; Vul et al., 2009; Yarkoni and Braver, 2010). A similar approach to examining inter-individual differences has also been taken using task-free or “resting-state” studies in fMRI, in which participants monitor a fixation cross or close their eyes and endogenous variation in brain activity is measured (e.g., Fox and Raichle, 2007). In these contexts, correlation of BOLD activity is calculated across pairs of brain regions or networks of interest for a given participant, which is then correlated across participants with behavioral measures of interest. This has been done in neurologically intact participants when studying particular domains of cognition, as well as in clinical studies of particular patient groups (e.g., Finn et al., 2015; Gotts et al., 2012, 2013; Jasmin et al., 2023; Kaiser et al., 2015; Ramot et al., 2019; Rosenberg et al., 2016; Sheffield and Barch, 2016; Stevens et al., 2017; Zhu et al., 2011; for discussion, see Martin et al., 2012).

A recent study has demonstrated some practical limits on our ability to achieve this goal, at least for inter-individual differences in behavior (Marek et al., 2022, “Reproducible brain-wide association studies require thousands of individuals”; see also Elliot et al., 2020). Marek et al. (2022) examined the sample sizes needed for replication in order to associate brain measures such as cortical thickness and resting-state functional connectivity with complex behavioral measures such as overall cognitive ability and psychopathology (referred to as Brain Wide Association Studies, or BWAS). Using the largest neuroimaging datasets that are publicly available (e.g., Human Connectome Project, Van Essen et al., 2013; ABCD, Casey et al., 2018; UK Biobank, Sudlow et al., 2015) they found that effect sizes were much weaker than expected (approximately 0.1 or below), and that thousands of participants were required for robust replication. This contrasts to the typical sample size of most BWAS studies (approximately N = 25), explaining the lack of replicated findings. These effect sizes might be improved by restricting the samples to those of high quality (improved signal, decreased sensitivity to scanning artifacts such as motion) and restricting acquisitions to single scanning sites (e.g., Spisak et al., 2023; c.f. Tervo-Clemmens et al., 2023). Nevertheless, even if these effect sizes were doubled, the overall expectations of smaller effect sizes (an effect size of 0.2 will require approximately 200 participants to detect an effect of p < 0.05 at 80% power) is still that much larger samples will be required to reliably observe brain-behavior associations than those typically acquired in individual labs.

However, Marek et al. (2022) also highlighted the potential utility of certain smaller-sample neuroimaging studies for assessing brain-behavior relationships, particularly those employing within-person designs with “induced” effects (such as tasks), as opposed to resting-state functional connectivity. Such studies can have increased measurement reliability and effect sizes. Here we provide a concrete example of this alternative, utilizing a task-based fMRI design with participants overtly naming pictures of common objects. Picture naming indexes several large cognitive domains including vision, conceptual processing, language, and motor functioning (e.g., Gilmore et al., 2019; Glaser, 1992; Gotts et al., 2021; Johnson et al., 1996; Kan and Thompson-Schill, 2004). In order to estimate single-trial responses to the task, we adopt a slow-event related design rather than the more common rapid-event related design (Figure 1A; see Bandettini and Cox, 2000; Gotts et al., 2021, for discussion). This allows us to isolate better the BOLD response to individual trials for which we have a measure of behavioral performance in the task, namely response time (RT). While the temporal variation in RT is small relative to the overall time course of the BOLD signal (e.g., RT: 500–1,500 msec; BOLD signal: ~ 16 s), different trial durations are expected to manifest as different amplitudes in the BOLD response (e.g., Yarkoni et al., 2009; Rao et al., 2014; Yamasaki et al., 2017). We further utilize multi-echo imaging to aid with reducing movement artifacts from overt speech and compare this to more standard single-echo acquisition. The data presented here are taken from a previous fMRI study examining the role of task (picture naming versus recognition memory) on stimulus repetition effects (Gilmore et al., 2019). However, we have previously only reported the condition averages from this prior study—not aspects of the single-trial responses, and here, we only analyze the data from the first two scanning runs prior to any stimulus repetition (Initial Naming Phase).

Critical to the evaluation of effect sizes is some form of cross-validation to avoid inflated estimates within-sample due to overfitting (e.g., Marek et al., 2022; Spisak et al., 2023). We satisfy this requirement here in a relatively simple way, using an orthogonal effect to the effect of interest: the mean BOLD response to the task versus the correlation of trial-level BOLD responses and behavior. Selecting voxels based on the mean BOLD response across trials has no biased relationship to the estimation of the trial-to-trial variability around the mean as it covaries with an independent behavioral measure.

In the current experiment, we evaluate the feasibility of detecting brain-behavior correlations at a trial-level per participant during object naming. Since test–retest reliability of a measure constrains its possible correlation with other measures (e.g., Nunnally, 1959), it is useful to estimate these values when possible. While we do not have ready estimates of test–retest reliability of the single-trial fMRI BOLD responses from prior studies, the pre-fMRI behavior-only phase of Gotts et al. (2021) involved each participant naming a set of 100 pictures three times (using the same pictures as in the current study, but with different participants). From this experiment, we are able to estimate the test–retest reliability of our response time (RT) measure by correlating the single-trial RTs across different naming attempts of the same pictures by each person. It is important to note that the very act of repeating a stimulus will alter behavioral and neural responses to it, leading to faster RTs and decreases/increases in BOLD (see Gilmore et al., 2019, for discussion). The decreased range of RTs to repeated stimuli in the presence of measurement noise/variability may therefore lead to slightly decreased estimates of test–retest reliability, but these empirical benchmarks still provide useful context. The test–retest estimates are shown in Figure 2 for the 58 participants from Gotts et al. (2021) with pre-fMRI naming data (Cohort 1 participants were assigned to the Covert Naming condition during fMRI for this experiment, Cohort 2 participants were assigned to the Overt Naming condition; all participants performed Overt Naming in the pre-fMRI session). Overall, participants had very low test–retest reliability of the single-trial RTs (mean = 0.1681), although values covered a large range for individual participants (−0.012–0.496). This suggests that many trials would be required for BOLD-RT correlations to be significant for the average participant (for an effect size of 0.1681, the expected N in trials to detect an effect at p < 0.05 with 80% power is 278 trials; Lachin, 1981), and many more still would be required if the test–retest reliability of the single-trial BOLD responses is low and/or if the true BOLD-RT correlation is low. Based on this prior experiment, we should not expect the mean BOLD-RT correlation across participants observed in the current study to be much larger than 0.17.

A single-participant example of the main quantities of interest in the current study for the ME-ICA processed data is shown in Figure 3. Figure 3A shows the stimulus effect (Stimulus vs. a baseline of 0 during fixation) on the BOLD response in the top panel (p < 0.005, FDR-corrected to q < 0.0193) and the correlation between BOLD and RT across trials in the bottom panel (p < 0.05, uncorrected). Despite the expected low correlations between BOLD and RT, the uncorrected map is quite similar to the overall task response—and similar to the brain regions known to be involved in picture naming from prior studies (e.g., left lateral frontal cortex, the fusiform gyrus bilaterally and the anterior cingulate; Gotts et al., 2021). As in prior picture naming studies, the distribution of response times for this participant ranges from 600 ms up to 1,500 ms with a mean of approximately 909 ms (Figure 3B). In Figure 3C, we have shown the first 100 TRs of the BOLD response (220 s) from a voxel in left frontal cortex showing both a significant task response and a correlation between BOLD and RT (highlighted by the green crosshairs in Figure 3A). Vertical red lines are placed at the beginning of the expected BOLD peak (4.4 s after stimulus onset) for each correctly named stimulus during that period (N = 23 correct responses). Visually, there is quite good correspondence between the BOLD peaks and the expected onsets for this voxel. Finally, a scatterplot of the single-trial BOLD peaks (x-axis) with the single-trial RTs (y-axis) is shown in Figure 3D for the same highlighted voxel (at T–T coordinate −43, +12, +27). As in previous studies examining BOLD-RT correlations in task-positive regions (e.g., Rao et al., 2014), there is a positive slope between BOLD and RT, with slower trials having larger BOLD responses [r(93) = 0.2848, p = 0.0052].

In an experiment with 40 participants performing an object naming task, we have observed robust and replicable effects of task and of trial-level BOLD-RT correlations. ME-ICA denoising of multi-echo data yielded the strongest BOLD-RT correlations in our experiment (Cohen’s d’s > 1.0), but strong effects (Cohen’s d’s of 0.9–1.0) were observed even for more traditional single-echo data. The slow event-related design used here is undoubtedly an impactful choice, allowing the improved isolation of individual trial responses over more traditional rapid event-related designs—which are primarily concerned with estimating condition-level mean BOLD responses across trials. Without the ability to estimate trial-level effects, we would be restricted to calculating brain-behavior correlations across participants as in most previous studies, potentially more limited in effect sizes (estimates of 0.4–0.6, e.g., Vul et al., 2009). While the BOLD-RT correlation magnitudes for individual participants were observed to be weak (correlation effect sizes of <0.1), these values were reliably different from zero across participants—permitting much larger effect sizes when considering the group-level means. As anticipated, this group-level reliability hinged on the number of trials included, with reasonably robust effects observable with 30 or more trials included (see Chen et al., 2022, for related discussion). With all correct trials included (more than 90 trials per participant on average), we estimated that as few as 10 participants might be needed in a sample to observe replicable results at p < 0.05. This is not to suggest that small sample sizes will suffice for all such effects; the needed sample size for a given study will depend on the particular effect sizes involved and the desired level of significance and power.

The conclusions of Marek et al. (2022) that BWAS studies will likely require thousands of participants to detect reliable findings has predictably set off intense discussion within the field of cognitive neuroscience about whether the current state of affairs is really so dire (e.g., Bandettini et al., 2022; Gratton et al., 2022; Liu et al., 2023; Rosenberg and Finn, 2022; Botvinik-Nezer and Wager, 2022; Spisak et al., 2023; Tervo-Clemmens et al., 2023; Westlin et al., 2023; Wu et al., 2022). In the current paper, we have detailed a task-based alternative to detecting brain-behavior relationships that primarily utilizes inter-trial variability in the BOLD response and behavior rather than inter-individual variability. Taking this approach in the current study turned trial-level effect sizes similar to what Marek et al. (2022) reported (< 0.1) into group-level effect sizes that are an order of magnitude larger (1.0 or more). The savings in sample size to produce replicable findings is dramatic, going from thousands to tens. For domains of cognition that can be studied with impulse-response type tasks with fMRI (stimulus duration of approximately 1 s or less, followed by an immediate response), this type of design provides a good option for making quick progress in individual labs. It is also worth revisiting the task-based approach reviewed by Vul et al. (2009) (see also Rousselet and Pernet, 2012; Yarkoni and Braver, 2010) that utilizes inter-individual variability in task activation and behavior. If unbiased effect sizes are actually in the range of 0.4–0.6, sample sizes of 30–40 participants may be sufficient to detect replicable effects. In one of our recent studies (Gotts et al., 2021), we correlated the magnitude of repetition-related BOLD decrease (referred to as “repetition suppression”) in left frontal cortex with the magnitude of behavioral repetition priming across 60 participants. This particular analysis was conducted as a replication of prior studies (e.g., Dobbins et al., 2004: N = 16 participants; Horner and Henson, 2008: N = 18; Maccotta and Buckner, 2004: N = 54), and indeed, it did replicate—despite utilizing inter-individual variability in BOLD and RT differences (OLD vs. NEW stimuli) [r(58) = 0.367, p < 0.004].

We observed larger BOLD-RT correlations and group-level effect sizes using ME-ICA processing. This adds another data point in favor or utilizing multi-echo acquisitions, and using ME-ICA, in particular (see also Beckers et al., 2023; Kundu et al., 2013; Gilmore et al., 2022; Reddy et al., 2024; Steel et al., 2022). The advantage in using ME-ICA here likely results from the removal of non-BOLD variation from the total variation, improving the signal-to-noise ratio (SNR) of the retained signal and strengthening the BOLD-RT correlation. Similarly, the likely benefit of the Optimally Combined preprocessing over Single-Echo is the reduction of thermal noise that occurs when averaging across the three echoes at each TR. At a field strength of 3 Tesla with 3-mm isometric voxel resolution, thermal noise still makes up a sizable portion of the time series SNR (e.g., Triantafyllou et al., 2005), and reducing it through local averaging (either temporally for multi-echo or spatially for single-echo acquisitions) should lead to larger brain-behavior correlations as observed here. Given the reduction of thermal noise when using multi-echo acquisitions without the need for spatial averaging, an added advantage of multi-echo protocols is improved spatial localization within individual participants.

While we do not have an explicit estimate in the current study of test–retest reliability of single-trial BOLD responses, they appear to have sufficient reliability that detection of BOLD-RT correlations is possible—despite the poor reliability of individual RTs (approximately 0.17). This suggests that slow event-related designs have a notable advantage over rapid-event related designs, namely that both the mean of individual conditions can be estimated as well as the values of individual trials. As can be seen from the single-participant example in Figure 3C, the peaks of individual trial responses are visible by eye without complex statistical analyses (see also Bandettini and Cox, 2000). This is the case in the current study, even without requiring interstimulus intervals greater than or equal to the full duration of the BOLD response (approximately 16 s); the average interstimulus interval in the current study was 9.6 s (minimum 6.3 s, maximum 12.9 s). A wide range of cognitive tasks can be conducted using this range of trial timings without participants losing focus. The loss in number of stimulus presentations when compared to rapid event-related designs is also mitigated by the fact that no additional baseline periods are needed in slow event-related designs (whereas approximately 30% of each run duration in rapid event-related designs are reserved for baseline periods). A final added advantage of this type of slow event-related task design is that it can provide a way to more cleanly estimate task-based functional connectivity in fMRI in a manner that is less confounded by the local stimulus response itself. Recently, Gotts et al. (2021) in the domain of repetition priming in picture naming showed how to use trial covariation as a measure of functional connectivity separate from the mean BOLD response to the task. By eliminating the up-and-down contour of the evoked response and focusing on the peak response to each trial, one can evaluate the covariation of individual trial amplitudes across pairs of voxels or brain regions, investigating differences in functional and/or effective connectivity for different stimulus conditions (e.g., OLD vs. NEW), as well as correlating task-based connectivity measures with behavior.

The current approach is limited to investigating brain-behavior correlations using the BOLD response in fMRI. This approach has the advantage of whole-brain coverage and good spatial resolution. However, the temporal resolution is relatively poor, and the averaging inherent in the BOLD response does not allow unambiguous estimates of the underlying neural activity. Combining this approach with a method like EEG that has the temporal resolution of milliseconds (e.g., simultaneous fMRI-EEG) may help to undercover additional features of brain-behavior relationships that are not possible to study with either neuroimaging method used in isolation. The behavioral task examined here is also relatively simple. More complex tasks that are more extended in time will likely require more sophisticated analyses (for example, see Gilmore et al., 2021; Jasmin et al., 2023). In these situations, the trial-level approach may be less applicable.

How could we extend the current approach to a richer investigation of behavior? Firstly, cognitive psychology has utilized response time and accuracy as primary measures of behavioral variability in tasks since the 1970s. These measures are themselves demonstrably rich. However, one can think of alternative ways to gain further insights. For example, if one were to norm a stimulus set on a variety of behavioral measures in an independent set of participants, one could then seek trial-level correlations with the BOLD response using these “external” measures of behavior per trial. Here, we could have used lexical frequency of the object names (e.g., Kucera and Francis, 1967), familiarity, or visual complexity (e.g., Snodgrass and Vanderwart, 1980) as trial-level measures to correlate with our single-trial BOLD responses rather than the simultaneously acquired response times. The role of the naming task then would be to force participants to deeply process and engage with the stimuli such that these alternate behavioral facets might be expressed in a subset of the brain regions detected in the task overall.