In the AX-CPT, each trial includes a cue (i.e., A or B), a delay period, and a probe (i.e., X, Y, or number). When an “A” cue precedes the “X” probe, a target response should be made; however, the presence of a “B” cue (i.e., any cue letter besides “A”) and/or a “Y” probe (i.e., any letter besides “X”) indicates that a non-target response should instead be made; digit probes (i.e., 1–9) indicate no-go trials, for which participants were instructed to withhold their response. These no-go trials were included to reduce the inherent bias to fully prepare probe responses following the cue. The proactive and reactive versions of the task had specific modifications designed to induce each mode of control, respectively. The proactive variant explicitly instructed participants to prepare a target response after seeing an “A” cue and a non-target response after seeing a non-A cue; practice sessions and reminders to use the strategy were meant to reinforce the use of proactive control for this session. The reactive variant instead included a red border color at a unique location presented immediately before probe onset to indicate an upcoming high-conflict trial (i.e., AY, BX, no-go trials). The proactive version was designed to bias participants to engage and maintain cognitive control during the cue and delay period; instead, the reactive version included a specific probe-linked feature that alerted participants to actively retrieve the cue and abstract rule only at the time of probe presentation.

In the Sternberg task, participants had to determine whether a probe word was also present in the previous word list (memory set) on trials that varied possible working memory loads (i.e., the lists could contain 2–8 words, which were presented across two encoding displays). If the probe word was indeed part of the memory set, the trial would be categorized as a novel positive (NP) one, and a target response would be correct. On both novel negative (NN) and recent negative (RN) trials, the probe item was not part of the memory set for the current trial; however, the latter was characterized RN because the probe item was present in the memory set of the immediately preceding trial (i.e., was recently encoded into memory), thereby causing increased familiarity and false positives for RN versus NN trials. Both the number of items in a word list and the proportion of RN, NN, and NP trials were manipulated in different ways to induce proactive, reactive, or neither/baseline mode of control. In the proactive condition, most trials had low-load (2–4 items) memory sets, which biased participants to actively maintain these items as an effective strategy to bias attention toward the probe. In both the baseline and reactive conditions, most trials instead had high-load (6–8 items) memory sets, which biased participants to instead use a familiarity strategy to determine whether to make a target response to the probe item. In the reactive condition, however, the high frequency of RN trials would instead cause probe familiarity to serve as an alerting signal for full memory set retrieval. Finally, each condition included a matched set of 5-item lists that were treated as “critical items” and used for cross-condition comparisons.

Although all versions of the color-word Stroop task utilized the classic contrast in performance between low-conflict congruent items (the font color and word match, e.g., BLUE in blue font) and high-conflict incongruent items (i.e., the font color and word do not, e.g., BLUE in red font), the proportion congruence (PC) within a task block was uniquely manipulated across conditions to induce different degrees of cognitive control use. A subset of diagnostic items that were equally likely to be congruent or incongruent (i.e., PC-50/diagnostic items) was included, so that directly matched comparisons could be made between conditions. Another subset of items was either mostly congruent or mostly incongruent (i.e., biased/inducer items), which distinguished the different conditions from one another. The baseline condition was designed to have high list-wide PC, in which congruent trials were relatively frequent and incongruent trials were rare, so that cognitive control demands would be relatively low throughout the task block. In contrast, the proactive variant was designed to have low list-wide PC which would bias participants to prospectively (i.e., before stimulus onset in a trial) utilize cognitive control throughout the task block, by attenuating attention toward the distracting ‘word’ dimension. In the reactive condition, the PC manipulation was item-specific such that the inducer items were mostly incongruent colors, while other filler items were always congruent; thus, the list-wide PC was matched to the baseline condition, but specific colors (inducer items) indicated high control demands. As a result, participants were expected use the color feature (i.e., detected after stimulus onset) as an indicator of whether cognitive control should be utilized (reactively) on that trial.

All stimuli in this task consisted of letter-digit pairs (e.g., A-1, B-2); however, a task rule cued at the beginning of each trial indicated the relevant feature dimension to attend on that trial (i.e., attend to the letter to make a consonant/vowel discrimination, or attend to the number to make an odd/even discrimination). Two overlapping manual responses were used for each task; thus, the meaning of each manual response was dependent on the task rule, with stimuli that could either be congruent (i.e., the two task rules are associated with the same response) or incongruent (i.e., the two task rules are associated with different responses). The baseline version followed closely to this general task structure, with the task cues appearing in red font and the task stimuli appearing in black font. In contrast, within the proactive and reactive variants, a subset of incentive trials was included to bias participants toward one mode of control. The proactive version included reward incentive trials, pre-cued by the green font of the task rule, so that participants would engage in anticipatory cognitive control before stimuli onset, to make faster and more accurate responses. In contrast, the reactive version included punishment incentive trials (if an error was made), cued by the green font of the target stimulus, and primarily on incongruent trials, so that the processing of incongruence itself was associated with potential monetary loss. A matched set of non-incentivized trials were utilized for cross-condition comparisons. These non-incentivized trials were also biased to be mostly congruent, increasing the likelihood of conflict and interference on incongruent trials.

Within the AX-CPT, a primary focus of theoretical interest is the BX error interference effect, an index of cognitive control which reflects the ability to appropriately utilize contextual cue information to correctly bias nontarget responses to the probe. In particular, the BX error interference effect occurs due to the relative difficulty of high-conflict BX trials (i.e., the same probe combined with an “A” cue would have required a target response) versus low-conflict BY trials (i.e., neither the probe nor the cue indicate a target response). However, the deployment of either proactive or reactive control is theoretically predicted to lead to reduced BX error interference relative to the Baseline condition; this metric should serve as a general marker of cognitive control. From this prediction, we used the HBR modeling approach as a tool to explore the consistency of predicted effects across datasets, specifically examining the magnitude of BX error interference in Reactive and Proactive conditions across the DMCC 2018 and 2020 datasets.

In the HBR analysis, the BX interference effect was estimated as the log odds of making an incorrect response for BX versus BY trials, with the following model run on BX/BY trial-level data: probeCorrect ~ 1 + trial type x mode + (1 + trial type x mode | ID)⁷. The outcome variable ‘probeCorrect’ was binary coded so that the model predicts the log odds of an error (i.e., correct = 1; incorrect = 0). The categorical variable ‘trial type’ was dummy coded, with BY trials as the reference category to estimate relative BX interference. The categorical variable ‘mode’ was dummy coded with Baseline as the reference category to compare how the Proactive and Reactive modes affected this estimate; a separate HBR model was run to compare each mode of control to Baseline. The interaction effect between trial type and mode (i.e., trial type x mode) was the key parameter of interest to examine the difference in BX error interference between modes. This interaction term was also treated as a random effect within each subject to more fully account for trial-level variability.

In line with theoretical predictions and previous results from Tang et al. (2023), the 2020 HBR models indicated strong evidence for a reduction in the BX error interference effect in both the Proactive ( β = − 0.43, se = 0.13, 95% HDI = [−0.70, −0.20], pd. = 99.97%) and Reactive modes ( β = − 0.52, se = 0.15, 95% HDI = [−0.82, −0.22], pd. = 99.96%) relative to Baseline. Our key focus of interest was to compare the prior and posterior distributions for both models of BX error interference reduction (Figures 2A,B). The posteriors of this measure had higher peaks and narrower distributions relative to their priors, signifying that the addition of new data reduced uncertainty in the estimate of this parameter.

Interestingly, there was a notable difference in the SDR results across the Proactive and Reactive BX error reduction models. The SDR of the Proactive versus Baseline BX error interference estimate was greater than one (1.44; Figure 2A), which indicated a substantial overlap between the prior and posterior distributions. This result can be interpreted as evidence that the estimate of the BX error interference effect reduction in Proactive was quite consistent across the 2018 and 2020 DMCC datasets. By contrast, the SDR of the Reactive versus Baseline BX error interference estimate was clearly smaller than one (0.66; Figure 2B), which indicates that the parameter estimate from the 2018 dataset underestimated the value obtained when both 2018 and 2020 datasets were included. Furthermore, the pd. was less than 97.5% in the 2018 HBR model (pd = 92.1%) but 100% in the 2020 one. The SDR findings that compare the updated posterior with its original prior highlight how accumulating more data enables a direct test of consistency. In this case, we demonstrate that the reactive BX error interference effect can still be treated as a general indicator of cognitive control engagement; yet at least for the reactive condition, the results point to a lack of consistency in the presence and magnitude of the effect across our two datasets.

The BX error interference effects from the AX-CPT illustrate how the sequential learning process inherent in a Bayesian framework can provide a cumulative quantification of the most likely values for a parameter of interest. Specifically, we demonstrate both a case where the accumulation of additional data increased our confidence in the initial estimate, as well as a case in which our confidence shifted from the initial estimate toward a new more likely one. Specifically, the magnitude of the Proactive BX interference effect was stable across both datasets, signaling that the Proactive variant generated a reduction in BX interference that was highly consistent across two dataset samples. By contrast, the magnitude of the Reactive BX interference effect was found to vary substantially across the datasets. Thus, one future direction would be to explore the reason(s) behind this discrepancy in the consistency of the Proactive and Reactive BX interference effects. The increased certainty in the original Proactive BX error interference estimates across these two datasets suggests that the effect should successfully generalize to new samples and contexts. Conversely, the contrasting magnitude of the Reactive BX interference effects could be due to either a lack of replicability or of generalizability. Interestingly, one meaningful difference between the two datasets was the order of experimental conditions. Within the 2018 datasets, the Reactive condition was last, and after the Proactive condition, whereas in the 2020 dataset, the Reactive condition was performed before Proactive and immediately after Baseline. Consequently, it would be ideal to collect future datasets with the same or different condition orders to see whether this factor might serve as a significant moderating variable on Reactive BX error interference. More generally, the sequential updating approach implemented in HBR can be useful in providing clues regarding when it might be profitable to conduct additional replication studies, and further how these might be conducted, in terms of the key factors to examine.

In the Sternberg task, the novel positive (NP) effect in reaction times (RT) is of theoretical interest as a potential index of proactive control. The effect is defined as the difference in response time to make correct responses for NP trials across control modes. While low-load and high-load memory sets were used to bias participants toward or away from proactive control, each condition also included an equal number of critical trials (i.e., those with 5 items per memory set) that could be used to make direct comparisons across conditions. The key theoretical prediction was that response decisions to probe items through proactive control would be directly based on active and accessible WM representations (i.e., in the focus of attention), which would enable such decisions to be made more quickly than other conditions in which the probe item had to be elicited via a retrieval cue to reactivate information.

In Tang et al. (2023), the NP effect was found to be statistically significant, but with unclear reliability in the Proactive versus Baseline contrast, while the effect for the Proactive versus Reactive contrast was numerically in the correct direction but not statistically significant. Here, we relied upon a key feature of HBR methods to explore these effects further, by directly assessing the evidence for each contrast, regarding both the alternative (Proactive < Baseline NP RT) and null hypotheses (Proactive = Reactive NP RT) respectively.

To specifically test for the relative evidence in favor of each hypothesis, a Bayes Factor (BF) model comparison approach was utilized, by running two types of models: one representing the null hypothesis, RT ~ 1 + (1 | ID), and the other representing the alternative hypothesis, RT ~ 1 + mode + (1 | ID). ‘Mode’ was dummy coded with Baseline and Reactive as the reference conditions to show NP performance in the Proactive mode relative to other conditions. Since BF model comparison was employed to determine if the fixed effect ‘mode’ had a strong impact on the NP RT results, we chose to not include ‘mode’ as a random effect for these HBR models; therefore, only the intercept was entered as a random effect nested within subject to account for trial-level variability. Additionally, we employed the ROPE method as an alternative approach to assess evidence for the null hypothesis, assuming that an RT difference within a range of 5 milliseconds would correspond to a negligible NP effect. Here we used an ex-Gaussian distribution to model RTs. As we discuss further for the Stroop task, the shifted log-normal distribution is also an excellent choice for modeling RTs; however, its transformation of the RT data into log-normal units makes it more challenging with regard to null hypothesis interpretability (i.e., which values should be used to define the ROPE). Both conventional 89% and conservative 95% HDI thresholds were used to assess whether this choice affects qualitative conclusions.

For this set of analyses, we focused on the 2020 HBR models which have the compounded benefits of incorporating informed priors to generate a cumulative posterior distribution, random effects to account for the hierarchical structure of the data, and an ex-Gaussian function to model its skewed properties. Even with these advantages, we did not find the NP RT effect to be a robust and general indicator of proactive control: although there were decisively faster responses for NP trials in Proactive versus Baseline ( β = − 19.92, se = 2.38, 95% HDI = [−24.57, −15.26], pd. = 100%), there was no evidence of an NP effect in the Proactive versus Reactive comparison ( β = 0.73, se = 2.25, 95% HDI = [−3.63, 5.19], pd. = 62.67%).

In both cases, ROPE and BF model comparison tests were performed to illustrate how these methods can either provide converging evidence for the alternative hypothesis in the presence of a strong effect, or how they can instead show relative evidence for the null hypothesis when an effect is essentially zero. For the Proactive versus Baseline NP effect, the BF model comparison approach indicated decisive evidence for the alternative hypothesis (M₁), with a BF₁₀ = 1.72 × 10⁸. In line with this result, the entirety of the 89 and 95% HDIs fell outside the ROPE region for this effect (see Figure 3A, for the 89% HDI result). In contrast, the Proactive versus Reactive NP effect exhibited strong evidence for the null hypothesis (M₀), both in terms of the computed Bayes Factor, which had a BF₁₀ = 0.038, and the 89% HDI falling completely inside the ROPE range (Figure 3B). However, follow-up analyses with the 95% HDI indicated that a small portion of the posterior estimate did fall outside the ROPE range (i.e., 0.43%). While this more conservative threshold would technically indicate there is inconclusive evidence for M₀ versus M₁, the default variant of this ROPE procedure combined with BF model comparison overall indicated strong evidence favoring M₀ over M₁. Finally, a sensitivity analysis employing LOO-CV confirmed there was a meaningful Proactive – Baseline NP RT effect (∆_M1–M0elpd_LOO = −20.4, ∆_M1–M0SE_LOO = 4.8) but no evidence for a Proactive – Reactive effect (∆_M1–M0elpd_LOO = −0.3, ∆_M1–M0SE_LOO = 0.2). However, it is important to acknowledge that by itself, the LOO-CV metric does not directly quantify evidence for M₀ in the latter case. Nevertheless, when considered together, the collective results demonstrate how the Bayesian framework provides various hypothesis testing methods, which can be used in a convergent manner to determine the degree of evidence favoring specific alternative or null hypotheses.

The HBR analyses strengthened both conclusions reported in Tang et al. (2023) regarding the NP RT effect. For the Proactive versus Baseline contrast, the HBR analyses confirmed that there was indeed very strong evidence for the reliability of the effect, both in terms of BF and ROPE methods. However, regarding the more theoretically and methodologically critical Proactive versus Reactive contrast, the HBR analyses even more strongly confirmed the presence of a null effect. The equivalence of Proactive and Reactive NP performance was already suggested in Tang et al. (2023), not only with RT patterns, but also with the results on NP error rates, which were in the opposite direction of theoretical predictions (and this pattern was confirmed with the current HBR analyses; see Supplementary materials). Yet, the hypothesis testing made possible through HBR analysis allowed the conclusion of a null effect in the Proactive versus Reactive comparison to be more strongly asserted beyond what was possible in Tang et al. (2023).

In contrast with the null hypothesis significance testing (NHST) framework, which permits only a binary assertion regarding whether the null hypothesis can or cannot be rejected, Bayesian hypothesis testing enables more fine-grained statements regarding the strength of evidence for both the null and alternative hypotheses. This advantageous property of Bayesian frameworks has enabled the development of heuristic (i.e., post-hoc) approaches from which to apply BF metrics, to data that were primarily analyzed with NHST approaches (e.g., as was done in Tang et al., 2023, which did report BF indices for the NP RT and error measures). Yet it is important to note that some researchers prefer the ROPE approach as a complementary and potentially stronger metric from which to identify effects that for all practical purposes can be treated as equivalent to zero. Estimation tests for ROPE are more directly aligned with the Bayesian rather than NHST framework (given that they rely upon generation of a full posterior distribution); yet because of this feature, they cannot be computed in a post-hoc, heuristic manner (Kruschke, 2014). Here, we took advantage of the fully-fledged HBR analyses, which enabled both BF and ROPE metrics to be computed.

Nevertheless, it is important to acknowledge both metrics still inherit some of the same limitations associated with NHST, in that they also rely upon categorical criteria to make somewhat broad inferences about a tested hypothesis. Furthermore, the derivations that underlie NHST and Bayesian metrics can lead to diverging conclusions in the case of large sample sizes, which will be biased toward the alternative or null hypothesis, respectively, (i.e., Lindley’s paradox) (Lindley, 1957). However, we still believe that, when utilized together, our specific application of ROPE and BF model comparison not only align with previous conclusions, but also provide more granular inferences than those allowed by NHST. More specifically, these metrics provided convergent, and thus stronger, evidence in favor of a null Proactive versus Reactive NP effect (i.e., both BF < < 1/10 and entirety of the [89%] HDI interval within the ROPE)⁸. In particular, the cumulative results across samples and analytic approaches permit the strong conclusion that, despite the goal of the DMCC project to validate a robust indicator of proactive control for the Sternberg task, this goal was not accomplished. Future work should reconsider task design alterations that have greater potential to elicit unambiguous behavioral markers of proactive control.

In the Stroop task, the congruency cost is of theoretical interest as a potential index of proactive control engagement. In particular, the deployment of proactive control is thought to bring not only performance benefits on high-demand incongruent trials, but also performance costs on congruent trials. This cost–benefit pattern should be most pertinent when contrasting Proactive and Reactive conditions, given that proactive control is postulated to involve preparatory mechanisms that enable biasing away from the ‘word’ dimension across all item types, whereas reactive control biasing is thought to occur only after detection of the relevant color feature and the presence of incongruence, so should not impact congruent items. Further, this key theoretical prediction is most stringently tested on diagnostic/PC-50 items.

Notably, the theoretical prediction regarding congruency cost was not confirmed in Tang et al. (2023). Specifically, in that analysis, the congruency cost was found to be statistically unreliable. However, the unreliability of the congruency cost effect reported in Tang et al. (2023) also contrasts with its observed robustness across multiple other companion papers (Gonthier et al., 2016a, 2016b; Bugg, 2017; Ileri-Tayar et al., 2025). Since the congruency cost is a small effect (~10 ms), making it potentially sensitive to both measurement error and incorrect assumptions regarding data structure, an important question arises on whether modeling the specific properties of the sample RT distribution impacts the identification of congruency cost effects, in terms of sensitivity and reliability. The flexibility of HBR models to assume both a hierarchical structure and non-Gaussian distribution of the data may therefore be a useful approach to elucidate the presence and strength of the congruency cost in these samples. We thus explored the robustness of the congruency cost metric as a function of the RT distribution employed in the analysis.

To evaluate the congruency cost, the RT data were first modeled using a shifted log-normal distribution to test for differences between the Proactive and Reactive control modes, restricting analyses to correctly responded diagnostic congruent trials. The following model was used for estimation: RT ~ 1 + mode + (1 + mode | ID). Mode was dummy coded with Reactive as the reference category to demonstrate the relative congruency cost (i.e., increase in reaction time) of proactive control for congruent diagnostic items. The intercept and mode were entered as random effects nested within subject, so that the congruency cost was also treated as a random effect.

In contrast to the results found in Tang et al. (2023), but in line with theoretical predictions and other reported findings, the congruency cost was reliably present in the 2018 dataset when analyzed with an HBR model that assumed a shifted log-normal distribution for RTs ( β = 0.02, se = 0.01, 95% HDI = [0, 0.05], pd. = 98.99%). Conversely, when this same effect was modeled with a conventional Gaussian distribution, the effect was not statistically reliable ( β = 8.92, se = 5.37, 95% HDI = [−1.81, 19.27], pd. = 95.09%). These contrasting patterns directly demonstrate that the choice of distribution can affect the inferences made about a small effect like the congruency cost.

To more fully understand this finding, we next sought to isolate which properties of the shifted log-normal distribution were essential for appropriately modeling this effect. To do so, the congruency cost was analyzed with a third distributional function: the ex-Gaussian. This distribution also captures the positive skew of response time data and additionally has a long prior history of work illustrating its increased sensitivity in modeling Stroop RT patterns (Heathcote et al., 1991). If capturing both the bulk of fast responses times along with a long tail of slow response times is necessary to accurately model the congruency cost, then the HBR model assuming an ex-Gaussian distribution should match the qualitative finding of the model using a shifted log-normal, but not Gaussian, function. Indeed, this prediction was confirmed, as a model assuming an ex-Gaussian distribution also found a statistically reliable congruency cost ( β = 9.87, se = 3.03, 95% HDI = [3.98, 15.8], pd. = 99.94%).

A quick visual inspection of the posterior predictive checks (PPCs) for each of these models provides clear evidence that the shifted log-normal and ex-Gaussian distributions produce an improved fit to the observed RT data, relative to the Gaussian distribution (Figures 4A–C). However, the difference in predictive accuracy between the ex-Gaussian and shifted log-normal is less clear with this approach; we therefore next sought to more directly quantify the relative predictive accuracy of each distributional function in modeling the congruency cost. Through BF and LOO-CV model comparisons, we defined the ex-Gaussian model as M₀, the shifted log-normal model as M₁ and the Gaussian model as M₂. The BF model comparison metric showed decisive evidence favoring the shifted log-normal (M₁) over the ex-Gaussian (M₀) (i.e, BF₁₀ = 7.4 × 10²⁴, or BF₀₁ = 1.35 × 10⁻²⁵), while the Gaussian model had the worst fit (M₀ / M₂; BF₂₀ = 0, or BF₀₂ = Inf). However, LOO-CV model comparison revealed that the difference in predictive accuracy between the shifted log-normal and ex-Gaussian was not meaningful (∆_M1–M0elpd_LOO = −66.9, ∆_M1–M0SE_LOO = 85.8), although the Gaussian was clearly worse than the shifted log-normal (∆_M1–M2elpd_LOO = −7563.9, ∆_M1–M2 SE_LOO = 338.3).

Given the discrepancy between the BF and LOO-CV model comparison conclusions, we next sought to investigate how the incorporation of additional information, from the 2020 sample, would impact the results. Interestingly, while the same quantitative and qualitative conclusions regarding the congruency cost were drawn from the cumulative 2020 HBR models that assumed either a shifted log-normal ( β = 0.03, se = 0.01, 95% HDI = [0.01, 0.04], pd. = 99.97%) or ex-Gaussian ( β = 11.11, se = 2.45, 95% HDI = [6.22, 15.77], pd. = 100%) distribution, the 2020 HBR model with a Gaussian function also showed strong evidence for this effect ( β = 11.55, se = 3.88, 95% HDI = [3.98, 19.21], pd. = 99.83%), aligning with the qualitative conclusions of the other models; however, the PPCs for the 2020 HBR models again demonstrated the same pattern of increased predictive accuracy of the skewed distributions, as before (Figures 4D–F). Furthermore, we once again ran BF and LOO-CV model comparisons. Despite the inclusion of additional data, we found the same contrasting pattern of results across model comparison metrics in which the BF measure favored the shifted log-normal over ex-Gaussian (i.e., BF₁₀ = 8.4 × 10²¹), while the LOO-CV indicated no difference in predictive accuracy between the two metrics (∆_M1–M0elpd_LOO = −36.4, ∆_M1–M0SE_LOO = 58.3), but again that they were superior to the Gaussian (BF₂₀ = 0; ∆_M1–M2elpd_LOO = −4079.3, ∆_M1–M2SE_LOO = 226.9). This consistent discrepancy regarding the differential benefits of the shifted log-normal and ex-Gaussian likelihood functions reinforces the value of applying multiple analytic approaches to avoid drawing potentially premature conclusions. More importantly to the conceptual principle of this section, both distributions were able to reliably identify the congruency cost within our data.

The results provide clear evidence that the identification of a subtle RT effect, such as the congruency cost, can be impacted by the chosen distribution to model it. With HBR modeling approaches, it is quite easy to select from a range of possible distributions, and then directly compare their performance to the available data via posterior predictive checks (PPCs) as well as BF and/or LOO-CV model comparisons. While such metrics are not meant to parse apart subtler differences in predictive accuracy between distributions that already capture the general properties of the data, the results from all model comparison metrics proved that the shifted log-normal and ex-Gaussian functions were both superior to the Gaussian distribution in reliably identifying the congruency cost. However, we also demonstrated that even the advantages garnered from skewed distributions versus the standard Gaussian distribution were influenced by the inclusion of both datasets.

At first blush, the findings related to the inclusion of both datasets may seem to indicate diminishing benefits to the use of more accurate distributions for RT data. Yet a more important interpretation relates to the efficiency and flexibility of the HBR framework. Specifically, the use of more appropriate distributions, such as the shifted log-normal or ex-Gaussian, may reliably detect RT effects of interest with less data than that required from a conventional Gaussian model. The degree of efficiency is likely to be moderated by the properties of each dataset. As a result, we advocate for a routine practice of directly comparing multiple likelihood functions to assess their relative predictive accuracy for the data on hand.

For this purpose, PPCs allow for a quick visual comparison of different likelihood functions on the observed data, while BF/LOO-CV model comparison approaches can be applied as computationally intensive, but more quantitatively direct, metrics to determine the best-fitting model. Critically however, these recommendations do not imply that predictive accuracy should be the decisive factor while selecting among different likelihood functions. Instead, likelihood functions can be selected and investigated in a flexible manner, to address various methodological goals. For example, it has been argued that a key advantage of the shifted log-normal function relates to its clear mechanistic interpretability (i.e., subject-level RT distributions should be decomposed into a non-decision time component and evidence accumulation process that results in a rightward skew) (Haines et al., 2025). Our goal here was primarily to illustrate the simultaneous predictive and theoretical limitations that come with fitting Gaussian distributions onto RT data. In addition to this more universal point however, these collective findings also practically demonstrate how the additional power gained by alternative likelihood functions can appropriately capture the congruency cost as a reliable metric of proactive control.

Within the Cued-TS paradigm, a primary indicator of interest for reactive control was the task-rule congruency effect (TRCE) for errors. The inclusion of punishment cues, presented shortly before stimulus presentation and paired with high-conflict items, was hypothesized to elicit late-stage reactive control, corresponding to enhanced performance on incongruent trials in the Reactive mode, relative to the Baseline and Proactive modes. Since a strong association should manifest between incongruence and punishment, the performance benefits were expected to extend even to non-incentivized incongruent trials, based on the detection of perceived conflict. This improved performance on incongruent trials was in turn expected to cause a relative reduction in the TRCE error effect within the Reactive mode. Initial evidence for this Reactive TRCE error effect was reported in Tang et al. (2023). However, the task design for the Reactive variant was novel at the time, so prior findings could not be considered. For such cases, the HBR approach is especially useful to either provide further validation, or to instead challenge initial findings from novel task manipulations.

When considering binary outcome variables, such as a correct or error response, the use of a logistic function operationalizes each observation in terms of the underlying probability that one outcome versus the other will occur. This statistical approach not only circumvents the distortion of results that can arise when making incorrect modeling assumptions with proportion data, but it also enables direct consideration of trial-level variability within each subject when applied through a hierarchical model. The trial-level granularity achieved by hierarchical logistic models cannot be assessed through the conventional use of error rates, which instead rely upon condition-based averages that omit such information. Indeed, this access to trial-level data allows for the effective modeling of more complex random effects within each participant. In other words, an error rate model will be more likely than a logistic regression model to run into convergence issues while modeling interaction terms that vary by subject. To effectively illustrate the specific advantages of modeling accuracy data as log odds versus error rates, both the logistic regression and error rate models were matched to include the same simple random effects (see Supplementary material Results, for qualitatively similar results from simultaneously modeling the TRCE error effect with a logistic function and maximal random effect structure). Here we illustrate how HBR models with a logistic function and a hierarchical structure (i.e., trials nested within conditions, further nested within individuals) can provide a more rigorous test of the Reactive TRCE effect than what was possible through the original analyses provided by Tang et al. (2023).

The task-rule congruency effect (TRCE) for errors was modeled as the relative log-odds of making an error on incongruent trials versus congruent ones, with the following model estimating the difference in this effect across modes: Correct ~ 1 + con.id*mode + (1 + con.id | ID). The outcome variable ‘Correct’ was coded so the model predicts the log odds of making an error (i.e., correct = 0; incorrect = 1). The predictor ‘con.id’ was dummy coded to assess the difference in performance between incongruent and congruent trials (i.e., the TRCE effect). The predictor ‘mode’ was dummy coded, with separate models either using Baseline or Proactive as the reference category to test the relative effect of the Reactive mode on the log odds to make an error. Finally, the interaction effect between ‘mode’ and con.id’ reflects the key parameter of interest as the difference in TRCE error effect between modes. Both intercept and con.id are treated as random variables nested within subject to model the TRCE error as a random effect. Only non-incentivized trials were selected for analyses to allow for direct comparisons between the Reactive mode versus Baseline and Proactive. Furthermore, only biased (i.e., mostly congruent) trials were included since they were expected to elicit the largest TRCE error across modes. The key hypothesis was that the TRCE error would be reduced in the Reactive mode versus the Baseline and Proactive modes.

In contrast to these predictions however, the 2018 HBR models indicated that the TRCE error was actually increased, rather than decreased, in the Reactive as compared to Baseline mode ( β = 0.37, se = 0.11, 95% HDI = [0.15, 0.58], pd = 99.98%) (Figure 5A); furthermore there was little evidence for a difference between the Reactive and Proactive modes ( β = 0.14, se = 0.11, 95% HDI = [−0.08, 0.35], pd = 89.62%) (Figure 5B). These findings were qualitatively distinct from those reported in Tang et al. (2023). Specifically, in Tang et al. (2023), it was reported that the TRCE error was reduced in Reactive relative to both Baseline and Proactive, with the latter contrast showing high statistical reliability. Given this discrepancy in findings, we were interested in testing whether the logistic function provided a more accurate way to model the TRCE error.

Rather than summarizing the error rates as proportions and incorrectly assuming a Gaussian distribution, the use of logistic regression directly models the probability of an error through a Bernoulli distribution, transforming the data into log odds so that the function can more sensitively capture differences between low and high probabilities. While standard approaches assume a linear effect between predictors and the outcome variable, probabilities typically follow a sigmoidal function, in which the strength of probability changes is dependent on their initial values (i.e., the largest absolute change will be around p = 0.5, and the smallest changes will be near their extremes, due to their bounded nature). By utilizing a logit link to transform the data onto a log-odds scale, logistic regression will appropriately strengthen differences between conditions when the reference group is close to floor/ceiling. Since a ceiling effect will particularly apply to congruent trials across modes versus incongruent trials, this transformation provides a potential explanation for why the current results were in the opposite direction of initial findings. To test this interpretation of the findings, the results were re-analyzed by using error rates rather than log odds as the outcome variable. Indeed, when running a HBR model on the 2018 data, but with error rates, a consistent pattern with Tang et al. (2023) is observed, with a strong reduction in the TRCE effect observed for Reactive versus both Baseline ( β = − 0.02, se = 0.01, 95% HDI = [−0.03, 0], pd. = 99.56%) (see Figure 5C) and Proactive ( β = − 0.06, se = 0.01, 95% HDI = [−0.07, −0.04], pd. = 100%) modes (see Figure 5D).

The Cued-TS findings provide a clear demonstration that initial conclusions regarding effects from novel paradigms may not be consistent across different analytic approaches and metrics, thus highlighting the importance of carefully selecting statistical methods that lead to accurate and precise results. More specifically, the analyses modeling the difference in TRCE error between modes found there was only very weak evidence for a reduced Reactive effect (relative to Proactive) when utilizing a logistic function. Although these results conflict with previous conclusions drawn by Tang et al. (2023), greater confidence can be had in the validity of the results obtained from the HBR analysis, due to the now well-established benefits of hierarchical logistic regression models (Jaeger, 2008; Dixon, 2008; Houpt and Bittner, 2018).

However, it is also important to clarify that these findings are not in strong contradiction with the theoretical claims of the DMC framework, regarding benefits of Reactive control for Cued-TS performance. Indeed, when restricting analyses to just the incongruent trials, there were clear observed performance benefits present in the Reactive mode, regardless of whether the outcome variable was modeled in terms of error rates or the log odds to make an error (see Supplementary material for additional analyses on the simple and main effects of condition on accuracy performance). The primary distinctions between the models were instead on congruent trials, which had overall very low error rates (i.e., less than 5%). As a result, differences between control modes for these trials were not reliably detected in the models using summary error rate measures but were found to be consistently greater when modeled in terms of log odds. Since hierarchical logistic regression models are still not the norm for analyses of error rates in cognitive control tasks, it is possible that other effects involving low error rate conditions (e.g., congruent trials) have also been missed in prior studies. Consequently, an important direction for future work would be to better understand the circumstances in which task manipulations to the Cued-TS paradigm, such as that of the reactive control variant, might lead to performance improvements for both congruent and incongruent items.