The perspective that planning, reasoning, decision making, and control requires information processing resources can be partially motivated by, but it can also partially explain, the concept of bounded rationality (Bratman, 1987; Friston et al., 2013; Lieder and Griffiths, 2020). Bounded rationality essentially emphasizes that limited cognitive capacities prevent us from thinking problems fully through, deeming our decisions sub-optimal (Simon, 1955). To save resources, we employ heuristics and often act habitually rather than in a deliberate, goal-directed manner, which leads to typical errors in our reasoning and decision making (Kahneman, 2003; Gigerenzer and Gaissmaier, 2011). Thus, sub-optimal behavior is caused by our limited information processing resources, which struggle to process more complex tasks optimally due to their higher demand of information processing resources.

Ortega and Braun (2013) proposed a free energy-based formalization of bounded rational decision making that is directly derived from thermodynamics. Decision making requires to steer distributions away from a prior distribution toward a decision-peaked distribution. Changing the distribution requires information processing resources (e.g., to solve a task), while its investment is rectified by the corresponding expected utility gain (e.g., actually solving the task). Genewein et al. (2015) enhanced this model by enabling it to selectively recruit (provided) hierarchical encodings to further minimize cognitive effort. For example, the group of food items or dangerous animals is recruited, subsuming the entities that match with a respective interaction class, leading to a more efficient decision making process.

Focusing only on the cost of cognition—implicitly assuming that the effort will be worth the while and thus maximum available effort will be spent—(Zenon et al., 2019) formalize an optimization model that consists of three summands: the mutual information I between (i) the expected and actually perceived state of the environment, (ii) the expected prior (preparatory) response and actual posterior response, given the actually perceived state, and (iii) the assumed context T dependent stimulus-response mapping vs. the actual posterior response. The formalism thus assumes independency between stimulus processing effort, task-dependent response effort, and context-dependent response effort. The authors then illustratively address inferences in the Strop task and also discuss the consequences in task switching and multitasking, which we will follow up upon below.

REPI enhances the model of Zenon et al. (2019) and Genewein et al. (2015) in that it assumes an inner, event-predictive, generative model, which contains distributed predictive encoding densities. In line with Ortega and Braun (2013) and Zenon et al. (2019), REPI equates cognitive effort with the focused activation of current mental content away from both residual, previous task priors and latent, habitual priors. The mental content, however, is made more explicit and the active inference dynamics as well as the involved effort is simulated as a process model.

While this paper does not address the question how cognitive effort is actually spent in the brain from a neuro-physiological perspective, it may generally be assumed that evolution has designed our brains to generate flexible and highly adaptive behavior—including socially-interactive behavior—in a very resource-effective manner. Accordingly, neuro-physiological evidence is accumulating that synchronized and coordinated neural firing is indeed effortful. It appears that processing resources are dominantly determined by the effort to generate and transmit signals via biochemical dynamics and neural spikes (Laughlin, 2001; Sengupta et al., 2010). Accordingly, energy is required for the maintenance of a wakeful state, in which neurons communicate with each other across cortical and sub-cortical areas by selective, rhythmic synchronization and coordination (Engel et al., 2001; Bastos et al., 2015; Fries, 2015; Daume et al., 2017; Misselhorn et al., 2019).

These and related insights are in agreement with the perspective that cognitive effort is needed to counter uncertainty, which, in neuro-physiological terms, is closely related to focusing the mind by means of precise, coordinated neural and biochemical activities (Daume et al., 2017; Misselhorn et al., 2019). Most recent research has even linked such resource constraints to predictive coding theory, showing that resource-efficient encodings are predictive and develop in recurrent neural networks naturally, when the goal is to solve a particular tasks resource-efficiently (Ali et al., 2021).

In sum, it may be said that the current's mind focus with its neuro-physiological activities essentially constitutes current mental content, Thereby, the maintenance of a stronger processing focus and dynamic refocusing is effortful. In order to make the mental content more explicit on a computational and algorithmic level, though, the active inference principle conjoint with event-predictive cognition offer suggestions.

ActInf essentially highlights that our brain predicts the future given its past experiences, even including its genetically-gathered experiences (Hohwy, 2013; Friston et al., 2015a; Clark, 2016). Importantly, though, our brain does not predict the future—or generate models of our environment—for its own sake. Rather, it does so for optimizing the generation of (highly adaptive) behavior, which is elicited to maintain and foster both internal homeostasis and model consistency (Friston et al., 2015b). As a result, the developing predictive encodings, which constitute our individual latent beliefs about the world, serve a self-motivated purpose. Together, our individual ActInf processes, which unfold within our individual predictive encoding structures, generate our individual current state of mind.

In relation to behavioral psychology, ActInf offers a formalization of the ideomotor principle: initial evolutionary-determined, reactive behavior becomes associated with its effects. Later, behavior is elicited by the desire to generate the associated effects again under different but related circumstances, leading to further learning (Hoffmann, 1993, 2003; Stock and Stock, 2004; Butz and Kutter, 2017). ActInf thus formalizes how internal generative structures inevitably reflect the (hidden) outside reality—or at least those components of reality that tend to affect the well-being and mere existence of the living being in question (Friston, 2009; Butz and Kutter, 2017).

The ActInf formalization subsumes three distinct, highly interactive inference processes:

In the REPI model, we focus on simulating the first two processes, providing presumably learned predictive encoding structures as the available generative model.

Since the reminder of this work focuses on the inference of behavior and involved effort, a formalization of prospective ActInf will be useful. To do so, it is necessary to specify the to-be-minimized free energy (Friston et al., 2015b, 2018; Gumbsch et al., 2021a). Given a particular policy π, which generates actions a in the light of the current belief state b_t and system needs n_t considering a particular, possibly adaptive, temporal horizon τ = {t, t + 1, t + 2, …, t + T} with depth T, the anticipated free energy when pursuing policy π can be written as:

where prior densities are denoted by P while expected posterior densities are denoted by Q. The KL divergence is calculated as the expected divergence of posterior observation densities, when following policy π starting from belief state b_t, from the prior desired observation densities P(o_τ | b_t, n_t) over the temporal horizon τ. Meanwhile, predicted uncertainty quantifies the entropy H over expected future observations dependent on expected policy-dependent future belief states E_{Q(bτ|bt, π)}, when following policy π starting from belief state b_t.

Driven by the purpose to minimize free energy, and thus to maintain internal structure, Equation (1) essentially quantifies a payoff, which may be equated with the decrease in free energy compared to the possible development of free energy given inactivity.

Meanwhile, though, invoking activities and focus will come at a cost, which has been equated with the change from a prior to a posterior distribution in terms of KL divergences (Ortega and Braun, 2013; Genewein et al., 2015; Zenon et al., 2019):

which quantifies the investment cost of changing a prior probability density over some representational generally continuous space x ∈ ℝ into a posterior distribution over this space given a source of evidence e.

When furthermore assuming a density E over these evidences—be they sensory information about body or environment or also internal model expectations or motor activity—a more accurate cost estimate takes the weighted mean over the possible evidences, yielding a form of mutual information:

According to this equation, the cognitive effort to direct behavior and thoughts toward desired states and away from expected observational uncertainty (cf. Equation 1) can be equated with the mutual information I_Q between the state-informative evidence density E, which we here assume to be discrete for simplicity reasons, and state estimation prior P(x), given the conditional model Q, which maps evidence onto posterior state estimations.

The system then ideally finds the optimal behavioral policy π^* that minimizes anticipated cognitive costs, that is, information investment costs, and anticipated free energy. When the gain in free energy minimization is larger than the cognitive costs, the effort is worth its while. Effort costs and utility gains thus need to be well-balanced. Accordingly, a weighting factor has been introduced as a Lagrangian multiplier that factors the optimization cost into the expected utility gain (Ortega and Braun, 2013), allowing an adaption in how much cognitive investment is currently worth the while. This factor may essentially be related to the current cognitive resources available to an agent, which is related to inner homeostasis and thus intrinsically part of Equation 1. In the future, the fixed horizon in Equation 1 may thus be subsumed by a dynamic horizon that may optimally take planning costs and available resources into account (cf. Gumbsch et al. 2021a for a similar proposal). For now, a simplifying alternative to balancing the two aspects lies in focusing on solving one particular task and assuming that maximal available cognitive effort is spent on solving it at any current point in time. In this way, cognitive costs dominate behavioral and cognitive processing and decision making: the larger the needed density changes to process a task, the slower the behavioral decision making.

To summarize: there is compelling evidence that our brain is developing generative predictive model(s) by means of ActInf. Concurrently, the developing models enable ActInf to focus our computational resources on anticipated needs. Combined with information processing cost considerations, ActInf thus controls how computational resources are invested, considering the future in the light of the present, the current needs, and our so-far gained behavioral and model-predictive knowledge. It continuously adapts the currently active internal models in the light of the accumulating evidence, develops these models further over time, and determines current attentional dynamics, reasoning, decision making, and behavioral control by attempting to minimize expected free energy (Equation 1).

The biggest criticism on ActInf and the information cost formalization from the cognitive science side lies in the testability and falsifiability of the involved formalism. This is particularly the case because the involved generative model is not specified. Indeed, ActInf may play out not only during ontogenetic development and the hear-and-now, but even on evolutionary time-scales, creating particularly well-suited bodily properties (such as our hands), developmental pathways (such as our brains' modular structure), and consequent inductive learning biases (Pfeifer and Bongard, 2006; Butz and Kutter, 2017)—all influencing generative model development. We thus now scrutinize likely properties of the developing generative model.

Considering the effort of focusing densities in relation to EPCog, it will be effortful to selectively activate precise event-predictions, or foci, away from previous foci as well as from uncertainty and general lethargy. That is, cognitive resources are needed both to activate new foci, to deactivate previous ones, and to invoke structure away from uncertainty, that is, away from a uniform distribution (maximal entropy).

Because the ease of activating particular densities will depend on the (i.e., learned) predictive encoding structures available, the more unusual a particular event is, the more effortful it will be to encode it. Similarly, the more unusual, unexpected, or counter-intuitive the binding of individual event components is, the more effortful the binding will be. Meanwhile, inhibiting competing events as well as considering multiple alternative events will be effortful. Even more effortful will be the successive activation or parallel maintenance of mutually inhibitory encodings, such as when a stimulus needs to be mapped first onto the right response side and then onto the left response side, due to a task switch.

Since considered events may be integrated into scenes, the complexity of and the prior knowledge about a considered scene will also play crucial roles. First, the ActInf of particular scene aspects and un-mentioned but event-relevant components for the production of coherent mental content will be effortful. The more common and well-known an encoded scene, the fewer cognitive resources will be needed to activate its compressed scene-characteristic event structures. The activation of scene-unusual, or even scene-contradictory, attributes will be resource-demanding, because their activation will be inhibited by the scene code. When alternative scenes need to be considered, their compatibility or logical temporal progression will influence the effort to maintain them in parallel.

For example, the entities and interactions necessary to prepare a cup of tea can be effectively encoded because we know all involved entities and interactions well. Its encoding is even less demanding, once we have developed an even more integrative event-predictive schema, which predicts the involved sub-events, entities and conditional event progressions a priori, for example, when considering “preparing a cup of tea” (Kuperberg, 2021). Similarly, much less cognitive effort is necessary when succeeding in processing a multitude of stimuli in an integrative, Gestalt-oriented manner—as is the case when, for example, recognizing a human figure walking in a point-light motion display (Johansson, 1973; Pavlova, 2012; Sadeghi et al., 2021).

In sum, the preparation and processing of actual events and event progressions within scenes can be assumed to be the more resource demanding:

The activation of a particular event will also depend on previous content activations: the more the current event differs from the previous one, the more effortful the switch. Moreover, the fewer this particular event switch has been experienced in the past, the more resource-demanding it will be, because it is less expectable.

When facing a concrete task, various components of cognitive effort are involved. During task preparation, suitable event-predictive task-specific priors away from previous posteriors, latent habitual priors, and uniform entropy need to be activated. These include priors on the expected, relevant sensory stimulus information, the expected mappings—or, more generally speaking, computations—needed to transform the stimulus into an appropriate response, as well as priors over potential responses. When processing a task-informative stimulus these priors will be adapted further. For example, in a priming task the prime may bias response tendencies. Similarly, when the stimulus informs about likely upcoming actual imperative stimuli or the upcoming task, stimulus priors, stimulus-response mapping priors, and response priors will be focused further.

When processing the task-imperative stimulus then, super-threshold activities need to be reached while inhibiting incompatible priors in perception, in competing task sets, and in the action choices. This principle is closely related to dynamic neural field theory (Schoener, 2020), where sub-threshold activities in an activation field correspond to preparatory, but also latent and residual, priors. When preparation is maximally effective, expected and fully prepared stimuli will be processed fastest and with least effort, and they will yield smallest response errors. Meanwhile, when a response is overly prepared, it may be executed prematurely, or incorrectly when the imperative stimulus does not conform to the expected stimulus. This has been, for example, studied in detail in go-nogo tasks.

In the following section, concrete examples of such encodings are provided and REPI is implemented as a dynamic processing model. Model simulations yield the Simon Effect and Task Switching costs. More detailed model evaluations are needed to further verify or falsify the ability of the architecture to model more intricate task switching observations. We end with a discussion that sketches-out the potential of REPI to model other psychological experiments and typical observable behavioral effects.

In order to explain the mentioned psychological behavioral phenomena not only qualitatively but quantitatively, we now introduce an actual algorithmic implementation of the REPI model. In this implementation, we focus on modeling standard tasks that elicit the Simon effect as well as task switching costs. Figure 1 shows a sketch of the task-oriented REPI model implementation, where sensory information flows into the model from the left, task-informative information from the top, response tendencies are sent toward the right, while anticipated response consequences are sent back to the left.

In all these tasks, we assume a dynamic perceptual space O, where at a point in time t prior densities are given by P(Ot) and posteriors by Q(Ot). Similarly, we denote the relevant task space by Ψ, with priors P(Ψ_t) and posteriors Q(Ψ_t). As an additional latent observational bias space, we model B. Moreover, we model response side dynamics in S and the actual response dynamics in R.

During a trial, these densities are dynamically adapted toward particular target values. In each iterative update step, all densities are adapted from priors to posteriors, which constitute the priors in the next time step. The update is computed as follows:

where α denotes a rate factor, ϵ_t the currently available cognitive resources per time step (e.g., per second), X∈{O,B,Ψ,S,R} denotes the density that is adapted, and G(Xt) denotes the current targeted density, that is, the ideal goal state REPI strives for via active inference. In the simulations below we set the simulation speed to 10 ms per iteration and thus α = 0.01, while we set the resource strength to ϵ = 6.

When focusing on one particular task, the task density P(Ψ) will strive to adapt toward a fully focused density, which would, for example, correspond to (Ψ|ψ_t = 1) = [1, 0, …, 0] in the case of a discrete one-hot encoding when focusing on the first task. While preparing for a particular task, or a set of tasks, the task densities will adapt to the average mixture of each possible task. Finally, to model a latent readiness to process stimuli habitually, a general latent readiness may be modeled with an offset factor λ, such that, for example, a maximal readiness for processing two equally probable tasks would correspond to:

where we assume additional dormant tasks in the middle with zero activity.

The observational density, which focuses on processing the imperative stimulus, will generate task-particular, top-down, goal-directed stimulus expectation densities G(Ot|P(Ψt)) dependent on the current task density Q(Ψ) during the stimulus preparatory phase. Once the stimulus is presented, though, processing will switch to bottom-up evidence accumulation by setting G(Ot|ot) to, for example, a one-hot encoding of the discrete stimulus signal. While information fusion options may be included here, at the moment the implementation sets G(Ot) either to the top-down or the bottom-up observational goal, dependent on the current stimulus availability. Meanwhile, potential stimulus biases may be processed within the bias space B, where no stimulus side bias corresponds to G(Bt)=[0.5,0.5]. The response side density is continuously updated task- and observation-estimate dependently. That is, G(St) is set to P(S|Q(Ot),Q(Ψt)), where the respective task sets are implemented by means of task-respective mapping matrices, which map individual objects to task-corresponding response sides. Finally, once a stimulus has been perceived, the actual response density is adapted away from a uniform response density toward the current processing side density, that is, G(Rt) is set to Q(St). The actual response is then triggered once a particular response threshold θ_r, which we typically set to 0.9, is reached.

Assuming, as specified above, that the full focus will lie on solving tasks during an experiment appropriately, the costs for solving a particular trial may then be quantified by the dynamically unfolding mutual information between current response tendencies and desired response tendencies given the beliefs in the current task and the current stimulus situation, that is:

In the future, several advancements in this cost computation and the distribution of cognitive resources are imaginable. These include (i) the computation and integration of costs based on mutual information in the other spaces, including the task and observational spaces, (ii) the adaptive, non-uniform, active-inference-driven distribution of computational resources over the considered density spaces, that is, adapting ϵ_t in Equation (4).

The Simon effect characterizes the effect that participants respond to a stimulus faster with their ipsilateral than with their contralateral effector even when the position of the stimulus presentation is irrelevant for the actual task (Erlhagen and Schöner, 2002; Cho and Proctor, 2010). From the event-predictive ActInf perspective, paradigms that study the Simon effect essentially study the presence of latent, spatial stimulus-response mappings and their latent activity strength relative to the task-specific activities, participants attempt to focus on. The latent response mapping corresponds to an ipsilateral response to lateral stimuli, such as by an overt eye saccade, by a covert direction of attention, or by an actual manual interaction—where in the last case handedness plays an additional important role. For example, we respond to a visual stimulus by looking at it, by touching it (e.g., a power switch), avoiding it (e.g., an insect), or actively pursuing interactions with it (e.g., when washing our hands or grasping a mug). As a result, ipsilateral, stimulus-response mappings, which are predictive by nature, are latently active in our minds, facilitating many everyday interactions. In a particular experiment that elicits the Simon effect, though, these latent mappings get in the way and are hard to fully inhibit.

To model the dynamics and estimate the relative effort to solve individual trials, we ran REPI on a simulation of a typical Simon effect-eliciting task. We assumed that the task was to respond to the form of an object, ignoring its color as well as its location on the screen. Objects were simulated to be presented either centrally to the right or to the left of the center. The trial started with the presentation of a fixation cross, followed by the imperative stimulus 500 ms afterwards. The inter trial interval was set to 100 ms.

In our REPI implementation, we used the general setup shown in Figure 1 focusing on the form task and leaving the latent activity active between a value of 0.1 and 0.2, dependent on if the fixation cross or the imperative stimulus are shown or not yet, respectively. Task cue and stimulus goal signals are set to 0.9 and 0.97, respectively assuming a latent task activity of 0.1 and a slim chance to perceive one of the other objects with a probability of 0.03, respectively¹.

Figure 2 shows the unfolding dynamics of the simulation over the first ten trials. Moreover, the dynamics of the costs, which focuses on solving the task quickly and correctly, are plotted. Figure 3 shows the simulated reaction times as well as the resource investment determined over the time from the imperative stimulus until the response was triggered. The results clearly show the Simon effect: a stimulus that appears on the right or left is responded to faster with the compatible response side, while a stimulus in the center does not exhibit this influence. Moreover, response repetition benefits are visible, when the response side needs to be switched. The cost measurement C (cf. Equation 1) confirms that faster responses go hand-in-hand with smaller cognitive effort. Future work may also model the STROOP effect with similar principles. Here, an open question is whether modifications of the task priors will be enough to model the dominance of the word reading response in all its detail.

The task switching literature is full of additional insights on how quickly event-predictive encodings can be selectively engaged, disengaged, and inhibited. When comparing blocks with single and multiple tasks, parallel task processing costs as well as switching costs are assessed. These are needed to pre-activate the task-specific stimulus encodings and stimulus-response mappings as well as to switch between the distinct, task-specific encodings and mappings.

In contrast, mixed task blocks reveal the flexibility of the cognitive processes involved when switching between tasks (Kiesel et al., 2010; Koch et al., 2018; Frings et al., 2020). Task switching costs—that is, worse performance (typically measured in response times and response errors) when performing a different task in a subsequent trial vs. the same task—reveal that it is difficult to fully erase the task set that was relevant in the previous trial from memory. The robustness of this finding is stunning: it can be found in blocks with predictable task switches, cued switches, intermittent instructions, and even when task selection is mostly voluntary (Kiesel et al., 2010).

The theory of magnitude (ATOM) (Walsh, 2003) suggests that space, time, and quantity are encoded by a common magnitude system. From the event-predictive inference perspective, the common encoding of magnitudes makes a lot of sense: in the real world, magnitudes are closely related across modalities. For example, a louder crashing noise is created by a stronger force or by a larger object or by taking a longer fall onto the ground. Similarly, numerously more objects will send more visual signals, will be heavier in sum, will take longer to swipe away, and will be more effortful to create, compared to fewer objects of the same type. Thus, over development, it seems highly likely that predictive encodings form that cross-correlate various magnitudes with each other, setting the stage for the discovery of ATOM.

One closely related, particularly well-studied paradigm addresses the SNARC effect: spatial-numerical associations of response codes. In this case, numbers have been shown to be spatially-distributed on a magnitude axis, leading from left to right in societies with left-to-right scripts, presumably because the reading and writing direction guides from the past into the future, determining a latent temporal axis in our minds. Given a particular task set then, particular sets of numbers will be pre-activated in our minds, leading to the SNARC effect: even if the number magnitude does not matter for the response itself (e.g., deciding between even and odd), smaller/larger numbers are responded to faster with a left/right response (Wood et al., 2008). This fact, however, depends on the numbers that are currently mentally active in working memory as well as their spatial grounding. Accordingly, a working memory-based account of the SNARC effect has been proposed (Abrahamse et al., 2016), which highlights the importance of currently active mappings between space and numbers. For example, imagining numbers on a clock face grounds the numbers seven to 11 left of the numbers one to five, while this grounding is reversed when imagining the number on a ruler (Baechtold et al., 1998).

From the event-predictive inference perspective, these prior spatial-numerical activations correspond to current predictive encoding activities, which temporarily and latently associate numbers with space. As a result, and in close correlation with the Simon effect, investigations concerning the SNARC effect, and various related effects concerning number processing (van Dijck and Fias, 2011; Lohmann et al., 2018; Cipora et al., 2020), essentially investigate the strength of latent and temporary associations between space and numbers as well as the activation of particular spatial axes, dependent on the task at hand. Further, much more detailed, modeling efforts by means of REPI may shed further light on the concrete encodings involved and their processing dynamics in the respective ATOM-related paradigms and particular experimental setups.

As a final example for behavioral psychological experiments, results from studies on crossmodal congruency may be explained in a similar manner. Related to the SNARC effect and the theory of magnitude, in crossmodal congruency paradigms typically irrelevant distractor stimuli in a somewhat irrelevant modality systematically interfere with stimulus processing in the response-relevant modality.

One prominent approach was originally motivated by single cell recording results in monkeys. Individual neurons were shown to encode peripersonal spaces, responding to stimuli touching or appearing close by a particular part of the hand or face (Fogassi et al., 1996; Maravita et al., 2003). From an event-predictive inference perspective, again this comes as no surprise. As discussed above, cross-modal associations occur very frequently in our world. Thus, latent predictive encodings can be expected to develop that associate these respective stimuli cross-modally as well as across respective frames of reference. In fact, the rubber hand illusion (Botvinick and Cohen, 1998) as well as the disappearing hand trick (Newport and Gilpin, 2011) fall in the same category, in which case the brain attempts to infer a consistent explanation for the multisensory or sensorimotor signals the participants focus on, leading to an adaption of internal body state estimations (Ehrenfeld and Butz, 2013; Butz et al., 2014). During a typical crossmodal congruency task, on the other hand, irrelevant stimuli close to response-relevant tactile stimuli will interfere with according responses (Spence et al., 2004), indicating that visual stimuli are mapped onto tactile perceptions a priori, because the two modalities typically offer compatible signals in the real world.

Interestingly, the more recently investigated anticipatory cross-modal congruency effect (Brozzoli et al., 2010; Belardinelli et al., 2018; Lohmann et al., 2019) shows that peripersonal space around the hand is projected onto a planned grasp even before the actual action execution commences. These results essentially indicate that event-predictive inference anticipates action results, pre-activating particular goal constellations and according multisensory mappings. Interestingly, as would be expected from the event-predictive ActInf perspective, when uncertainty is high while approaching the object, the focus should lie on controlling the actual arm movement, thus temporarily not yielding a significant anticipatory crossmodal-congruency effect (Lohmann et al., 2019). A first variational implementation of the effect can be found elsewhere (Weigert et al., 2021).

Earlier studies on simpler action-effect compatibilites also fall into the same category: the effect of an action and the latent or temporarily activated compatibility with the action code itself, can lead to anticipatory cross-modal interactions (Elsner and Hommel, 2001; Kunde, 2003). On the dynamic event encoding level, studies on rotations have shown yet again related predictive encoding interactions: the processing of a rotating tactile stimulus on the palm, for example, interferes with the active rotation of a visual stimulus in a dual task paradigm (Lohmann et al., 2017). Earlier, visual perception was shown to be biased by the rotating tactile stimulus (Butz et al., 2010), indicating shared temporal dynamic encodings across modalities. We believe that REPI may indeed be used to model all of these effects—although open questions include the stimulus design choices, the actual involved predictive encodings, as well as the exact means to process the anticipated horizon and its effect on actual stimulus processing dynamics.

In our social realities, we experience scenes, and events within, often with a multitude of entities and other agents involved. These other agents have their own minds. Thus, to foster effective and adaptive interactions between these agents, it is beneficial to know something about the others' minds, that is, to develop a theory of mind (ToM) (Frith and Frith, 2005). When engaging in social interactions, then, ActInf may consider the ToM of others, including their perception of one's own mind and particularly one's own personality. Accordingly, building up scenes with a multitude of interaction-critical agents becomes effortful and can be strenuous. The more diverse agents (colleagues, friends, co-workers, team-members etc.) one is willing and able to consider, the more effort will be needed. Their parallel activation without between-agent interference (in one's mind), is per se strenuous because it is impossible to maintain too many distinct predictive densities simultaneously—particularly seeing that each set of events about another agent contains recursive agent events.

This recursiveness with respect to social interactions enables us to pursue rational social reasoning, as formulated in the rational speech act model (Frank and Goodman, 2012; Goodman and Frank, 2016). On the other hand, it also requires much more computational resources, as others need to be co-processed in one's own mind. Accordingly, a rather large corpus of studies suggests that our brains, particularly when acting under time pressure, have a very hard time to consider the perspective of others, failing to avoid, for example, undesired ambiguous situations during conversations (Ferreira and Dell, 2000; Ferreira, 2008). Due to the highly resource-demanding challenge to maintain the perspective of an interaction partner besides ones own perspective, we often sidestep this perspective taking problem by simply assuming that others have the same state of knowledge as we do. Nonetheless, when retrospectively interpreting ones own utterance and the observed actual behavioral response of the current conversation partner(s), we are able to inversely infer aspects of their motivations, intentions, and state of knowledge (Baker et al., 2017; Liu and Spelke, 2017; Achimova et al., 2022), enabling us to learn about others.

In sum, social interactions will require additional cognitive resources because effective and adaptive social interactions with particular others will need to activate our ToM about the other. Moreover, the more complex that social interaction and the more unusual the other(s); the more effortful the ActInf-based control of these interactions will be. It remains an open challenge to shed further light on the involved processes and resource demands by means of socially-extended REPI-like models.