Cue-outcome learning is a cornerstone of intelligent action. Learning that a stimulus (the cue) can be used to predict a second event (the outcome) affords adaptive decision making. If an animal can use environmental cues to predict the presence of food (an appetitive outcome) or a predator (an aversive outcome), then it can potentially act to maximize the likelihood of the former and minimize the latter. This capacity also allows for the development of complex associative webs of knowledge. For example, it allows humans to associate a visual icon with an auditory phoneme (and thus read), or to connect new information with existing knowledge. These are fundamental capacities of intelligent agents.

The capacity to learn cue-outcome mental associations is crucial. However, forming mental associations may be cognitively costly, so it is important to be selective about which associations are learned, and thus to prioritize which stimuli gain access to the learning process. One way to achieve this efficiency is to leverage prior knowledge to focus primarily upon those events (cues and outcomes) that are most likely to be meaningfully associated (e.g., Mackintosh, 1975; Le Pelley, 2004, 2010; Mitchell and Le Pelley, 2010; Esber and Haselgrove, 2011).

There are three logically distinguishable components of a person's prior associative knowledge that could be used to guide the selectivity of subsequent cue-outcome learning. First, there is the knowledge about specific cue-outcome associations that have already been learned (represented as associative strength, or V). Second, there is knowledge about the cueing stimuli themselves (represented as cue associability, or α). Finally, people could use prior knowledge about the outcome stimuli. The influence of the first two forms of associative knowledge in guiding subsequent learning has been extensively investigated. However, the influence of the third form (i.e., information about outcome stimuli) has been largely overlooked in the learning literature. We argue herein that this oversight of the potential role of outcome predictability in shaping learning offers fertile new ground for the science of learning. Before making this case, we first briefly note how associative strength and cue associability have been shown to guide learning.

There is a vast and detailed literature showing that the associative strength of cue-outcome relationships shapes the manner in which subsequent cue-outcome learning takes place. Perhaps the clearest example of this is the observation that learning curves are negatively accelerated (e.g., Rescorla and Wagner, 1972). That is, when little is known about a cue's relationship with an outcome, the person's knowledge about this association (V) increases rapidly. However, as more is known about that specific cue-outcome association, the rate of new learning about that association decreases, and eventually an asymptotic value for the associative strength is reached. In addition, learning about other associations can be impaired too, as demonstrated in the blocking effect (Kamin, 1969), perhaps the most widely studied learning phenomenon. An initial cue (A) is paired with an outcome, but is withheld for a control group. Then for both groups, a new cue (B) is shown alongside the pre-trained cue (A) and they are both followed by the outcome. Those that had already learned an association between A and the outcome fail to associate (the redundant) cue B with the outcome, relative to the control group that did not receive pre-training. The activation of the already existing cue-outcome association by its cue A blocks learning about cue B's new association with the same outcome (see superconditioning for the opposite effect; Rescorla, 1971; Wagner, 1971).

The second source of knowledge that is well-known to guide the selectivity of subsequent cue-outcome learning is the associability of the cue. Several authors have argued that animals learn which cues are valid predictors of outcomes, and this knowledge shapes how readily those cues become associated with outcomes in the future (see LePelley et al., 2016 for a review). Specifically, Mackintosh (1975) suggested that cues that are good predictors of outcomes are more associable; they more readily enter into associations with outcomes in future. Empirical evidence generally supports this claim (but see Pearce and Hall, 1980; Griffiths et al., 2011). People (and other animals) readily learn which cues are valid predictors, and this cue-specific learned information guides subsequent selectivity in cue-outcome learning (Le Pelley, 2004). Perhaps the clearest example of this effect is the “Learned Predictiveness” effect (Le Pelley and McLaren, 2003), whereby cues shown to be previously predictive of important events are more rapidly learned about subsequently.

The third possibility is that acquired, specific knowledge about outcome stimuli may bias subsequent cue-outcome learning. While it is well-known that physical properties of the outcome influence the speed of learning (Annau and Kamin, 1961), we suggest that people may learn about an abstract property of outcomes – their “predictability”– and that this modulates the formation of subsequent cue-outcome associations involving those same outcome events. Notably, traditional formal models are silent as to the possibility of such a “Outcome Predictability” effect, whereas the effects of prior cue-outcome learning and learned cue associability (see above) are both well-predicted by these models.

However, there is now growing empirical support for the hypothesis that people encode and use information about outcome predictability to guide learning. The first empirical indications that the experienced (un)predictability of an outcome might impact upon subsequent learning comes from the learned irrelevance and learned helplessness paradigms (e.g., Mackintosh, 1973; Baker, 1976; Overmier and Wielkiewicz, 1983). However, many of these findings were adequately explained by a variation of the blocking effect, termed “context blocking” (Baker et al., 1981). Even if no distinct cue with an established association (such as cue A in the earlier example) is present during subsequent learning, the diffuse and ever present contextual cues will activate their cue-outcome associations and may block new cue-outcome learning. Therefore, both effects may also be explained as the product of prior cue-outcome associations, and need not demonstrate an effect of prior learning about the unpredictability of the outcome itself.

More recently, Griffiths et al. (2015) demonstrated that people learn novel cue-outcome associations more rapidly if those associations involved outcomes that had previously been shown to be predictable, as compared to otherwise equivalent, novel cue-outcome associations involving outcomes that were previously shown to be unpredictable. Moreover, this effect is not readily attributed to context blocking (but see Liu et al., under review). In their procedure, people were tasked with learning which foods a fictional patient, Mr. X, was allergic to. They were shown the meals Mr. X ate on different days, and whether or not he had a reaction to that meal. In the first stage, Mr. X ate only vegetables, and sometimes experienced stomach (nausea, cramping) or skin (itching, swelling) reactions. Stomach reactions were predictable on the basis of vegetables ingested, but skin reactions were not. Then, in a second stage, Mr. X ate only fruits, and both stomach and skin reactions were predictable on the basis of the fruits ingested. Despite both types of allergic reaction (outcome events) being predictable in the second phase, people learned more rapidly about the associations between fruits and the previously predictable (stomach) reactions than between fruits and the previously unpredictable (skin) reactions. This bias toward learning associations involving previously predictable outcomes has since been replicated by two independent research groups using the same allergy task (Thorwart et al., in preparation), in a serial reaction time task (Quigley et al., under review) and in a human goal-tracking task (Liu et al., under review).

Rescorla and Wagner's (1972) model (RW) is often viewed as the quintessential associative model of predictive learning. Using four psychologically plausible variables, it describes how learned associations between cues and outcomes change with experience. Two of these variables, associative strength (V) and cue associability (α) are well-known to affect cue-outcome learning. An interesting theoretical question posed by the Outcome Predictability effect is whether this effect is explicable in terms of the two variables related to the outcome stimuli, the outcome associability β or the outcome efficacy λ. Although both variables are typically interpreted as fixed properties of the outcome (depending on its physical salience or intensity, respectively), it is possible to simulate what would happen if they were allowed to vary as a product of experienced unpredictability. In Figure 1, we describe simulations of models in which previously unpredictable events either lose associability (β) or efficacy (λ) as each of the parameters is made dependent on the prediction error (see also below and Figure 1 for details). An Outcome Predictability effect is evident by the more rapid ascent of the solid red line than the broken red line in both simulations but not the original RW model. More importantly, a more thorough consideration of how these parameters may be influenced by unpredictability, and then subsequently influence learning, could elicit important, novel hypotheses for the field.

We have argued that the properties of the outcome are an important aspect of associative knowledge that guides selectivity in learning, akin to the effects of cue-associability and cue-outcome associations. We have identified (and simulated) two mechanisms whereby such effects could occur within the gold-standard RW model of associative learning, and one without (declarative reasoning). Whichever of these mechanisms best account for the effect, the existence of the effect itself offers potential insights into other learning effects. One example is the partial-reinforcement extinction effect (e.g., Haselgrove et al., 2004), whereby people are slow to learn about the absence of an outcome that was previously shown in a partial reinforcement arrangement (i.e., was unpredictable). This mainstay of the animal conditioning literature is consistent with, and indeed may be an instance of, the Outcome Predictability effect. A second possibility is that outcome predictability may modulate outcome-mediated biases in action selection and attention (e.g., Gozli et al., 2014; Gozli and Ansorge, 2016), whereby the presence of stimuli that resemble the sensory consequences of an action (i.e. the “outcome”) affects the speed with which that action is subsequently performed. One might expect the control exerted by these outcome stimuli to be dependent upon their prior predictability. This hypothesis remains to be tested.

OG and AT were equally responsible for the conception, drafting and revising of the paper.

AT contribution was supported by Project TH 1923/1-1 awarded by the German Research Foundation (DFG). OG contribution was supported by an Australian Research Council (ARC) Discovery Early Career Research Award (DE150100667).