Affectively salient stimuli have been found to impact cognitive processing in a variety of ways. For instance, a survival-relevant stimulus, such as a fearful face, can exert a powerful exogenous pull on attention (LeDoux, 2000; Ohman et al., 2001), even if it is irrelevant to the task at hand (Mathews and MacLeod, 1985; McKenna, 1986; Isenberg et al., 1999; Phelps et al., 2006; Reeck and Egner, 2011). In addition to attentional effects, a rich animal and human research literature has documented that affectively salient stimuli or situations also modulate episodic memory processes. Specifically, it is well-established that emotional arousal at the time of encoding enhances long-term memory consolidation in comparison to emotionally neutral conditions (McGaugh and Roozendaal, 2002; McGaugh, 2004; for reviews, see Hamann, 2001; LaBar and Cabeza, 2006). Other studies have shown that emotional stimuli also have more immediate effects on memory encoding, which are likely attention-mediated and may enhance episodic memory in the short-term (that is, prior to consolidation into long-term memory) as well as in the long term (Hamann et al., 1999; Hamann, 2001; Tabert et al., 2001).

In the latter work, an important distinction has been drawn between the recall of a specific object that carries emotional salience (item memory) and the binding of that item and its context (source memory). Specifically, a number of studies suggest that emotional arousal facilitates the recall of emotional stimuli themselves but either does not enhance (Mather and Nesmith, 2008) or is detrimental to the recall of contextual features that are not part of the salient object, resulting in poor context or source memory (Mather et al., 2006; Mitchell et al., 2006). These effects have been theorized to reflect the fact that emotionally arousing stimuli attract attention at the expense of other, co-occurring stimuli, which results in a boost to emotional item memory but poorer mnemonic integration of incidental, non-emotional stimuli into episodic memory (Mather, 2007; but see Hadley and Mackay, 2006). In the current study, we sought to further elucidate the interplay between (negative) emotional stimuli and mnemonic feature binding processes, but with a focus on immediate (or very fast) binding effects, which occur at a time-scale that has been largely ignored in the previous literature.

The temporal lag between memory encoding and retrieval in studies that assessed emotion effects on short-term memory is typically in the range of several minutes (LaBar and Phelps, 1998; Hamann et al., 1999; Tabert et al., 2001; Sharot and Phelps, 2004), and at the least in the range of 7–8 s (Mather et al., 2006; Mitchell et al., 2006). By contrast, in the present study, we sought to evaluate the influence of emotional stimuli on very fast-acting processes in episodic memory associated with “feature integration,” the seemingly obligatory moment-by-moment binding of stimulus and response features into compound episodic memory traces referred to as “event files” (Hommel, 1998). These binding processes appear to take effect almost instantaneously, as their consequences can be observed at a time-scale of less than 1 s (Hommel, 1998) and they last at least several seconds (Hommel et al., 2004). Note that this type of feature integration involves not only the binding of different perceptual features, which has typically been the focus of previous emotional memory studies (see Mather, 2007), but also the integration of these stimulus features with a motor action on the part of the subject (Hommel, 1998).

Specifically, building on the notion of “object files” – the momentary binding of object features at attended locations into a cohesive percept and memory representation (Kahneman et al., 1992) – Hommel has shown convincingly that both task-relevant (and sometimes task-irrelevant) stimulus and response features comprising an “event” appear to be integrated into a common short-term memory trace, such that processing costs are incurred when this integrated memory has to be bypassed or “unbound” subsequently (e.g., Hommel, 1998, 2004; Hommel et al., 2004). For instance, if a participant performs a right-hand button-press response that temporally coincides with the presentation of a particular visual stimulus (e.g., a blue square), the stimulus and response features are thought to be bound together into an episodic event file. If, shortly afterward, the subject has to perform a left-hand button-press in the presence of the same stimulus features (a “partial repetition” of the previous event), the response will be slowed in comparison to conditions in which either all response/stimulus features repeat or they all change. This is argued to reflect the fact that the reoccurring stimulus features will reactivate (or retrieve) the strongest or most recent memory trace involving those features, including their associated response, which now conflicts with the required response; this conflict has to be overcome for the correct response to be selected, resulting in slower response times (RT; for a similar scheme, see Logan, 1988). These partial repetition costs incurred on partial repetition trials as compared to complete repetitions or alternations can therefore serve as a behavioral index of feature integration in short-term episodic memory, and they will be employed as the dependent variable for testing the effects of emotional stimuli on fast short-term memory binding processes in the present study.

The major motivation for assessing the impact of (negative) emotion on these feature integration processes in the current study stems from the fact that this type of feature binding represents a well-known confound in many studies investigating trial–transition effects in traditional conflict tasks (like Stroop and Simon tasks; Hommel et al., 2004; for review see Egner, 2007). Here, conflict (or interference) effects are typically reduced following an incongruent, high-conflict trial compared to a congruent, low-conflict trial, because performance on successive incongruent and successive congruent trials tends to be faster than performance on incongruent trials following a congruent trial and on congruent trials following an incongruent trial, respectively (Gratton et al., 1992). These inter-trial dependencies in behavioral performance are typically attributed to the operation of a conflict-driven cognitive control mechanism that enhances the selection of task-relevant stimulus information following an incongruent trial, thus reducing the performance difference between congruent and incongruent trials under these conditions (Botvinick et al., 2001; Egner and Hirsch, 2005). However, this sequence effect can often equally well be interpreted in terms of feature integration processes, because in conflict tasks with small stimulus sets, successive congruent and incongruent trials tend to be associated with either an exact repetition of all stimulus and response features or a complete change of these features, whereas congruent trials following an incongruent trial and incongruent trials following a congruent trial tend to be partial repetition trials (Hommel et al., 2004; Egner, 2007).

Importantly, recent studies have suggested that emotional states modulate these trial–transition effects (van Steenbergen et al., 2009, 2010; but see Stürmer et al., 2011), whereby negative emotion is held to potentiate inter-trial dependencies, and these effects have been interpreted as reflecting the modulation of conflict-driven cognitive control by emotion (van Steenbergen et al., 2009, 2010). However, the designs of these studies did not allow for segregating the effects of conflict-driven control from those of feature integration, such that it is presently not clear whether the emotion effects observed were carried by modulation of conflict-driven control or by modulation of feature integration processes. Here, we sought to clarify this issue by assessing the impact of negative emotion-evoking stimuli on a pure measure of feature integration, outside of the context of a conflict task: if negative emotion were to potentiate feature binding, the previously reported effects might be attributable to this type of modulation, whereas if there were no evidence for emotion-modulation of feature integration, the potentiating effects of negative emotion on trial–transition effects in previous studies are likely attributable to the modulation of conflict-driven control processes.

Given this particular perspective on emotion–memory interactions, our design entailed an additional distinction from many previous studies of emotional memory, in that we did not assess memory for the emotional stimulus itself, but rather whether the mere presence of a (task-irrelevant) background stimulus thought to evoke emotional arousal would modulate episodic binding processes concerning task-relevant foreground stimuli and responses. This approach mimics more closely the type of designs used in investigating emotion-modulation of cognitive control processes (Dreisbach and Goschke, 2004; van Steenbergen et al., 2009, 2010; Stürmer et al., 2011). From the item vs. context perspective in the emotional memory literature highlighted above, this design could be cast as assessing whether emotional arousal modulates fast binding processes of contextual stimuli that are task-relevant but not part of the emotional stimulus itself. In sum, our goal was to gauge whether threat-related emotional (as compared to neutral) background stimuli would influence the short-term episodic binding phenomena underlying the moment-by-moment integration of stimulus and response features.

We approached the question of whether emotional stimuli modulate feature integration by adopting the basic task design developed by Hommel (1998). Specifically, in each trial, a multi-featured stimulus (a colored shape) was first paired with a pre-cued response in an arbitrary manner (i.e., the response was not determined by the stimulus features), followed by a second stimulus presentation where the task-relevant stimulus feature (color) determined which one of two possible responses to perform. This design allows one to independently vary repetitions vs. changes in the task-relevant stimulus feature (color), the irrelevant stimulus feature (shape), and the response (left vs. right) from the first to the second stimulus presentation. Consequently, the degree of integration between the response and the relevant and irrelevant stimulus features can be gauged via the respective partial repetition costs (i.e., the relative increase in RT in partial repetition trials as compared to complete repetitions and alternations). In Experiment 1, we simply aimed at replicating the basic stimulus–response feature integration effects reported by Hommel (1998) in the absence of any additional stimuli. In the second and third experiments, we then tested whether presenting additional task-irrelevant background stimuli of varying emotional valence (face stimuli with neutral vs. fearful expressions) either at encoding (Experiment 2) or at encoding and retrieval (Experiment 3) would differentially modulate the episodic binding between the foreground stimulus features and the response. For a different approach to investigating emotional modulation of feature integration effects, see Colzato et al. (2007), which will be addressed in detail in the Section “General Discussion.”

Our first experiment simply served to ascertain that we could replicate the standard feature integration effects reported by Hommel (1998) in a simple task where the task-relevant stimulus feature consisted of stimulus color and the task-irrelevant feature consisted of stimulus shape. We anticipated observing reliable binding effects between stimulus color and response and weaker (if any) binding effects between stimulus shape and response.

Our basic research question concerned the impact of emotion on the short-term episodic memory phenomenon of feature integration. In Experiment 1, we established the basic procedure for obtaining reliable feature integration effects. In Experiment 2, we aimed at adding a manipulation of emotion to that procedure. As emotion-provoking stimuli, we chose photographs of male faces with fearful expressions, to be contrasted with the same individuals’ faces posing neutral expressions. A large literature has established that fearful face stimuli are potent emotional stimuli that evoke heightened arousal (as measured by galvanic skin response; e.g., Williams et al., 2001), trigger strong responses in neural fear circuits centering on the amygdala (e.g. Breiter et al., 1996), and can have immediate effects on perceptual and cognitive processing (e.g., Phelps et al., 2006; Reeck and Egner, 2011). In order to assess the effects of threat-related, negative emotion on feature integration, in Experiment 2 we therefore simply added a task-irrelevant background face stimulus (neutral vs. fearful) during the encoding phase (that is, at S1) of the feature integration task we had employed in Experiment 1 (see Figure 1B).

One aspect of the design of Experiment 2 that may have impeded the expression of emotion-modulation of feature integration is that the background face stimuli were shown at encoding (S1) but were not displayed at S2. While this approach makes intuitive sense when one is interested in how emotion impacts encoding processes, the fact that there was a major feature change in the stimulus display between S1 and S2 could have feasibly confounded the results. Specifically, it is likely that the face stimuli themselves will form part of the event file that is encoded at S1, such that the non-appearance of faces at S2 would essentially render every trial a partial repetition trial, in that even if all other S1 and R1 features repeat, one salient aspect of the stimulus display does not repeat. In order to avoid any possible confounds stemming from the non-repetition of the background face stimuli, in Experiment 3 we simply added the background faces both to S1 and S2, such that on each trial, the exact same face (identity and expression) was shown both at encoding and during S2/R2 (see Figure 1C).

We adopted Hommel’s (1998) experimental set-up for gauging rapid episodic memory integration effects of stimulus and response features (Experiment 1) and tested whether such feature binding was susceptible to modulation by emotional arousal, as evoked by background fearful (vs. neutral) face stimuli (Experiments 2 and 3). The results were clear-cut, in that we reliably reproduced the behavioral signature of the basic feature integration effect (partial repetition costs) (see Table 2) but were unable to detect any evidence for emotional modulation of this effect, regardless of whether threat-related background stimuli were displayed at encoding (S1) or at both encoding and retrieval (S1 and S2). In fact, a direct comparison of feature integration effects across Experiments 1–3 (with experiment serving as a between-subjects factor) reveals that the two-way interaction between the relevant stimulus feature (color) and response transitions (that is, the standard feature integration effect) did not differ across the experiments [three-way interaction of color × response × experiment, F(2, 62) = 0.5, P = 0.61]. Moreover, a between-experiment analysis involving only Experiments 2 and 3, which included two subtly different emotion manipulations, revealed no effect of experiment on the relationship between emotion and color/response transitions [four-way interaction of emotion × color × response × experiment, F(1, 43) = 1.7, P = 0.20]. Thus, feature integration between the relevant stimulus feature and response was entirely unaffected by the emotion manipulations, both within and across experiments.

In the context of these null-effects, it is crucial to note, however, that the emotion manipulation per se was nevertheless found to be effective, as the expression of the background face stimulus did significantly affect R2 performance in both Experiments 2 and 3. In Experiment 2, fearful facial expressions in S1 sped up R2 RT compared to neutral expressions, and in Experiment 3, the emotional face expression interacted with the response transition factor, due to slower R2 RT for response repetitions than response changes in the presence of neutral faces, but similar RT for response changes and repetitions in the presence of fearful faces. These data allow us to rule out a number of alternative reasons for not obtaining emotion-modulation effects on feature integration. First, the fact that R2 RT in both Experiments 2 and 3 were significantly modulated by the emotional expression of the task-irrelevant face stimuli clearly shows that these stimuli effectively influenced information processing, thus ruling out the possibility that our manipulation of emotion was too weak to affect subjects’ behavior. Second, these significant effects demonstrate that our design was sensitive enough to detect the influences of facial expressions on behavior. Finally, these findings also rule out the possibility that subjects were able to somehow filter out the task-irrelevant emotional face stimuli. Given these data, it can be concluded that our study should have been able to detect negative emotion-modulation of short-term feature binding processes if such effects existed, unless those bindings were, for some unknown reason, subject to a substantially higher threshold for emotion-modulation than response selection and transition effects.

As noted above, the emotion manipulations in Experiment 2 and 3 had qualitatively different effects on behavior. Recall that in Experiment 2, face stimuli were shown only at encoding (S1) whereas in Experiment 3 they were shown both during encoding and retrieval. This change in procedure could evidently be responsible for the differential effects. Alternatively, these distinct effects could in theory have stemmed from a difference in how the face stimuli were processed during S1/R1. Accordingly, we analyzed mean R1 RT as a function of facial affect and experiment. However, no differences in the response to neutral vs. emotional face stimuli was observed across experiments [F(1, 43) = 0.34, P = 0.56]. Thus, it appears that the procedural differences between the two experiments were responsible for the distinct effects of emotion. However, we remain ignorant (and agnostic) about the exact reasons for this differential impact of emotion as well as the specifics of its interaction with the response transition factor in Experiment 3.

Our main motivation for assessing the effect of emotion on rapid feature integration processes stemmed from recent reports of emotion-modulation concerning trial-by-trial effects of conflict-driven control in experiments that did not tease apart pure conflict–control effects from possible feature integration effects (van Steenbergen et al., 2009, 2010). Specifically, these studies documented that negative emotional states potentiate trial–transition effects in conflict tasks, and the present experiments were geared at testing whether these findings may have been mediated by potentiation of feature integration rather than cognitive control processes. We observed no evidence of emotion modulating feature integration processes. Vis-à-vis these and other previous studies of emotion manipulations on trial–transition effects (e.g., Dreisbach and Goschke, 2004; Stürmer et al., 2011), the current results therefore suggest that it is unlikely such effects are carried by emotion-modulation of rapid feature integration, and are thus probably reflective of emotion-modulation of cognitive control processes. A feasible avenue for the rapid modulation of cognitive control processes by positive or negative emotional stimuli is provided by the well-known interactions between the midbrain dopamine system and the prefrontal cortex (e.g., Braver and Cohen, 2000). An important caveat concerning the current conclusions, however, is that we only assessed the effects of a negative, threat-related emotion manipulation (in comparison to neutral stimuli), from which one cannot necessarily extrapolate to effects of positive emotional stimuli or rewards. Thus, it remains a possibility that feature integration effects of the type we investigated here are susceptible to modulation by positive emotional states.

Some suggestive evidence for this possibility has been supplied by a study on feature integration and emotion by Colzato et al. (2007). In a similar protocol to the current study, these authors presented either negative or positive task-irrelevant emotional picture stimuli not during encoding of S1/R1 but just prior to the retrieval of these S–R ensembles. Specifically, emotional stimuli were displayed 1,000 ms after S1 offset, starting from 200 ms before and lasting until the presentation of S2 (and execution of R2). Thus, this study likely gauged the effects of emotion on the retrieval of recent S–R bindings, in contrast to the present study’s focus on the encoding of S–R bindings. Colzato et al. (2007) found some evidence for feature integration effects following negative stimuli to be less pronounced than following positive stimuli, though this effect was only observed when the task-relevant S2 feature was stimulus shape but not when it was stimulus location or color. The authors interpreted the latter effect by suggesting that the processing of positive/negative picture stimuli would require the processing of shape but not of color or location information. Given that the Colzato et al. (2007) study did not include an emotionally neutral baseline comparison, it is impossible to tell to which degree this modulation was carried by the positive or negative stimuli. But, considering the present findings of a lack of differential feature integration effects when comparing negative and neutral stimuli, it could be speculated that the Colzato et al. (2007) results were likely driven by the positive emotional stimuli modulating the retrieval of recently formed S–R bindings.

However, given the considerable differences in study design between the present experiments and those of Colzato et al. (2007), any conclusions drawn from contrasting their respective results must be regarded as very tentative at best. In addition to the use of positive and negative stimuli in Colzato et al. (2007) as opposed to negative and neutral stimuli in the present study, differences in results could also stem from the fact that the two studies targeted different time points in the binding process (encoding vs. retrieval) and/or due to the use of different task-relevant stimulus features. Specifically, unlike Colzato et al. (2007), the present study only employed color as the task-relevant stimulus feature, such that it is in principle not certain whether our findings can be generalized to contexts where other stimulus features are task-relevant. In any case, however, even though there is thus some evidence for (positive) emotion-modulation of feature integration (Colzato et al., 2007), the direction of this effect (positive emotional stimuli enhancing feature integration) is precisely opposite to the effects of emotion manipulations in cognitive control tasks, where it is negative emotional states that appear to potentiate trial-by-trial performance dependencies and positive states that appear to loosen these dependencies (Dreisbach and Goschke, 2004; van Steenbergen et al., 2009, 2010). Thus, our conclusion that the previously observed effects of emotion on trial–transition metrics of cognitive control are unlikely to have been mediated by emotion-modulated feature integration effects is actually supported by this prior study (Colzato et al., 2007).

From the perspective of the emotional memory literature, the current data could be argued to provide support for the “object-based” framework of emotional arousal put forward by Mather (2007), which holds that emotional arousal benefits the recall of an emotional item itself but not that of other stimuli providing the context to that item. The present results demonstrate the latter, namely, that the presence of emotional stimuli does not confer any memory advantage (or disadvantage) on other non-emotional stimuli presented concurrently (see also Mather and Nesmith, 2008; Mather et al., 2009). Moreover, the current data extend this notion to a shorter time-scale than has been investigated in previous studies, and to the binding of concurrent stimulus and responses features rather than of stimulus features alone. While the present results are thus in principle in line with an object-based binding framework of emotional arousal, additional experiments would be required to determine whether memory for the emotional stimuli themselves would actually be enhanced in the type of protocol we employed here, as would be predicted by this theory (Mather, 2007).

In sum, the rapid integration of task-relevant stimulus and response features into episodic “event files” is not modulated by concurrent presentation of threat-related negative emotional stimuli at encoding and/or retrieval. This finding is concordant with the proposal that emotional arousal facilitates emotional item memory but does not extend this mnemonic benefit to other concurrent stimulus or response features. Moreover, this lack of emotion-modulation of feature integration effects has important implications for the interpretation of emotion-modulated trial–transition effects in studies of cognitive control, in that the latter effects are unlikely to be mediated by the influence of emotion on feature integration effects that are commonly confounded with the effects of cognitive control.

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.