Albeit there are many theories related to emotions and their comprehension, the present study focuses on the idea that facial expressions are of major importance in understanding the mental and emotional states of others (e.g., Tomkins, 1962–1963; Ekman, 1992, 1993; Izard, 1994; Russell, 1997). In this respect, two main approaches have been developed so far: the categorical and the dimensional approach (Aviezer et al., 2008). The categorical approach, in accordance with the facial dominance perspective (Tomkins, 1962–1963; Carroll and Russell, 1996), holds that basic facial expressions index specific emotions, reducible into clearly different categories (basic or primary emotions vs. complex or secondary emotions) (e.g., Ekman and Friesen, 1971; Ekman, 1992, 1993).

Conversely, the dimensional approach holds that emotions are defined by the different neurophysiological mechanisms of valence and arousal, the first mechanism referring to the degree of pleasantness/unpleasantness of an emotion, the second referring to the intensity of an emotion (which can vary from calm to excited). Thus, a certain emotion would be the result of a distinct combination of values of these two dimensions (e.g., Aviezer et al., 2008; Hajcak et al., 2011). In this vein, facial expressions convey information related to both the degree of pleasantness/unpleasantness (valence) and of physiological activation (arousal) (e.g., Russell, 1980, 1997).

As stressed by Aviezer et al. (2008), both approaches share the idea that “affective information… is read out from the face by a process that is relatively immune to context” (Aviezer et al., 2008, p. 724). This is the reason why most studies on the perception and comprehension of emotions so far have used isolated emotional facial expressions as stimuli. However, there is evidence that our real experience during social interactions in fact is not as independent from other influences (e.g., de Gelder et al., 2006). Already 20 years ago, for instance, Carroll and Russell (1996, p. 207) demonstrated that the same facial expression could convey different meanings depending on the context in which it was located: the perceiver can infer the emotion expressed by a facial expression referring to pleasantness, arousal and “quasi-physical” information, all of which are interpreted in light of available information about the expresser’s situation.

In the same vein, Wieser and Brosch (2012) highlighted how faces and facial expressions are always perceived in a wider context involving not only within-face features (e.g., eye gaze; Boll et al., 2011), but also within-sender features (e.g., body postures; Meeren et al., 2005; Aviezer et al., 2008), external features (e.g., emotional labels, verbal descriptions or visual scenes; Kim et al., 2004; Mobbs et al., 2006; Barrett et al., 2007; Schwarz et al., 2013), and within-perceiver features (e.g., personality traits; Calder et al., 2011). This is also in line with the behavioral ecology view of facial expressions (e.g., Fridlund, 1994), which stresses the relevance of situational context and communication (Barratt et al., 2016).

Already in the early twentieth century, the Soviet filmmaker Kuleshov (1899–1970) argued that such situational context could significantly change our interpretation of facial expressions. He designed an experiment in which he edited two close-ups of the Russian actor Ivan Mozzhukhin’s neutral face with three different emotional contexts: happy (a little girl playing with a doll), sad (a dead woman in a coffin), and hungry (a bowl of soup) (e.g., Pudovkin, 1970; Barratt et al., 2016). The viewers of the three film sequences reportedly perceived the actor’s neutral face as expressing an emotion congruent with the preceding context (Kuleshov, 1974; Barratt et al., 2016). The story has been passed on as a demonstration about contextual priming in movies, also known as the Kuleshov effect (Carroll, 1993).

Barratt et al. (2016) recently described in detail the characteristics of a Kuleshov-type sequence, arguing that it can be understood as a crossover between Soviet montage editing and classical continuity editing. In terms of the latter, a Kuleshov-type sequence can be regarded more precisely as an instance of point-of-view (POV) editing. A typical POV structure shows a first shot of a character looking off-screen in the direction of an object/event (glance-shot), followed by a second shot of the object/event in question (object-shot) (Branigan, 1984; Carroll, 1993; Persson, 2003; Barratt et al., 2016). When the object is presented from the perspective of the character, we have a “true POV” (Brewster, 1982; Persson, 2003; Barratt et al., 2016). The glance-shot can be either shown before or after the object, in what has been, respectively, called “prospective” and “retrospective” POV structure (Branigan, 1984; Carroll, 1993; Barratt et al., 2016).

To our knowledge, there have been only three previous attempts at replicating the original Kuleshov experiment. Prince and Hensley (1992) showed an actor’s neutral face, a static emotional image and the actor’s neutral face again, and then asked participants to evaluate the actor’s emotional performance, selecting from a list of emotions on a check-sheet: happiness, sadness, anger, fear, surprise, disgust, hunger, “no emotion,” and “other” (see the categorical approach; e.g., Ekman and Friesen, 1971). Results did not demonstrate a Kuleshov effect as most of participants chose the “no emotion” option, and those who reported perceiving emotions, chose an option unexpected with respect to the particular context.

More recently, Mobbs et al. (2006) revised the Kuleshov effect paradigm to investigate, by means of functional magnetic resonance imaging (fMRI), the neural correlates of contextual modulations on facial expression and mental-state attributions. Participants were asked to rate the emotional expression and mental state of a still image of a face, crosscut with an emotional image, using a two-dimensional rating scale (see the dimensional approach; e.g., Russell, 1980). Behavioral and fMRI results substantiated the Kuleshov effect with higher ratings of valence and arousal for faces paired with positive and negative contexts than for those paired with neutral contexts, and enhanced BOLD responses in several brain regions including the amygdala.

However, as stressed by Barratt et al. (2016), both studies showed limitations regarding the experimental design, that make comparisons difficult: while the negative results of Prince and Hensley (1992) could be attributed to problems of statistical power (single-trial experiment), the second study diverged from the traditional Kuleshov paradigm in various details.

Bearing in mind these limitations, Barratt et al. (2016) recently replicated the Kuleshov experiment with an improved experimental design attempting to respect as many rules as possible in order to increase the possibility that participants would infer that the glance shot and the object shot were spatially related (Persson, 2003; Barratt et al., 2016, p. 7). Thirty-six participants were presented with 24 film sequences of neutral faces (rendered dynamic with the zoom-in effect) paired with contexts belonging to six different emotional conditions (happiness, sadness, hunger, fear, desire, and “null condition”). As the contexts could be either static or dynamic objects, the authors used either a photograph with a slow zoom-in effect or a video clip. In order to combine the categorical and dimensional approach to emotion, participants were asked to rate both the valence and arousal of the target person’s emotion, and to explicitly categorize the type of emotion by choosing among different options. During the experiment, eye movements were recorded. Results showed significant behavioral effects pointing in the expected direction (from both a categorical and dimensional point of view). Specifically, neutral faces paired with sad contexts were rated as the most negative and least aroused, while neutral faces paired with desire contexts were perceived as the most positive and the most aroused (Barratt et al., 2016; pp. 15–16).

With the present study, we aimed at investigating and exploring further Barratt et al.’s (2016) results with some variations with respect to the original paradigm making up for an even more ecological design (for details, please see sections “Materials and Methods and Discussion”). Furthermore, we aimed at verifying the persistence of the effect despite these variations in order to employ the same experimental paradigm in a future electroencephalographic study to explore the contextual modulations on emotion processing at both the physiological and cortical levels. Participants were shown 18 film sequences of neutral faces crosscut with scenes evoking two different emotions (happiness, and fear, plus neutral stimuli as a baseline condition). Hence, from a dimensional point of view (e.g., Russell, 1980), we chose emotions characterized by distinct combination of values of these two dimensions: happiness (positive valence and medium arousal), fear (negative valence and high arousal) and neutral (neutral valence and low arousal) (see also Lang and Bradley, 2007). We employed only two emotional contexts (happy and fearful) in order to keep the design as simple as possible, and to highlight the differences between opposite emotional conditions in terms of valence. In particular, we adopted fear as a negative emotion because, from an evolutionary point of view, it is capable of directing our attention to potentially dangerous stimuli activating one of the two major motivation circuits (defensive vs. appetitive motivational systems; e.g., Bradley et al., 2001a,b; Lang and Bradley, 2010). Since we focused on both a dimensional and categorical approach to emotion (e.g., Ekman and Friesen, 1971; e.g., Russell, 1980) in order to have as much information as possible about participants’ experience (see also Barratt et al., 2016), we adopted happiness as a positive emotion. In comparison to desire (which is capable of activating the appetitive motivational system; Bradley et al., 2001a,b; Sabatinelli et al., 2001), happiness is more clearly reducible to one of the basic emotional categories (e.g., Ekman and Friesen, 1971). Thus, the task was to rate the emotion displayed by a target person’s face in terms of valence, arousal, and category. As contextual stimuli, we employed dynamic scenes in order to study the context-sensitivity of emotions under more ecologically valid conditions. We expected to find a significant difference between the ratings of valence, arousal, and category attributed to neutral faces paired with emotional contexts (both fearful and happy) and those attributed to neutral faces in neutral contexts. More specifically, we expected neutral faces in fearful contexts to be rated with more negative valence and higher arousal scores than neutral faces in neutral contexts, and neutral faces in happy contexts to be rated with more positive valence and higher arousal scores than neutral faces in neutral contexts.

In accordance with the previous study by Barratt et al. (2016), we rescaled the valence and arousal scores for each participant so that a value of zero corresponded to the mean rating across all three conditions, respectively. This was done in order to evaluate whether, for each participant, a condition mean was higher (positive value) or lower (negative value) than the overall mean in terms of valence and arousal.

In order to investigate the modulation of rating by context condition, we performed a linear mixed effects analysis. We entered the rating score as a dependent variable, and Measure (2 levels: Arousal and Valence) and Context (3 levels: Neutral, Fearful, and Happy) as independent fixed variables. We entered intercepts for stimuli and subjects, and by-subject slopes for the effect of Context as random effects.

Tukey’s test was used for post hoc comparisons among means. Visual inspection of residual plots did not reveal any obvious deviations from homoscedasticity or normality. P-values were obtained by likelihood ratio tests of the full model with the effect in question against the model without that effect (Winter, 2013).

Regarding the categorization task, we computed the percentage of answers given by participants to each emotion category for each emotional condition (Happiness, Sadness, Fear, Anger, Disgust, Surprise, and Other emotion). For all analyses, we used R (R Core Team, 2012) and lmerTest (Kuznetsova et al., 2015).

The aim of the present study was to investigate the influence of contextual cues on our evaluation of facial expressions of emotion. In order to do this, we connected the field of research on emotion perception with the field of research on the perception of films. Specifically, we aimed at replicating the Kuleshov effect (e.g., Pudovkin, 1970; Kuleshov, 1974) by means of an improved experimental design based on Barratt et al., (2016), introducing some modifications which, in our opinion, added value to the experimental design (see section “Materials and Methods” for details).

In order to study the context-sensitivity of emotions under more ecologically valid conditions, we used dynamic scenes as contextual stimuli. Participants were shown 18 film sequences of neutral faces across three emotional contexts conditions (Neutral, Happiness, and Fear). The task was to rate the emotion displayed by a neutral target person’s face in terms of valence, arousal, and category. Hence, we adopted both a dimensional and a categorical approach to emotion (e.g., Ekman and Friesen, 1971; e.g., Russell, 1980) in order to have as much information as possible about the participants’ experience.

Our results confirmed the presence of a significant effect in terms of both valence and arousal for the Fear context only. More specifically, participants rated neutral faces in fearful contexts as significantly more negative and more arousing than neutral faces in both neutral or happy contexts. Moreover, participants tended to categorize the target person’s facial expressions choosing the emotion categories appropriate with the preceding context (positive emotions for the Happiness condition vs. negative emotions for the Fear condition). Hence, while from a dimensional point of view our results suggest the presence of a significant effect when neutral faces were paired with fearful contexts, from a categorical point of view our participants tended to choose the emotion categories congruent with the preceding context also when neutral faces were paired with happy contexts.

On the basis of the affective prediction hypothesis (Barrett and Bar, 2009; Barrett et al., 2011) the Kuleshov effect could be explained by a mechanism which detects the visual sensations of the emotional context and interprets them by means of the corresponding affective representations, generating a prediction to signal neutral faces as emotional faces. In our view, however, a more suitable explanation for the Kuleshov effect is that the context triggers the arousal and the emotional reaction in the observer who then attributes an emotional value to a neutral face.

More specifically, our results differ from Barratt et al., (2016) findings in the following terms: while Barratt et al. demonstrated the presence of the effect only when faces were paired with contexts of desire or sadness, our results also showed a significant effect when faces were paired with fearful contexts, but not if they were paired with happy contexts. As stated before, we employed only two emotional contexts (Happy and Fearful) in order to keep the design as simple as possible and to highlight the differences between opposite emotional conditions (see section “Introduction”). More specifically, we adopted fear as a negative emotion because, from an evolutionary point of view, it is capable of directing our attention to potentially dangerous stimuli (such as the scenarios depicted in our fearful contexts). In this regard, an interesting explanation is provided by the motivated attention theory (Lang et al., 1997; Bradley et al., 2003) also stating that cues that signal danger activate one of the two major motivation circuits (defensive vs. appetitive motivational systems; e.g., Bradley et al., 2001a,b; Lang and Bradley, 2010) pushing amplified orienting and attention responses (Bradley et al., 2012). Moreover, since the activation of these motivational circuits can be elicited also by pictures (e.g., Bradley et al., 2012), the “defensive” response is amplified when phobic or fearful individuals view fear-related pictures (Globisch et al., 1999; Sabatinelli et al., 2001; Bradley et al., 2012). Additionally, it has been demonstrated that this aversive response, defined by modulations in self-report, physiological, and behavioral systems (e.g., Lang et al., 1993; Sabatinelli et al., 2001), could also persist after slide offset (e.g., Globisch et al., 1999). We suggest that the same mechanisms are elicited when using fear-related videos, thus explaining our results. For these reasons, future studies aiming to assess this effect using fearful and phobic contexts should include an evaluation of phobic traits by means of dedicated questionnaires (e.g., Snake and Spider questionnaires, SNAQ and SPQ, respectively; Klorman et al., 1974).

The absence of a significant modulation of valence and arousal ratings when neutral faces were paired with happy contexts could be ascribed to the kind of positive scenarios we proposed to our participants. Indeed, among stimuli rated as pleasant, erotic materials elicit the strongest affective reactions (Bradley et al., 2001a,b; Sabatinelli et al., 2001, p. 719). As a matter of note, Barratt et al. (2016) demonstrated a significant effect exactly with desire contexts. In our opinion, altogether these results seem to suggest that this kind of contextual effect emerges more clearly when employing strong arousing emotional contexts as stimuli. Future studies should further clarify this aspect.

Taken together, our results again highlight the context-sensitivity of emotions and the importance of studying them under ecologically valid conditions.

MC, KH, and DB designed the experiment. MC, KH, and FS performed data acquisition and analyses. MC, KH, DB, FS, MU, and VG interpreted the results. MC wrote the paper. All authors have contributed to, seen and approved the manuscript.

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.