Facial expressions and the emotional perceptions made from those expressions have been confounded from the earliest facial inferencing studies. The facial expression stimuli perceived as happy, angry, fearful, disgusted, sad, or surprised have been consistently identified by these emotion labels (Ekman and Friesen, 1971). However, facial expressions and emotional interpretations (labels) are not the same thing. From the perceiver standpoint, facial expressions are physical stimuli on the faces of perceived persons. The emotional interpretations of the expressions are inferential labels made by perceiving persons about internal states of the perceived.

If participants in facial inferencing studies label these facial expressions with the expected emotion label (for example, a smiling face labeled as happy), the labeling was listed as “accurate” or “correct,” because the labels coincided with the original Ekman and Friesen pairings of facial expressions and emotion labels. However, even in the earliest studies, some participants made unexpected emotional inferences of facial expressions, such as labeling a scowling face as disgusted (Tompkins and McCarter, 1964). The emotional mislabeling of faces (according to the original Ekman and Friesen facial expression-emotion label pairings) continues to occur in facial inferencing research (Widen and Russell, 2008). The mislabeling almost always occurs between faces showing the negative emotions: anger, disgust, fear, and sadness, with the highest frequency of mislabeling occurring between anger and disgust (Egger et al., 2011).

Separating facial expressions from perceivers’ emotion labels lead to a question: Does the facial expression stimulus or the emotion label perception have more effect on trait inferences made from faces? Previous research has supported each factor. Sauter et al. (2011) demonstrated that facial inferences can be made from facial expressions alone, regardless of emotion labels or lexical context. In contrast studies by Tiedens (2001), Gendron et al. (2012) and Fugate et al. (2018) demonstrate that varying emotion labels to the same expressions can shift facial inferences. Clearly both facial expressions themselves and the attached emotion labels are determinants of facial inferencing. Further research is needed to determine when and how each factor influences the inference process.

The concepts of emotional valence and arousal may help to disentangle the inference influence of facial expressions versus emotion labels. Valence and arousal are two of the core dimensions of an emotional experience (Barrett, 1998), and Mehu and Scherer (2015) demonstrated that both valence and arousal influence the classification of emotional facial expressions into discrete categories.

Emotional valence was initially defined as either a positive (pleasant) or negative (unpleasant) affective response to an external or internal stimulus (Lewin, 1951). Since then the concept of emotional valence has become essential to defining emotional experience (Solomon and Stone, 2002; Colombetti, 2005; Fridja and Scherer, 2009). Experiencing emotional valence does not require the recognition of a specific emotion, since perceiving a stimulus as either positive or negative can initiate a motivational and behavioral sequence (Barrett, 2006). Support for the idea that valence can be experienced without specific emotion labeling comes from two studies that have examined emotional valence and physiologic and brain activity. Anders et al. (2004) assessed participant verbal and physiologic reactions and brain activity, while they viewed pictures of pleasant and unpleasant events. Their results indicated that different brain mechanisms and pathways (specifically relating to the amygdala, anterior parietal cortex, left supramarginal gyrus, and insular cortex) underlie the physiologic and verbal responses participants made to either pleasant or unpleasant stimuli. Viinikainen et al. (2010) also exposed participants to either pleasant or unpleasant pictures and discovered that different brain mechanisms and pathways were specifically associated with either pleasant or unpleasant pictures.

Shuman et al. (2013) presented a multi-faceted approach to valence, in which they distinguished between macro and micro-valences. Macro-valence derives from the basic pleasant/positive and unpleasant/negative distinction discussed above, and micro-valences, which are evaluations of various internal characteristics of the perceived individual and various environmental circumstances. Shuman et al. specify four micro-valences, one of which, goal conduciveness, is relevant to the emotion labeling and trait inferencing process. Goal conduciveness is defined as a perceiver appraisal of a situation in terms of the perceiver’s need satisfaction or goal achievement. According to the theory, faces would be evaluated as either pleasant or unpleasant, and then more specifically in terms of the perceiver’s needs and goals. It is possible that facial expressions themselves initiate macro-valence and that specific emotion labels result from consideration of various micro-valences.

Presumably arousal as well as valence plays a role in the facial inference process. Emotional arousal is defined as the level of activation or excitement that an individual experiences in an emotional episode (Rule and Nesdale, 1976). Arousal can be measured physiologically via heart rate and numerous other measures, and it can be assessed by self-report (Barrett et al., 2004). Although valence and arousal are considered to be separate variables influencing emotional experience, researchers have examined their interaction. Kuppens et al. (2013) reviewed this research across a variety of emotional experiences, assessing whether there was any discernable pattern between valence and arousal. The results of some reviewed studies revealed a V-shaped pattern shown on a two-dimensional graph, with arousal on the Y axis and valence on the X axis. The valence scaling went from intensely negative through neutral to intensely positive. Arousal went up as reported valence approached either valence extreme. Support for the V-shaped valence-arousal model comes from studies by Jallais and Gilet (2010) and Kuhbandner and Zehetleitner (2011) where arousal increased with the presentation of more extremely valent stimuli, such as an extremely angry face versus a mildly angry face.

The purpose of this study is an attempt to further understand the inferential impact (personality, temperament, and social traits) of facial expressions versus the inferential impact of the emotion labels attached to the facial expressions, in the context of valence and arousal. As stated above, perhaps, facial expressions themselves initiate the macro-valence of pleasantness or unpleasantness, and the emotion labels attached to the expressions reflect different micro-valent evaluations and differential arousal which then initiate specific emotion-related trait inferences. In this study, if the participants perceive the same scowling expressions as disgusted rather than angry, does this further change the personality and social inferences beyond those that are made to the facial expression itself? Two experiments address this question. In Experiment 1, we compared participant inferences to scowling and smiling faces, and in Experiment 2, we compared participants who labeled the scowling faces as angry versus those who labeled the same scowling faces as disgusted. We hypothesize that scowling faces will initiate negative macro-valence and overall negative inferences compared to the positive macro-valence initiated by smiling faces, regardless of the emotion label assigned to the faces (Experiment 1, Hypothesis 1). We further hypothesize that the anger emotion label made to the scowling faces as opposed to the disgust label will result in more negative inferences (Experiment 2, Hypothesis 2). We also hypothesize that participant reports of the pictured models’ arousal will be greater for those participants who labeled the smiling faces as happy, and the scowling faces as angry rather than disgusted (Experiment 1; Experiment 2, Hypothesis 3).

The data support the hypotheses that both the facial expression itself and the emotional interpretation of a facial expression guide the direction of the personality, temperament, and some social perception trait inferences. In our study, negative valence was demonstrated by negative inferences made to scowling faces, contrasting to the positive valence, and positive inferences made to smiling faces. Furthermore, participants who perceived the scowling faces as either angry or disgusted all made negative inferences, indicating that the expression itself creates negative valence in the perceiver, supporting Hypothesis 1 (scowling faces will initiate negative macro-valence and overall negative inferences compared to smiling faces, regardless of the emotion label assigned to the faces) and indicating support for the macro-valence concept. Participants labeling the scowling faces as angry inferred greater negative inferences while participants who perceived the same scowling faces as disgusted made less negative inferences, supporting Hypothesis 2 [that the different emotion labels (micro-valences) made to the scowling faces will result in differences in the negativity of inferences] and lending support for the micro valence concept.

Hypothesis 3 (that self-reported arousal ratings would be high for smiling faces labeled as happy and higher for scowling faces labeled as angry, rather than disgusted) was also supported. These results support the V-shaped model of the relation of arousal to valence proposed by Kuppens et al. (2013). The results also suggest that arousal is a component in both macro-valence assessment of a facial expression, and in the micro valence assessments leading to an emotion label of the facial expression. This is not surprising given that arousal is one of core components of both emotional expression and perception.

Of the four social perceptions, the two variables that failed to yield significant emotion label differences, Attractiveness and Facial Maturity, deserve explanation. Previous research exploring the effects of gender and age on the inference process showed that male faces were perceived as less Facially. Mature and less Attractive than female faces (Radeke and Stahelski, 2020). Additionally, and as expected, older models were perceived as more Facially Mature and less Attractive than younger faces (Radeke and Stahelski, 2020). We speculate that the results of this study, which combined age and gender, reflect a combination of the age and gender effects, resulting in the two non-significant results.

Additional support for Hypotheses 1 and 2 comes from research that examines behavioral reactions to perceived facial expressions. Mirabella (2018) examined participant reactions to happy faces versus fearful faces. Reactions are positive to happy faces and negative to fearful faces only if participants are focused on the emotions shown in the faces, as opposed to focusing on the gender of the individual in the pictured face. This finding demonstrates one, the positive/negative macro-valence of happy versus fearful faces, and two, the influence of the micro-valent appraisal of situational goal relevance.

Mancini et al. (2020), using the same research paradigm, compared reactions to angry faces, in addition to happy and fearful faces. They found that threatening faces (angry or fearful) increased reaction times and errors more than non-threatening faces when the facial expressions were relevant to the behavioral choice, and that angry faces increased reaction times and errors much more than fearful faces. Participants presented with angry faces presumably viewed them as direct threats, a micro-valent goal/need relevance appraisal correlated with increased arousal. Fearful faces, like faces labeled as disgust, may indicate a different micro-valent goal/need appraisal – that there is a possible indirect threat somewhere in the surrounding environment, an appraisal associated with reduced arousal. These results are similar to our results comparing anger and disgust labeling. Their measure of valence indicated that happy faces were rated positively and both threatening faces were rated negatively, supporting the macro-valence concept, and the different evaluations of the angry and fearful faces support the micro valence concept.

The emotion labeling of facial expressions is an important part of the facial inferencing process. We believe that the results presented here increase the overall understanding of when and how emotion labeling influences facial inferencing, and our understanding of the limitations of emotion labeling. Unlabeled facial expressions act as stimuli influencing the perception of emotional valence. The emotion labels of expressions varying in arousal cause shifts within overall negative valence.

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

The studies involving human participants were reviewed and approved by the Central Washington University, Human Subjects Review Council, https://www.cwu.edu/hsrc/. Authors are students or faculty at Central Washington University. Written informed consent for participation was not required for this study in accordance with the national legislation and the institutional requirements.

AS and MR contributed to the conception and design of the study and collected the data. AA, MR, and NB contributed to the data analysis. AS, MR, and AA contributed to the manuscript construction and editing. All authors contributed to the article and approved the submitted version.

Funding for this research was provided by the Central Washington University Provost Faculty/Student Research Grant 2020-2021.

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.