Facial expression is indispensable in human life, and it plays a crucial role in social interaction. However, the way our brains comprehend this non-verbal language remains a mystery. Studies have shown that recognizing expressions involves different processing stages (Luo et al., 2010; Zhang et al., 2013; Calvo and Nummenmaa, 2016), which suggests that the human brain must accumulate enough information from different aspects (e.g., arousal, threat degree, approachability) of a presented expression before finally understanding it. Emotional valence is one such aspect as it refers to the intrinsic positive or negative character of expressions (Colombetti, 2005) and provides fundamental information for expression recognition (Russell, 2003).

In accordance with the dimensional model (Russell, 2003), which proposed that emotional information of the orthogonal dimensions of affective valence and arousal was perceived initially to assist later processing, behavioral evidence suggests that the processing of valence occurs early and automatically (Lipp et al., 2009; Calvo et al., 2010; McLellan et al., 2010). For example, McLellan et al. (2010) used 100 ms SOA and found priming effects of emotional valence on participants’ response time, this finding is consistent with the opinion that valence information is involuntarily extracted and might serve as an important part of later emotion-related processing (Russell, 2003).

In contrast, ERP studies have produced ambiguous results on the time course of valence processing. Some have found that the P1, N1 and P2 components can be modulated by emotional valence (Smith et al., 2003; Huang and Luo, 2006; Lithari et al., 2010; Zhang et al., 2013; Alguacil et al., 2017). However, other studies have shown that valence can also affect later components, such as LPP (Bernat et al., 2001; Schupp et al., 2004), P3 (Huang and Luo, 2006; Lithari et al., 2010; Alguacil et al., 2017), and LPC (Huang and Luo, 2006). It has been reported that these components are related to differentiation of certain expressions (Luo et al., 2010; Calvo and Beltrán, 2013; Zhang et al., 2013; Calvo and Nummenmaa, 2016), which should occur after valence processing according to the dimensional model. Due to these inconsistent results, it is unclear whether valence information is processed at an early stage.

Categorization is a fundamental property of the human brain. It is a ubiquitous way to simplify input information from the outside world (Harnad, 2003). Categorical perception (CP) also occurs automatically, and we involuntarily lean to categorize stimuli (Maiste et al., 1995; Freeman et al., 2010; Xi et al., 2010). According to Harnad (2003), the CP effect can be operationally defined by between-category separation or within-category compression, i.e., faster RTs or greater accuracy for between- than for within-category stimuli in behavioral experiments. This effect has been found in several stimulus domains, such as color (Fonteneau and Davidoff, 2007; Mo et al., 2011), speech (Liberman et al., 1957; Maiste et al., 1995; Xi et al., 2010), face (Campanella et al., 2000; Morel et al., 2009) and facial expression (Campanella et al., 2002a,b; Kiffel et al., 2005). However, little evidence of the CP of valence information has been reported, though it is a common sense that emotional valence can be categorized. Specifically, it remains to be explored whether the subjective dichotomy of emotional valence has an objective basis and how early this processing can occur.

Because of its excellent time resolution, ERP was used in the present study to explore the time course of the CP effect of emotional valence. The N170 component which peaks approximately 150–170 ms post-stimulus is thought to be sensitive to face and facial expression processing (Schacht et al., 2008; Calvo and Beltrán, 2013; Zhang et al., 2013; Cao et al., 2014). The N170 component has also been found to reflect CP (Campanella et al., 2002a,b; Jacques and Rossion, 2006; Ganis et al., 2012). Campanella et al. (2002b), for example, used morphed emotional faces to test the CP of basic facial expressions and found that the amplitude of the N170 was significantly larger for between- than for within-category stimuli. In addition, previous studies have found that the amplitude of the P1 component is larger and the latency is shorter under between-category conditions (Holmes et al., 2009; Maier et al., 2014). Moreover, evidence indicated that the P1 can also be modulated by complicated stimuli such as faces (Itier and Taylor, 2002, 2004). These results make the P1 an additional ERP component of interest in the current study. Importantly, apart from the objective basis of the emotional valence dichotomy, the present study aimed to explore how early this processing could occur. Previous studies have shown that the CP effect is revealed by both early and relatively late components, which might reflect different levels of categorical information processing (Fonteneau and Davidoff, 2007; Holmes et al., 2009). For example, Holmes et al. (2009) found a CP effect of color on the P1 and the P2, indicating that both early perceptual and later post-perceptual processes are involved in this processing. In the current study, we tested the P1 and the N170 to explore whether the categorization of emotional valence depends on merely low-level perceptual processes or needs more sophisticated information.

Four basic emotions (happiness, anger, disgust and surprise) were chosen in the present study to constitute the within-valence-category condition and the between-valence-category condition. A “same-different” judgment task was adopted, as in previous studies concerning the CP effect (Bornstein and Korda, 1984; Campanella et al., 2000, 2002b; Liu et al., 2010; Hu et al., 2014), to test whether there is CP of emotional valence. After a behavioral version of the task, ERP was recorded while participants were performing the same task. In the ERP experiment, the mean amplitude of the P1 and the N170 were compared across within- and between-category stimuli to assess the time course of the CP of emotional valence, with the contributions of simple perceptual differences being eliminated between them. We hypothesized that the CP of emotional valence not only exists (that is, there should be RTs or accuracy index showing the CP effect), but also occurs at a relatively early stage, such as the P1 and the N170, in accordance with the idea proposed by the dimensional model (Russell, 2003).

The mechanism of emotional valence processing was ambiguous, even though this issue has been widely explored. It remains to be investigated whether the subjective dichotomy of emotional valence has an objective basis and at which stage this processing might occur. In the present study, we focused on the CP effect of emotional valence to address this issue. We observed an obvious CP effect in the behavioral experiment, and the N170 component had a larger amplitude in the within- than in the between-category condition. These results not only confirm the objective basis of the dichotomy of emotional valence, but also provide additional support for the view that the processing of emotional valence occurs at a relatively early stage (Russell, 2003). Furthermore, compared with the insignificant results on the P1, the effect shown on the N170 indicated that fairly more complicated information was needed for categorization of emotional valence whereas merely low-level perceptual processes were insufficient. This finding aligns with previous evidence showing that the N170 reflects some sophisticated processes (Batty and Taylor, 2003; Blau et al., 2007; Krombholz et al., 2007).

Whether the N170 reflects the processing of facial expressions is controversial, and the present study may shed some light on this issue. Traditionally, the N170 was thought to merely index face-specific structural encoding (Eimer, 2000; Itier and Taylor, 2004; Rellecke et al., 2013; Wang et al., 2015; Wang et al., 2016), consistent with the two-stage model of face processing which assumed that the encoding of structural information and emotional information occurred independently (Bruce and Young, 1986). However, studies pertaining to this issue have produced incongruent empirical evidence. Some found that the N170 was not affected by facial expressions (Holmes et al., 2003; Ashley et al., 2004; Holmes et al., 2005; Pourtois et al., 2005), whereas others found that facial expressions could modulate the N170 (Batty and Taylor, 2003; Blau et al., 2007; Krombholz et al., 2007). As mentioned above, facial expressions involve information from different dimensions (Russell, 2003). It is possible that the N170 reveals processing of certain dimensions rather than the integral facial expression. This notion might partially explain those inconsistent results and it has been supported by previous studies. For instance, the multi-stage account of facial expression recognition proposed that the N170 reflects the stage of distinguishing emotional and neutral expressions (Luo et al., 2010; Zhang et al., 2013; Calvo and Nummenmaa, 2016). Furthermore, the N170 was shown to be modulated by valence (Hietanen and Astikainen, 2013; Alguacil et al., 2017). For example, Hietanen and Astikainen (2013) used an affective priming paradigm and found that the amplitude of N170 elicited by neutral faces was affected by the valence of primes. The current study extended past work by providing direct evidence of the N170 reflecting the CP effect of emotional valence, indicating that the N170 component reflects the categorization of emotional valence, which is crucial for further stages of emotional processing of faces (Russell, 2003). Therefore, in addition to structure encoding, the N170 is also involved in facial expression processing, especially for the early processing of emotional valence.

As a well-known face-related ERP component, the N170 was also found to reflect the categorization of different aspects of facial information (Campanella et al., 2000, 2002a,b; Jacques and Rossion, 2006; Ganis et al., 2012), including face identities and facial expressions. Interestingly, however, Research has not yet considered the influence of emotional valence. Campanella et al. (2002b) used morphed faces to explore the CP effect of two basic expressions: happiness and fear. They found that the amplitude of the N170 component elicited by the second facial image decreased in the within-category condition. Importantly, two expressions chosen in that study apparently belong to different valence categories, such that the N170 effect might be contaminated by the CP effects of emotional valence. The present study directly explored the CP effect of emotional valence while keeping the physical distance between pairs identical. The results showed an effect on the N170 component: the amplitude of the N170 component was significantly larger for within- than for between-category stimuli. This effect on the N170 component seems to reverse the pattern of the previous study (Campanella et al., 2002b), but the important differences in the experimental paradigm and the stimuli should be considered between these two studies. First, in the previous study images were presented successively, this manipulation involves a matching procedure and led to a pattern similar to repetition suppression or an adaptation effect (Feng et al., 2013). In contrast, in the current study, the N170 component was more likely to be affected by the difficulty of comparison between different conditions, considering that two images were presented simultaneously. Second, the within-category stimuli from the previous study shared the same expression and the same emotional valence. This set might reduce the mental distance between stimuli more than the present study, in which within-category stimuli shared the same emotional valence but had different expressions. Nevertheless, the present study provides direct evidence that the N170 reflects the CP effect of emotional valence.

No significant CP effect on the P1 was found in the present study, implying that the P1 might be insensitive to the CP of emotional valence. This was different from the results of a previous study, in which the P1 amplitude was found to be significantly higher in the between- than in the within-category condition (Maier et al., 2014). However, this study focused on the influence of the semantic content of verbal categories and used pictures of tools. An oddball paradigm was adopted in their study whereas a matching task was used in the current study, which may to some degree influence the results. Furthermore, the P1 is thought to be mainly sensitive to physical rather than emotional attributes (Calvo and Nummenmaa, 2016). At most it is related to the differentiation between threatening faces and other faces, which must be processed early for self-protection and might involve different processing mechanisms than other expressions do (Luo et al., 2010; Zhang et al., 2013). Therefore, consistent with the finding that the N170 but not the P1 indexes the earliest time for the CP of faces (Ganis et al., 2012), the present results further showed that the N170 but not the P1 indexed the earliest time for the CP of emotional valence.

It should be noted that the results of the present study do not provide direct evidence that the categorization of emotional valence will support later processing of facial expression, and further studies are needed to resolve this issue. Previous studies have found evidence that facial physical properties are enough for expression recognition (Dailey et al., 2002; Susskind et al., 2007) and that valence information might be futile in expression recognition (Calvo and Beltrán, 2013). However, as mentioned above, previous studies have seldom considered emotional valence, and it is inappropriate to eliminate the contribution of this process. Whether and how categorization of emotional valence contributes to expression recognition remains to be explored.

There were several limitations in the present study. Although the CP effect on the N170 was significant, its size was small. This small effect might result from the artificial faces used in the experiment, which were found to influence the amplitude of the N170 (Mühlberger et al., 2009). In addition, the physical distance between the pairs in the current study might be too short. Thus, in further studies real emotional faces should be involved, and the physical distance between pairs should be enlarged to increase this effect. In addition to the small effect, previous studies have found that the N170 component can be modulated by mental resources allocation (Morgan et al., 2008; Senholzi and Ito, 2012; Müller-Bardorff et al., 2016). Thus, amplitude of the N170 under the within-category condition was larger probably because more resources were needed when task difficulty was higher. However, although the difficulty level was also higher for identical pairs than for between-category pairs, there was no significant difference in the amplitude of the N170 across these two conditions. Therefore, variations of task difficulty and mental resources allocation cannot entirely explain the results. However, these confounding factors must be better controlled in future studies.

In sum, the present study substantiated the existence of the CP of emotional valence and indicated that the N170, but not the P1, its neural correlates. The categorization of emotional valence occurs at a relatively early stage of processing.

RQ and SF designed the study. RQ collected data. RQ and HW analyzed data. RQ, HW, and SF wrote the paper.

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.