A total of 33 (16 males) right-handed undergraduates ranging from 18 to 26 years participated in the study. Five participants were excluded: four due to equipment problems, and one for reversing the task instructions. Thus, there were 28 effective participants. All participants had a normal or a corrected-to-normal vision. All participants volunteered to participate in the experiment and provided written informed consent.

Images of facial and bodily expressions were taken from two image databases: the Chinese Affective Picture System (CAPS) and the Bochum Emotional Stimulus Set (BESST) picture library. All the body pictures in the BESST picture library were covered with faces, the background of all pictures was gray, and all limbs appeared in the middle of the pictures. Eighty images were selected, half of which were male and female, and half were positive and negative.

Ten psychological postgraduates scored the valence and arousal of 80 face pictures and 80 body pictures using a 9-point Likert scale. Valence refers to the degree of pleasure expressed by the picture itself, from very pleasant to very unpleasant. Arousal refers to the degree of arousal of the emotion, from calm to excitement. There was no time limit for stimuli presentation or response from participants. The average score for each picture was taken as the average score of the 10 participants. Finally, 60 pictures each of effective facial and bodily expressions were selected. Positive pictures only included happiness, while negative pictures included anger, disgust, fear, sadness, and surprise. ANOVA showed that the valence of negative (2.958 ± 1.014) facial expression pictures was significantly lower than that of positive (6.223 ± 1.042) facial expression pictures [F(1, 9) = 72.01, p < 0.001, η² = 0.889] and the valence of negative (2.893 ± 0.803) bodily expression pictures was significantly lower than positive (6.537 ± 1.384) bodily expression pictures [F(1, 9) = 34.33, P < 0.001, η² = 0.792]. There was no significant difference between the arousal degree of negative facial expression pictures (4.840 ± 1.904) and positive facial expression pictures (4.490 ± 1.691) [F(1, 9) = 0.40, p > 0.05] or between the arousal degree of negative bodily expression pictures (4.657 ± 1.840) and positive bodily expression pictures (5.290 ± 1.901) [F(1, 9) = 1.72, p > 0.05].

Adobe Photoshop CS6 was used to combine facial and bodily expressions. Two kinds of facial expressions and two kinds of bodily expressions were combined to produce four kinds of face–body combination stimuli: positive face–positive body, negative face–negative body, positive face–negative body, and negative face–positive body. Among them, the first two were expression consistent combinations, and the latter two were expression inconsistent, each with 15 pictures, for a total of 60. The contrast and luminance of face–body combination stimulation were controlled. The face-to-body ratio was approximately 1:7, so the combination picture was as realistic as possible and close to the real proportions of the human body. An example is shown in Figure 1.

The same faces and bodies were presented separately as the control stimuli. The stimulus presented alone remained as the composite stimulus with the same size and position on the screen. Only the outer outline of the head and the gray background of the face are “filled in” bodily expressions. Facial stimulation covered the head only.

A 2 (facial expressions: positive, negative) × 2 (bodily expressions: positive, negative) × electrode points (P1: O1, Oz, and O2; N170: P7, P8, PO7, PO8; N2: F1, FZ, and F2; P3: PZ, P1, and P2) design was selected. The dependent variables were the peak, latency, and average amplitude of the ERP components.

When isolated facial and bodily expressions were presented, the experimental design was 2 (stimulus: face, body) × 2 (expressions: positive, negative).

This is a two-part experiment: the consistency experiment, which presents the face–body compound stimuli, and the control experiment, in which the same participants see the face and body separately. Each part included two blocks, and all stimuli were randomly repeated twice in each block. Thus, each block had 120 trials, and each part had 240 trials, for a total of 480 trials. The order of pressing keys was counterbalanced among participants, and the order of the blocks was random.

The experiment was conducted in an electromagnetically shielded and quiet electroencephalographic (EEG) laboratory. Participants sat comfortably in a chair with their eyes approximately 80 cm away from the screen. The size of the face–body compound stimulus presented on the screen was approximately 4 × 8 cm. The stimulus included isolated bodily expressions, isolated facial expressions, and compound expressions. The participants pressed “F” for positive and “J” for negative, and the left and right buttons were counterbalanced among participants. Figure 2 shows the flowchart of the trial procedure. Each trial started with a 500-ms fixation cross “+,” slightly above the center of the screen, which is the “chest height” of the body in the picture. Then, an empty screen was presented for 300 ms, the stimulus (isolated body expression/isolated facial expression/compound expression) was presented for 300 ms, and then an empty screen was presented for 1,500 ms. After the participant responded, the stimulus disappeared. If there was no response within 1,500 ms, the blank screen disappeared automatically and the next trial began after a 1,000-ms interval. In the compound presentation, participants were asked to judge the facial expressions in half of the trials and the bodily expressions in the remaining half. In the isolated presentation, the participants were asked to judge the expressions conveyed by the face or the body. This study utilized a block design, in which the block was a random factor.

The EEG was recorded from a 64-channel Ag/AgCl electrode cap of the international 10–20 system. A vertical electrooculogram was recorded above and below the left eye, and a horizontal electrooculogram was positioned 10 mm from the outer canthus of each eye. During the online recording, the central forehead electrode was grounded, and the overhead electrode was used as the reference electrode. Signals were sampled at 1,000 Hz using AC mode sampling. The impedance was maintained below 5KΩ.

Curry 7 software was used for the offline analysis. All EEG signals were re-referenced to the average of the entire head. The EEG data were segmented into periods of 1,000 ms, including a 200-ms pre-stimulus baseline. Offline correction of eye movement artifacts was performed. Trials with voltages exceeding ± 100 μV at any electrode were discarded to exclude artifacts. After artifact rejection, no more than 20% of the trials were excluded. For isolated positive and negative face and body, and compound stimuli, about 48–56 trials were retained. A zero-phase waveform was selected for filtering, low-pass 30 Hz, and behavioral data were fused for segmentation and superposition.

P1, N170, N2, and P3 were analyzed according to the experimental purpose, group average waveform, and related literature (Meeren et al., 2005; Gu et al., 2013). P1 (60–140 ms) occipital sites were O1, Oz, and O2. N170 (120–200 ms) occipito-temporal sites were P7, P8, PO7, and PO8. N2 (200–300 ms) sites were F1, FZ, and F2. P3 (200–650 ms) sites were PZ, P1, and P2. The peak amplitudes and latencies of P1, N170, and N2 were analyzed, and the average amplitude of P3 was analyzed. The isolated and compound stimuli, P1 and N170, were analyzed, and for N2 and P3, only face and body compound presentation was analyzed. Repeated-measures ANOVA was employed to analyze the amplitude/latency of P1 and N170. For N2 and P3, the positive-face with positive-body and negative-face with negative-body were combined into one level (consistent), while the positive-face with negative-body and negative-face with positive-body were combined into one level (inconsistent), and the data were analyzed by one-way ANOVA.

When presented with isolated faces and bodies, the main effect of stimulus was significant on accuracy [F(1, 27) = 17.432, p < 0.001, η² = 0.395] and RTs [F(1, 27) = 12.433, p < 0.01, η² = 0.321]. The RTs and accuracy for the face were faster and higher, respectively, than those for the body (606.142 ± 103.023 ms vs. 661.563 ± 97.296 ms; 87.322 ± 8.633 vs. 81.399 ± 8.578). The values reported in brackets are mean ± SD (M ± SD).

When judging facial expressions for the compound stimuli, the interaction between facial and bodily expressions was significant for both accuracy [F(1, 27) = 14.302, p < 0.01, η² = 0.357] and RTs [F(1, 27) = 19.481, p < 0.001, η² = 0.422]. Simple-effect analysis showed that when the faces were accompanied by bodies with congruent expressions relative to incongruent expressions [F(1, 27) = 10.309, p < 0.01, η² = 0.297], participants made significantly faster (655.431 ± 120.158 ms vs. 633.021 ± 103.750 ms) and better (88.731 ± 7.963 vs. 83.021 ± 8.381) decisions. When judging bodily expressions for the compound stimuli, the interaction between facial and bodily expressions was significant for both accuracy [F(1, 27) = 17.792, p < 0.001, η² = 0.412] and RTs [F(1, 27) = 21.312, p < 0.001, η² = 0.448]. Simple-effect analysis showed that when the bodies were accompanied by faces with congruent expressions relative to incongruent expressions [F(1, 27) = 13.661, p < 0.01, η² = 0.342], participants made significantly faster (613.226 ± 98.430 ms vs. 649.106 ± 102.619 ms) and better (87.837 ± 8.364 vs. 81.264 ± 7.981) decisions.

The results showed that the main effect of a stimulus was significant. For P1, the peak amplitude of the body was larger but the latency was shorter, which is inconsistent with the results of Meeren et al. (2005). In our study, positive and negative facial and bodily expressions were used, while Meeren et al. used negative emotions (fear and anger). However, significant differences were found only for the presentation of isolated stimuli and not for compound stimuli, as in Meeren et al. It is believed that P1, as an exogenous component of early perceptual processing, is sensitive to low-level threat information of stimulation. However, bodily expressions are lower-frequency information than facial expressions, thus inducing a greater P1 component (Pourtois et al., 2005). The shorter latency of the body indicated that the body received attention before the face, which is consistent with the findings of Gu et al. (2013). However, the inconsistency with Gu et al. (2013) was that P1 had no bias to negative emotions. A possible reason is that the body is larger than the face, so the participant pays attention to the body first. Therefore, the proportion of face–body size in this study is worthy of attention. This study did not find a bias of P1 toward negative emotions, which may be due to the different materials selected in this experiment. The negative emotions selected in this study included anger, disgust, fear, sadness, and surprise, while the positive emotions contained only one expression (happiness). Thus, the group of positive expressions is more perceptually homogenous, as they depict only a single expression, while the negative expressions are more heterogeneous, as they depict several emotional expressions. Indeed, there is a possibility that different emotions in compound stimuli could have different effects (Karaaslan et al., 2020). Therefore, further research is required. Although the main effect of expression was not found, the amplitude of the negative body was larger than that of the negative face, indicating that the P1 component was affected by negative emotion under the influence of stimulation. In addition, our research conducted using different tasks found no difference except for the electrode.

The main effect of a stimulus was significant, the peak amplitude of the face was larger, and the latency was longer. N170 reflects the processing of configuration, and many studies have found N170 is induced by the face (Zhang et al., 2015). The body also triggered N170, but the amplitude was not as large as that of the face. To a certain extent, facial and bodily expressions have the same processing characteristics (Aviezer et al., 2012b; Kret et al., 2013; Poyo Solanas et al., 2018). The results suggest that, from P1 to N170, facial emotions began to play a role. The main effect of emotion was significant, and the positive peak amplitude was greater than the negative peak amplitude, indicating that emotion was further processed at this stage. Therefore, at this stage, we not only distinguish the stimulus but also distinguish emotional valence. This may be because facial expressions are the primary way to judge emotion in daily life and positive emotions attract more attention. However, some studies have suggested that N170 does not reflect emotion, while others have found that it has an emotional effect (Eimer and Holmes, 2002; Batty and Taylor, 2003; Hietanen and Nummenmaa, 2011; Schindler et al., 2021). The main effect of bodily expressions was significant, and the positive effect was greater than the negative effect. This indicates that bodily expressions were processed at this time, and some studies have found that the effect of bodily expressions was found earlier (Gu et al., 2013).

When judging facial expressions for the compound stimuli, the body had little effect on the face, and there was no significant effect. However, the face advantage of N170 did not exist in comparison with the isolated presentation. This indicates that the influence of the body on the face may have occurred at a relatively early stage; when judging bodily expressions, the effect began to appear. The interaction between facial and bodily expressions was significant, and the consistent latency was longer than the inconsistent latency (Syrjänen et al., 2018). The N170 of compound stimulation seemed sensitive to both the inconsistency between facial and bodily expressions and the stimulus. A possible reason is that individuals have completed the preliminary judgment of the body in the early stage and then turned to the structural coding of the face. In this stage, they can process the face and body simultaneously. The different tasks of the experiment led to different results. For the compound stimuli, the faces and bodies were fairly clearly photoshopped together, which may have disrupted the perception of participants of them as a single stimulus.

The main effect of consistency on amplitude and latency was significant, showing greater amplitude and longer latency for inconsistency than for consistency.

This finding indicates that the emotional conflict information was sensitive and lasted for a long time. The interaction between facial and bodily expressions was significant at P3. The average amplitude was greater for consistency than for inconsistency, indicating continuous attention to consistent emotional information. Participants may have first paid attention to inconsistent information because it is more prominent than consistent information and then classified and evaluated emotional stimuli using high-level cognitive processing.