Data from 14 volunteer participants between 18 and 29 years of age (mean age 21.6 years; six females; all right-handed) were included in this study. Handedness was confirmed using the Edinburgh Handedness Inventory (mean handedness quotient of 88R). All participants had normal or corrected-to-normal hearing and vision and reported no previous history of neuropsychiatric disorders. ERP data from an additional four participants were collected, but were excluded due to excessive and persistent movement or blink artifacts. Informed consent was acquired for the experiment, as approved by the Institutional Review Board of West Virginia University.

Stimuli consisted of gray scale video clips of a full-on face with a neutral expression and eyes looking directly at the observer. The eyes would randomly change to blink, to gaze upward and back, to gaze leftward and back, or make a duration-matched (with gaze conditions) eye closure (Figure 1). The direct gaze face was present for the entire experimental run (see also Puce et al., 2000), such that it appeared the face was looking at the observer and periodically blinking, closing the eyes, or looking up or away, always returning to the baseline direct gaze state. These apparent movements made up the four eye conditions: Blink, Eye Closure, Leftward Gaze, and Upward Gaze. All gaze aversion conditions (Eye Closure, Leftward and Upward gaze) had equal durations (100 ms) and were designed to look natural, whereas the eye blink was very brief (33 ms). The direct gaze face (no blank screen, or scrambled stimulus) remained on the screen between trials, with an interstimulus interval of 2 s between each eye change. There were two experimental runs of 4 min each, one with a male face and the other with a female face, and presentation order was counterbalanced across participants. Both faces were unfamiliar to all participants. To encourage participants to maintain their attention on the stimulus display, an additional target detection condition was included. Participants were required to detect a transparent checkerboard pattern briefly superimposed on the direct gaze face (target stimulus) and make a button press response each time it appeared. There were a total of 40 trials for each eye movement condition, as well as 40 target trials (half of these occurred with the female face, the other half with the male face).

All video stimuli were presented using Presentation software (version 11; Neurobehavioral Systems, CA, USA). Video clips of the faces were dynamic and featured apparent motion of the eyes, which maintained their end positions for 100 ms (Leftward gaze, Upward gaze, and Eye Closure) or 33 ms (Blink) before returning to the baseline direct gaze condition. To maintain consistent stimulus timing, target duration (superimposed checkerboard pattern) was also equal to the duration of the presented eye change stimulus. Although there were only three video frame changes from direct gaze to eye change back to direct gaze in this apparent motion stimulus, as in a previous study (Puce et al., 2000), all participants described the eye movements as realistic. The stimulus faces were projected on a white screen placed directly in front of the participant at a distance of 195 cm, and subtended a visual angle of 14.3° × 16.7° (horizontal × vertical). The two eyes in each face subtended a visual angle of 3.4° × 0.73°. Stimulus condition order was randomized across trials.

Each participant was tested individually in a quiet, dimly lit room, while seated in an armchair. Visual stimuli were projected onto a white screen in front of the participant rather than viewed from a computer monitor to improve visibility of the facial movements and make the face seem more realistic (projected dimensions were approximately life size). Each participant completed two experiments in one ERP session in randomized order. Participants were given a short rest period (several minutes) between experiments. The second experiment had an unrelated purpose and will not be reported here. Before the experiment, participants were instructed to minimize their own eye and face movements, to keep their gaze centered on the bridge of the nose of the stimulus face, and to press the response button to targets as quickly and accurately as possible.

The electroencephalogram (EEG) was recorded from participants using a 128 channel Neuroscan Electrocap with sintered Ag/AgCl electrodes, with a midline frontal ground and dual reference electrodes placed on either side of the nose. Four electro-oculographic (EOG) electrodes captured horizontal and vertical EOG components (Figure 1 in Appendix). EEG and EOG signals were recorded with a gain of 5000, sampled at 250 Hz/channel, and band pass filtered at 0.1–100 Hz. A continuous EEG recording was made during each experimental run and included a digital time stamp to indicate the onset of each stimulus condition.

At the completion of the recording session, electrode locations were digitized with a Polhemus 3 Space Fastrak digitizer (see Appendix).

Electroencephalogram data were segmented offline into epochs of 1100 ms, which included a 100 ms pre-stimulus baseline. A semi-automated artifact detection routine identified participant eye blinks in segmented EEG epochs, and data were also visually inspected for additional electromyographic artifacts. Artifactual EEG epochs were excluded from subsequent analyses. An average of 10.6 out of 40 trials were excluded per participant per condition. There were no significant differences in artifact removal between conditions (average for viewing Blink 10.6, Eye Closure 10.7, Upward gaze 9.6, Leftward gaze 11.2 trials). Segmented EEG epochs were low pass filtered using a filter cut-off of 40 Hz, then baseline corrected with respect to the pre-stimulus baseline, and averaged as a function of each stimulus condition for each participant. ERPs elicited in response to target stimuli were excluded from further analyses.

The averaged ERPs for each participant were used to create grand average ERPs for each stimulus type across the participant group. Potential latency ranges for a subsequent semi-automated peak identification analysis were defined for each ERP component from the grand averaged ERPs waveforms. Peak latencies and amplitudes for all stimulus types in individual participants and grand averaged ERPs were identified within those pre-selected time intervals. From the average ERPs of each individual participant, ERP component peak latency, and amplitude measures were calculated for each condition.

Topographical voltage maps and Laplacians (second spatial derivative of the voltage map; Puce et al., 2007) of the grand average ERPs at ERP component peaks were created using EMSE software (Source Signal Systems, CA, USA).

Based on inspection of topographic voltage maps, clusters of electrodes were selected whose mean ERP amplitude and latency values were calculated for P100, N170, and amplitudes for later ERP components. Data for P100 and N170 components were entered into two-way repeated measures analysis of variance (ANOVA) using SPSS V15 to test for Condition and Hemisphere. Amplitude data for later ERP components were analyzed using a repeated measures ANOVA for Condition.

Participants indicated by keyboard press when they detected a transparent black and white checkerboard overlaid on the direct gaze face. The mean reaction time for target detection was 501.0 ms (SD = 138.6 ms), with 95.1% accuracy.

All eye change conditions, including the viewed eye blink, produced a robust ERP complex consisting of a P100 and N170 seen mainly at temporo-occipital electrodes (e.g., centered on equivalent 10–10 system sites P07 and P08). Additionally, large later ERP components, were also observed to all stimulus conditions, and these later ERPs tended to be more extensively distributed across the scalp (see white markers on topographic maps in Figure 3). We discuss each ERP component below.

To test the hypothesis that late ERP components would show an amplitude gradient based on putative social significance, we examined later ERPs at clusters of electrodes showing the greatest amplitudes. A series of positive late ERPs were observed, occurring at around 250, 450, and 600 ms post-stimulus, i.e., P250, P450, P600. To facilitate distinction of spatial distributions of the later ERPs, Figure 3 displays Upward and Leftward gaze topographic voltage maps with higher voltage thresholds than those for Eye Closure and Blink (Figures 3A–C; for Leftward gaze at same threshold see Figure 2 in Appendix). In terms of spatial distribution and extent, the scalp distributions of the later ERP components showed some differences between the Blink and the other conditions (e.g., Figure 3). We discuss each of these ERP components separately.

Finally, we examined the data of individual participants for potential differences in the duration of significant activation across the Blink and Leftward gaze conditions. We did this over multiple electrodes which included locations over the right occipito-temporal, midline parietal, central, frontal, and right anterior frontal scalp (Figure 5). To do this, we measured the time elapsed after which no significant peaks occurred; an effective return to baseline (RTB). To determine the RTB we first calculated the t-value vs. zero for each point on the waveform (using Neuroscan Scan 4.3 software). The RTB was defined as the end of the last significant peak of sustained (20 ms, 5 time points) significance (t > 1.67 for n = 22–46 epochs, see Appendix for details). The plot in Figure 5 displays time interval post-stimulus on the y-axis and individual participant data on the x-axis. A group mean is also depicted. A right temporo-occipital electrode, analogous to PO8 in the 10–10 system, which typically exhibits a maximum for N170 amplitude, showed that for both Blink and Leftward gaze, every participant exhibited a significant N170 peak (Figure 5, see gray overlay). Interestingly, not all individuals showed later ERP activity. For other midline electrodes (Cz, Pz, Fpz), more participants showed late peaks for the Leftward gaze condition compared to the Blink condition (e.g., for P250 at Cz: 14 for Leftward, 10 for Blink; for P600 at Pz: 13 for Leftward, 9 for Blink; for late frontal at Fpz: 11 for Leftward, 8 for Blink).

In sum, for late ERPs, contrasts revealed a general trend in progressively decreasing ERP amplitudes, in that Leftward ≥ Upward > Closure = Blink. There were also laterality differences and scalp distribution differences across conditions. Although fewer individual participants showed significant late potentials for the Blink condition, on average, there were significant potentials, even at 600 ms. The P450 at the posterior temporal electrodes and the P600 at the midline parietal scalp for Leftward gaze, the condition with the most potential social and attentional meaning, had significantly greater amplitudes than all the other conditions.

Our second hypothesis that the Blink condition would elicit reduced N170 amplitudes, compared to the other conditions, was not sustained; robust N170s were elicited by all conditions including Blink. Similarly, no significant differences were seen in N170 latency as a function of Condition. We had previously described significant differences in N170 amplitudes and latencies as a function of eye gaze direction – with larger N170 observed to gaze aversions relative to when eyes look directly at the observer (Puce et al., 2000, 2003). Type of gaze aversion (left, right, upward) does not appear to influence N170 amplitude, irrespective of whether an explicit social context or social judgment is required or not. As expected our tested gaze aversions (Leftward, Upward) also showed no significant differences in N170 amplitude or latency, however, in this study we used transient aversions of around 100 ms in an attempt to move toward more ecologically valid stimulus displays. Here we also tested Eye Closure, which might be regarded as a form of eye aversion, using a similar stimulus duration of 100 ms, in addition to a Blink – a quicker type of eye closure stimulus. In these latter two stimulus conditions, N170 amplitudes were as large as for the gaze aversions, indicating that the brain registers and monitors all types of eye changes. The N170 has been previously suggested to potentially be sensitive to changes in the social attention of another individual (Itier and Batty, 2009), but the results in this study would suggest that N170 might more likely reflect a detector of change in the eyes, or perhaps changes in the face in general (see also Puce et al., 2000).

There was an effect of Hemisphere for the N170, with the right hemisphere showing significantly greater amplitude than the left hemisphere in all four conditions. In our own previous studies, N170s elicited to viewing lateral gaze shifts to both the left and right were of equal magnitude, with a tendency for larger N170s to be seen in the right hemisphere regardless of the gaze shift direction (Puce et al., 2000). The right hemisphere preference for face related stimuli, be they the onset of static faces or dynamic faces in ongoing facial displays, has been observed in studies by other laboratories (Rossion et al., 1999; Itier and Batty, 2009). The existing literature, which is devoted largely to examining N170s elicited by static face onset, shows mixed findings with respect to facial changes related to affect or social significance. For example, some studies showed larger N170 amplitudes to static fearful vs. neutral face onset (Pizzagalli et al., 2002; Eger et al., 2003), while others showed no differences (Eimer and Holmes, 2002).

We also examined the behavior of the P100, an ERP component which is thought to reflect low-level stimulus attributes in faces as well as other stimulus categories (see Rossion and Caharel, 2011). We did not observe any differences in P100 attributes as a function of stimulus Condition. Our data indicate that differentiation of neural activity as a function of taking the eyes away from another only occurs at latencies beyond 250 ms (see below). It is our belief that these earlier ERPs signal the change in the facial display (P100, N170), and potentially socially salient stimuli relative to the observer (N170). This latter signal may encourage further attention to a stimulus for potential social or attentional relevance.

Average amplitudes of late ERP components were greater for Leftward and Upward gaze aversions compared to Blinks and Eye Closure, although clear and discernable late peaks were observed even in these latter two conditions. This was true for P250, P450, and P600, supporting our hypothesis that late potential amplitudes may reflect attentional and/or social significance, as seen in a previous study, where social context was explicitly manipulated (Carrick et al., 2007). Traditionally, later ERPs have been associated with higher-order cognitive and affective processes (Sutton et al., 1967; Kutas et al., 1977; Kutas and Hillyard, 1980; Hillyard and Kutas, 1983; Rosler et al., 1986), and more recently have been described in social cognition and theory of mind tasks (Liu et al., 2004; Sabbagh et al., 2004; Puce et al., 2007). Therefore the later responses observed in this study may indicate that simple eye movements, including blinks, have some social importance to us.

Event-related potential studies using static face onsets suggest that P250 may be related to consolidating configural information of faces (Halit et al., 2000; Milivojevic et al., 2003), while ERPs beyond 350 ms may be related to assessment of stimuli, social cognition, memory, and attentional processes (Cacioppo et al., 1993; Sabbagh et al., 2004). In addition, later ERP amplitudes are known to increase with attentional or emotional salience (Talsma and Woldorff, 2005; Schupp et al., 2007). A recent study from our lab using dynamic facial displays showed differences in later ERPs to averted gaze when embedded in multiface displays where an explicit judgment of social significance is required (Carrick et al., 2007). Even in the present study in which participants did not make explicit social judgments, these changes in later ERPs were robust and reliable. Perhaps this is because, in everyday social interactions, seeing an upward or lateral gaze shift is usually a strong indicator that the attention of the observed person is being directed away from the observer, conceivably toward important external people or stimuli, or as a result of social discomfort (Kleinke, 1986).

It is possible that as we observe the eye movements of another, including blinks, we may activate an embodied response within the oculomotor system. Oscillatory changes for self-blinks in occipital regions at 200 ms and in orbital regions at 400 ms have been documented with MEG (Bardouille et al., 2006). Furthermore, fMRI studies of voluntary self-blinks show activation in visual and orbital frontal cortices (Tsubota et al., 1999; Yoon et al., 2005). For self-blinks, however, these changes might also reflect a transient interruption of input to visual cortex. Activation in different cortical structures may help explain the distributional differences observed in later ERP components across conditions. Examples are the more posteriorly distributed P250 and P600, and more anterior and right lateralized P450 for the Eye Closure vs. Leftward and Upward gaze change conditions. Future studies using neural source modeling might be able to shed some light on these differences.

Given that we appear to be aware of the blink rates of others (Omori and Miyata, 2001; Mann et al., 2002) and that social perception is known to be influenced by blink rates of the observed person, a study in which another's blinking behavior was explicitly associated with particular social attributes might be a way to systematically investigate how late ERPs are modulated by these social attributes. Correlating the degree of higher-order blink processing with blink rate (normal, fast, or slow) and personality assessments would be useful in understanding how non-verbal cues can influence our judgments. A tantalizing possibility for why blinks result in higher-order cognitive processing is that we have learned associations between blink rate and level of interest, arousal level, stress and/or other factors integral to social interactions. These cognitive and affective states can in turn affect dopaminergic systems which have a direct relationship with blink rate (Brozoski et al., 1979; Karson, 1983; Coull et al., 1995; Coull, 1998; Hirokawa et al., 2000; Dreisbach et al., 2005).

Reflexive attentional shifts that result from observed non-predictive (and sometimes anti-predictive) eye gaze changes (Friesen et al., 2004; Schuller and Rossion, 2005) might potentially drive the observed differences in later ERPs in our study. For example, in the current study, if objects or another face or person were located above or to the left of the face, attention may have shifted to these objects/people causing differences in attentional focus, ERPs, and distraction level (Fichtenholtz et al., 2007). Adding affective expressions to the gaze changes would likely create even more complex attentional effects, which might be influenced by such factors if the depicted emotion encourages approach or withdrawal on the part of the observer (Fichtenholtz et al., 2007, 2009).

Our task used an apparent motion task with two-dimensional real human faces and near-realistic timing. However, there can be differences in neural responses to viewing our stimuli vs. observing live interactions (Ponkanen et al., 2008). The higher-order response to blinks observed in our study could conceivably occur as a result of the artificial and isolated nature of our computer-presented stimuli. It is possible that in this relatively impoverished laboratory environment, less important stimuli may be processed to a greater extent than they would normally be in a natural environment. In addition, multiple repetitions of the limited stimulus set may have led to habituation, which could have in turn have led to reduced higher-order processing of all eye movements. However, we believe this to be unlikely, as significant differences between conditions were seen in later ERPs and not the early components.

Our sample size (14 subjects) may have resulted in the absence of significant differences in the N170 amplitudes across conditions, although in our previous studies using this sample size, significant differences in N170 amplitudes were observed when social attention was manipulated.

Lastly, there was a stimulus duration difference between the Blink condition and the other conditions. However, the Eye Closure condition was matched in duration with the Leftward and Upward gaze conditions. Eye Closure did not show significant differences from Blink, and, like the Blink condition, was often significantly smaller than the Leftward gaze condition. However, a shorter gaze change condition may be necessary to fully address this duration difference.