Nineteen healthy participants (ages 19–24, median age 20, 5 men) took part in the study. The main analyses were done in seventeen participants who had sufficient numbers of epochs per condition, as described below. All participants gave written informed consent. The study was approved by Institutional Review Board No.2 (Research Ethics Committee) of RIKEN Brain Science Institute (Wako-shi, Japan) where we conducted the experiment.

We used 48 pairs of color photo images of real-world scenes from the study by Rensink et al. (1997). The images were 28° wide and 22° high. There were three types of differences between images in a pair: color, position, or presence/absence of an object. Color difference involved either an object or part of the background. Position difference referred to displacement of an object by several degrees of visual angle. Presence/absence involved the occurrence or non-occurrence of an object in the display. Stimuli were presented using custom-made software written in Python using Vision Egg interface (Straw, 2008).

Stimuli were presented on a 21-in. CRT Gateway monitor placed at 85 cm from the participant in a dimly lit room. In a practice session participants were familiarized with examples of three types of difference between images. In the main session, after stable fixation on a central crossways reached, the first image of a pair, the memorization display, was presented for 20 s. Then, after a 1-s mask, the second image, the search display, was presented until response but no longer than 20 s. We asked participants to memorize the first image and detect a change in the second image. Participants had to respond with two mouse clicks: first, as soon as a change was detected, second, after placing the cursor on the change region. Feedback was immediately given by showing the image with a red ellipse over the changed region. The order of images was counterbalanced across participants, such that half of the time each image was used as a memorization or search display.

Eye movements were recorded with a video-based infrared eye-tracking system (EyeLink 1000/Tower, SR Research Ltd., Ontario, Canada). The participant's head was stabilized using a chin and forehead rest. The right eye was tracked with a sampling rate of 500 Hz. The eye-tracker was calibrated using nine points: in the center, four corners and mid-points of the four sides of the screen. Mean difference between calibration and validation measurement was kept below 1.5°.

EEG was recorded with a Nihon Kohden MEG-6116 amplifier using an ECI electrode cap (Electro-Cap International, Inc., Eaton, USA) with 15 electrodes (F3, F4, C3, C4, P3, P4, O1, O2, T3, T4, T5, T6, Fz, Cz, Pz) placed according to the international 10/20 system with electrode FCz as ground and the linked mastoids as reference. Data were recorded with 0.5 Hz high-pass and 100 Hz low-pass online filters (and a 50 Hz notch filter) and digitized at 500 Hz.

The analog output of the EyeLink system was connected to the EEG amplifier and the eye movement horizontal (X) coordinate was recorded as an additional channel (the EyeLink channel) along with the regular EEG channels. To mark the eye movement events in the EEG data, we translated the time stamps of the events from the EyeLink data file. To that end, we correlated the EyeLink channel (recorded by the EEG acquisition computer) and the X coordinate time-series from the EyeLink data file (recorded by the eye movement acquisition computer). In each trial (which consisted of a presentation of two displays, a memorization and a search display), we selected a blink-free segment and determined the lag of maximum correlation between the EyeLink channel and the time-series from the EyeLink data file using normalized cross-correlation. The correlation coefficient for synchronized segments was never below 0.98. Finally, the time stamps of the eye movement events (computed by the EyeLink system) were adjusted for this lag and inserted in the EEG data as markers of saccades and fixations.

We analyzed eye movements and EEG from the 20-s presentations of the memorization display.

Onset time, location, and duration of saccades and fixations were determined by EyeLink software. In our eye movement and EEG analyses we defined a fixation-related epoch by the combination of parameters of a fixation and a following saccade: the fixation has to be longer than 200 ms and shorter than 2000 ms, and the following saccade has to be shorter than 60 ms. Presaccadic intervals containing blinks were excluded.

For each image and each individual participant, heat maps were computed as a function of fixation duration and fixation density (Figure 1). All fixations collected during presentation of a memorization display were accumulated to compute a heat map using visual kernel density estimation (Jones et al., 1996). Contribution of each fixation was represented by a Gaussian kernel with unit amplitude and spread proportional to the fixation duration ti(X,Y)=exp  [−0.5((X−xi)2+(Y−yi)2σ2) ] where t_i is the temperature contribution of the ith fixation, (X, Y) is the pixel coordinate within the stimulus image, (x_i, y_i) is the center of ith fixation, and spread (σ = τ_i/k) is proportional to the duration (τ_i) of the fixation. The linear bandwidth parameter k = 20 was determined visually to obtain a smooth unimodal density function above each cluster of fixations. A larger bandwidth covers a wider area and reduces the relative advantage of pixels in near proximity of the center of the fixation. Longer fixations allow more time and space for exploration of the surroundings of the center of fixation using micro-saccades. The final heat map was the average of all contributions T = sum_it_i.

We superimposed for each saccade the start and end points on the images (Figure 1) and extracted their temperatures as the average of all temperatures located within a radius of 0.5° of visual angle. The radius was chosen in accordance with the measurement error in the eye-tracking equipment.

High temperature on the heat maps indicates areas with long fixation durations and small saccade sizes, i.e., regions which were carefully scrutinized using a local scanning strategy. Low temperature areas, by contrast, indicate regions that were only occasionally visited during global scanning or not visited at all.

As an indicator of target selection for the next fixation we used the difference in temperature between the start and end point of a saccade. The difference can be understood as a contrast in anticipated task-relevant information between locations. We considered a saccade as going in the positive direction if the temperature at the end point was higher than at the start point, and vice versa if the saccade direction was negative.

Since we proposed that the corresponding processes depend on viewing strategies as reflected in saccade size, we divided up the data into three bins according to saccade size: short, medium and long. Three bins were used in order to secure a minimal number of epochs per condition (which was set to 50) for a sufficiently large number of participants. We still had to exclude two participants who did not have enough epochs in one of the conditions. Thus, eye movement and EEG analyses were performed in 17 participants.

For the EEG analysis we used Brain Vision Analyzer software (Brain Products GmbH, Gilching, Germany). The EEG signal was filtered with a Butterworth zero-phase filter with a high cut-off frequency of 30 Hz, 12 dB/oct. From the 15 recorded channels, two temporal channels (T3 and T4) were excluded because of artifacts resulting from muscle activity.

We analyzed EEG in the fixation intervals preceding saccade onsets. We used only those EEG epochs that corresponded to fixation intervals without blinks. These epochs are by definition free of artifacts caused by eyelid and eyeball movements (Dimigen et al., 2011). However, if a blink had occurred in the preceding fixation interval or if the fixation was short, the tail of the activity evoked by the blink or the saccade would still have had a chance to contaminate the current fixation interval. We used ICA to remove these effects from the EEG data (Jung et al., 2000) as follows. We selected a 300-s interval from 100 to 400 s of continuous EEG recording, which included a number of blinks and saccades. The data from 13 regular EEG channels in this interval constituted the training dataset for computing the unmixing matrix. Then in each participant we identified the ICA components which picked up eye movement artifacts. The time course of these components mirrored blinks or saccades in the “EyeLink channel” recorded together with EEG channels. These components had typical topography with a maximum at the frontal (F3, Fz, F4) sites: the blink-related component had a symmetrical maximum at the frontal electrodes; the saccade-related component had a characteristic asymmetrical topography over the frontal sites reflecting the directionality of the saccades. Finally, the whole duration of the EEG was reconstructed without these components. As expected, ICA artifact correction primarily affected the activity at the frontal sites. Since the unmixing matrix was computed using an EEG interval encompassing all experimental conditions, it is unlikely that ICA selectively or systematically altered the presaccadic activity in a certain condition.

The markers of saccade onset were computed by EyeLink software and were incorporated into the EEG time series. EEG was segmented into epochs from −200 ms to 50 ms relative to saccade onset. Using a semi-automatic artifact rejection procedure, we excluded epochs if the absolute voltage difference exceeded 50 μV between two neighboring sampling points and if the amplitude exceeded +100 or −100 μV. After the artifact rejection the mean number of epochs per condition was 158 (SD: 36; range 109–220).

For each participant, we set the number of epochs to be equal in all conditions. To that end, we first identified the condition where the number of epochs was minimal. Then for all other conditions we randomly selected a number of epochs equal to this minimum.

Based on visual inspection of the grand averaged potentials (Figure 4A), we selected the interval −100 to 20 ms before the saccade onset for statistical evaluation of the presaccadic activity. We computed the mean amplitude of this interval. In addition, we evaluated the peak-to-peak amplitude of the saccadic spike potential by measuring the difference between the positive and negative peaks of this potential.

We averaged the epochs for each participant and condition separately. As we discussed in the Introduction, one of our goals was to distinguish between activities over the frontal and parietal brain areas. Therefore we preselected anterior (F3, F4, Fz, C3, C4) and posterior (P3, P4, Pz, O1, O2) groups of electrodes and averaged the potentials across electrodes within each group.

For the baseline correction we selected a 20-ms interval in the beginning of the presaccadic epoch (i.e., −200 to 180 ms before the saccade onset).

We considered three main factors: Saccade size (short, medium, long), Correctness (Correct detection vs. failure) and Saccade direction (positive vs. negative). For the analysis of the eye movement measures, unless otherwise stated, we used univariate repeated-measures ANOVA with these factors. As for the event-related potentials (ERP) analysis, the effect of scalp locations on amplitude differences is not an additive effect but a multiplicative one; therefore additive ANOVA models cannot unambiguously evaluate topographic differences (McCarthy and Wood, 1985). For this reason, we treated amplitude in the anterior and posterior electrode groups as two dependent variables in a multivariate design (MANOVA). Whenever MANOVA revealed a significant effect, we proceeded to univariate ANOVA follow-up analyses and post-hoc tests, in order to identify the specific dependent variables that contributed to the effect. In the univariate ANOVA we applied the Huynh-Feldt correction (ε) of p-values associated with more than two degrees of freedom, in order to compensate for violation of sphericity. For post-hoc analyses we used Fisher's LSD (Least Significant Difference) test.

For presaccadic activity, the correlation between anterior and posterior signals was r₍₁₇₎ = 0.35. This is in accordance with the MANOVA requirement that the dependent variables should be moderately correlated with each other (i.e., 0.20–0.60; Meyers et al., 2006). The mean peak-to-peak amplitude of the saccadic spike potential, however, highly correlated between the anterior and posterior electrode groups [r₍₁₇₎ = 0.97]. We therefore ran two separate ANOVAs on the anterior and posterior saccadic spike potentials.

First, we tested for changes in viewing strategy. We divided 20-s memorization intervals into five 4-s time bins. In each bin we computed the saccade duration for correct detection and failure. We found that saccade duration decreased after the first bin and then remained unchanged (Figure 2A). A repeated-measures ANOVA with factors of Time Bins (5 levels) and Correctness (correct detection vs. failure) revealed an effect of Time Bins [F_{(4, 64)} = 8.9, p < 0.001, Huynh-Feldt ε = 0.82] and no correctness effect nor interaction. Post-hoc test showed that the effect of Time Bins occurred because of longer saccades in the first bin than all other ones (all p < 0.001). This finding can be understood as a shift in emphasis from global to local viewing strategy in the course of free viewing, consistently with previous reports (Unema et al., 2005; Pannasch et al., 2008; Graupner et al., 2011).

The shift in viewing strategy may affect the processes of visual encoding and fixation target selection. Since we determined strategy according to saccade size (as reflected in saccade duration), we divided up the data into three bins accordingly: short, medium and long. Figure 2B illustrates the ranges of saccade durations after binning. A repeated-measures ANOVA with factors of Saccade size (short, medium, long), Saccade direction (positive vs. negative), Correctness (correct detection vs. failure) revealed that the saccade durations did not differ between negative and positive saccade directions [F_{(1, 16)} = 0.09, p = 0.77] and between correct detection and failure [F_{(1, 16)} = 1.7, p = 0.22]. This suggests that the oculomotor component of saccade preparation was similar for all conditions within a saccade size bin.

For the three saccade sizes, we considered the duration of their preceding fixations (Figure 3A). Saccade size conditions showed a prominent effect on preceding fixation duration [F_{(2, 32)} = 10.7, p < 0.001, Huynh-Feldt ε = 1.0] and there was an interaction between Saccade size and Saccade direction [F_{(2, 32)} = 37.1, p < 0.001, Huynh-Feldt ε = 1.0]. These effects indicate shorter fixation durations for positive than for negative direction for medium (p < 0.001) and long (p < 0.001) saccades. Correctness did not yield an effect [F_{(1, 16)} = 0.4, p = 0.52]; no further interactions were found.

In contrast to fixation duration, the temperature difference between fixation locations appears to be sensitive to correctness of change detection (Figure 3B). The absolute values of the temperature difference was higher in correct detection than in failure [F_{(1, 16)} = 5.7, p = 0.03].

In our study we used the temperature difference as an indicator of fixation target selection since it represents the contrast of task-relevant information between two fixation locations. The absolute temperature difference was larger for positive than negative direction [F_{(1, 16)} = 33.7, p < 0.001]. In order to evaluate how the target selection depends on saccade sizes we compare the absolute temperature differences for short, medium and long saccades. After a short saccade the gaze is likely to land into an image region with similar temperature, whereas after a long saccade the gaze may land into a region with a different temperature. As expected, we found a prominent increase of the temperature difference with saccade size [F_{(2, 32)} = 93, p < 0.001, Huynh-Feldt ε = 1.0].

In addition, the processes of target selection are different for the short and long saccades. We observed an interaction between Saccade size and Saccade direction: F_{(2, 32)} = 4.98, p = 0.03, Huynh-Feldt ε = 0.70) which was occurred because of the larger temperature difference in the negative than positive direction for the long (p < 0.001) and medium (p < 0.001) saccades, but not for short (p = 0.57) ones (Figure 3C).

Figure 4A shows the scalp topography of the spike potentials averaged over 17 participants relative to saccade onset for three saccade sizes.

What are the factors most likely affecting the amplitude of the presaccadic activity?

The presaccadic interval includes both a shift in covert attention to a novel target and motor preparation for a saccade to that target. Since we did not observe any effects on saccade size (Figure 2B) and the amplitude of the saccadic spike potential (Figure 4B) of our conditions, the oculomotor components of the saccades are equal in strength within each size bin. Therefore the amplitude of the presaccadic activity is likely to reflect a covert shift of attention. An attentional explanation of the presaccadic activity is consistent with previous interpretations of the scalp-recorded presaccadic potentials (Wauschkuhn et al., 1998; Gutteling et al., 2010; Krebs et al., 2012).

Furthermore, the amplitude of the antecedent potential indicates trans-saccadic remapping (Parks and Corballis, 2008). This does not contradict the attentional explanation above since only attended visual features are remapped (reviewed in Mathot and Theeuwes, 2011). In our study the amplitude of the presaccadic activity was larger for saccades in the positive than in the negative direction (Figure 6B). We defined the positive saccade direction, according to fixation heat maps, by higher temperature at the end than start point of a saccade. The temperature difference was higher for saccades in the negative than in the positive direction (Figure 3C), because the preceding fixation was shorter for saccades in the positive than in the negative direction (Figure 3A). The fixation following was, correspondingly, longer for saccades in the positive than in the negative direction. Since fixation duration indicates amount of attention(Henderson, 2007), the presaccadic activity reflects attention deployed to the following fixation location. Thus, the larger amplitude of presaccadic activity for saccades in the positive than in the negative direction may reflect remapping of attended information across a saccade.

It is known that visual information collected during the last 100 ms before a saccade (so called “saccadic dead time”) does not influence the destination of the current saccade (Becker, 1991), but influences the following saccade (Caspi et al., 2004). This suggests functional dissociation of two processes occurring in this interval: accumulation of information for later use and remapping of previously collected information across a saccade to maintain visual stability (Mathot and Theeuwes, 2011). We propose that both processes are reflected in presaccadic activity, taking place in parallel.

Factors of overall alertness or enhanced voluntary processing are less likely to affect selectively the presaccadic activity, because systematic amplitude changes were revealed for saccades of various sizes and directions occurring within the same memorization display, i.e., short enough to render unlikely any systematic changes of alertness or volition.

On the fixation heat maps, saccades followed by correct change detection had larger temperature differences (in absolute values) than ones followed by detection failure (Figure 3B). Since temperature difference reflects the informational contrast between two fixation locations, we conclude that successful encoding depends on entering scene regions that are expected to be task-relevant and containing a high concentration of information.

Remarkably, even though the computation of temperature considers fixation durations, these are not sensitive to correctness (Figure 3A). Since temperature also considers fixation density (the spatial distribution of the fixation locations), this must be the variable most directly relevant for correct detection. This conclusion is consistent with previous observations: the number of regional fixations rather than their duration matters for memorization of localized information (Loftus, 1972). Consequently, the heat maps differ for correct detection and failure: in correct detection the information is concentrated in widespread clusters, whereas in failure it is more randomly scattered. Thus, scanning strategy in successful encoding consists of thorough scrutiny of scene regions which expected to be task-relevant.

An effect of correctness on the amplitude of presaccadic activity was found only over anterior brain areas. This effect depended on saccade size and appeared to be associated with short saccades only (Figure 5B). Short saccades mainly occur towards the end of exploring a display (Figure 2A), when potential targets have already been localized and the visual strategy changes to scrutinizing the local regions (Unema et al., 2005; Tatler and Vincent, 2008; Graupner et al., 2011). The amplitude of presaccadic activity was larger in correct detection than in failure (Figure 5A). Since we attributed the presaccadic activity to attention being deployed to the fixation location following, this finding implies that attention is needed for successful encoding. Specifically, successful encoding may depend on scrutiny of the local regions guided by top-down attention rather than global visual exploration of a scene¹.

The absolute values of the temperature difference between start and end points of saccade increase with saccade size (Figures 3B,C). Small differences are associated with short saccades because after a small shift in fixation the gaze is likely to land into an image region with similar temperature, whereas after a large shift the gaze may land into a region with a distinct temperature.

The absolute temperature difference was larger for the saccades in the negative than in the positive direction; however, this happened for the medium and long, but not for the short saccades (Figure 3C). Since temperatures depend on fixation duration, this is a consequence of the shorter duration of fixations preceding the medium and long saccades in the positive than in the negative directions (Figure 3A). Temperature indicates how attractive a certain region is as a fixation target. The effect, therefore, may indicate facilitation of fixation target selection if the next fixation target is in an attractive location.

Selection of a saccade target seems to be guided by attractiveness of the next target location only for the medium and long (>30 ms) saccades (Figure 3C). This effect may occur because of the predominant role of bottom-up guidance in long saccades. During initial exploration, visual salience determines the attractiveness of scene regions. The initial exploration is accomplished with long saccades (Figure 2A) (Unema et al., 2005; Pannasch et al., 2008; Graupner et al., 2011). The bottom-up character of the initial exploration offers an explanation for why we did not observe the difference in the presaccadic activity for long saccades (Figure 6B).

Once potential targets have been localized with long saccades, the visual strategy changes to scrutiny of the local regions (Unema et al., 2005; Graupner et al., 2011). This change in strategy is accompanied by a shift from bottom-up to top-down saccade guidance (Findlay and Walker, 1999). The local scanning strategy involves short saccades (Unema et al., 2005; Tatler and Vincent, 2008; Graupner et al., 2011). For the short saccades the informational contrast between current and next target locations is low and, correspondingly, the temperature difference for the positive and negative saccade direction is about equal (Figure 3C). Thus, local scanning is not guided by salient informational content at the next fixation locations. Instead, it may be guided by a top-down mechanism directing the saccades to scene regions which are expected to be task-relevant. This is reflected in the larger presaccadic activity in the positive than in the negative saccade direction over posterior areas (Figure 6B).

Overall, our findings support the notion of different selection mechanisms for short and long saccades, in line with previous observations (Tatler et al., 2006; Foulsham and Kingstone, 2012).

The medium saccades may take up an intermediate position and are guided by combination of bottom-up vs. top-down processes. This is reflected in the difference between positive and negative saccade directions observed in both eye movement and EEG measures (Figures 3C, 6B).

In sum, scalp-recorded electrical brain activity in the presaccadic interval reflects processes related to trans-saccadic perception. We provide evidence for the sensitivity of the presaccadic activity to encoding of visual information and to selection of a target for the next fixation. In these processes the presaccadic activity reflects systematic tendencies in oculomotor behavior which differ in attentional demand.

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.