In the partner-oriented state person A focuses her attention on person B. The eyes automatically attract visual attention (Laidlaw et al., 2012) and possibly convey information about personal attributes including gender, age and identity (Schyns et al., 2002), as well as emotional and attentional states (Baron-Cohen et al., 1997; Emery, 2000).

Eyes that focus on the viewer will be preferentially looked at (Senju and Hasegawa, 2005) or evaluated much more positively (Stass and Willis, 1967), modulate attention (Senju and Hasegawa, 2005; Dalmaso et al., 2017), increase emotional empathy (Schulte-Rüther et al., 2007) and modulate cognition suggesting a substantial ‘eye-contact-effect’ for diverse aspects of socio-emotional perception (Senju and Johnson, 2009). Among distractor stimuli, viewer-directed gaze is detected easily and much faster than averted gaze (von Grünau and Anston, 1995; Conty et al., 2006; Senju et al., 2008). Profound effects of viewer-direct eye gaze on preference (Hains and Muir, 1996) and attentional modulation (Farroni et al., 2002) have also been demonstrated in infants. This is probably the most thoroughly studied gaze state.

The effect of eye contact is much stronger during dynamic interactions with real persons than when confronted with static pictures (Hietanen et al., 2008; Pönkänen et al., 2011). This requires interactive approaches with dynamic face-to-face interactions (Pfeiffer et al., 2012, 2013a; Risko et al., 2012, 2016; Schilbach et al., 2013; Schilbach, 2014; Oberwelland et al., 2016).

In the object-oriented state person A’s attention is focused more or less entirely on an object in the shared environment, but not on the other person (as opposed to joint attention states described below during which person A oscillates between objects and person B). That is B’s presence and behavior are likely to influence A to some level but merely coincidentally and probably without A’s awareness. The exploration of different objects in a visual scenery is affected by the saliency of objects and thus the probability of persons directing their attention toward the objects (Itti and Koch, 2000). However, top-down as well as bottom-up processes are actively working together or compete for attention (Egeth and Yantis, 1997). Again, our attention and behavior toward objects are altered by actions or even the mere presence of another person looking at us (Senju and Hasegawa, 2005). Gaze cueing can automatically lead the attention toward particular objects (Frischen et al., 2007), even overriding the effect of higher psychophysical saliency (Borji et al., 2014). This brief instance of social interaction might induce a lasting attentional shift from a state of OO to the state of RJA [as examined in section Responding Joint Attention (RJA)]. However, even in the absence of any active gaze cuing, the presence of another person can attract covert attention (Kuhn et al., 2016; Laidlaw et al., 2016). Furthermore, the mere knowledge of the possibility of someone else watching their gaze lets participants control their gaze behavior with respect to its social adequacy (Risko and Kingstone, 2011).

In this state person A neither focuses on objects nor on persons in the environment but only on his inner experience. Attentional disengagement from the outside world has been shown to correlate with a decrease in saccade frequency and an increase in saccade amplitude (Benedek et al., 2017) and, accordingly, a decrease in fixation frequency and an increase in fixation duration (Reichle et al., 2010; Benedek et al., 2017). Furthermore, in these situations blinking rate can increase (Smilek et al., 2010) and blinking duration can be prolonged (Salvi et al., 2015; Benedek et al., 2017). INT seems to show more variability in pupil diameter than episodes of directed attention to outward stimuli (Smallwood et al., 2011; Benedek et al., 2017). A higher variability of eye vergence (Benedek et al., 2017) suggests a less focused gaze (Solé Puig et al., 2013).

While it is intuitively obvious that these changes are indicative of a reduced responsiveness to events in the outside world (Smallwood et al., 2011; Benedek et al., 2017), it is an open question whether the reduced responsiveness to external stimuli and the overall change in gaze behavior are both the result and an epiphenomenon of INT, or whether changes such as a decrease in the frequency of microsaccades during INT may represent active visual disengagement as a strategy to achieve reduced responsiveness (Benedek et al., 2017). Another strategy participants adopt in situations of high cognitive load is to avoid looking into the eyes of an observer because this would entail higher demands on cognitive processing (Glenberg et al., 1998; Doherty-Sneddon and Phelps, 2005; Phelps et al., 2006; Markson and Paterson, 2009). Interestingly, the additional cognitive demands of mutual gaze do not seem to originate in the physical properties of the stimulus (e.g., the eyes) but in the interactive character inherent in this situation (Markson and Paterson, 2009). It is therefore crucial to consider introspective attentional states as potentially socially influenced by the presence of another person.

In the responding JA state person A waits for B to initiate and lead the interaction, e.g., B chooses an object and A follows B’s gaze toward the object. Gaze following reactions that respond to the invitation of another person thereby establishing a rudimentary form of JA appear to be deeply rooted in human behavior (Pfeiffer et al., 2011). The gaze of another person automatically cues one’s own attention even when it is uninformative (Friesen and Kingstone, 1998), and participants exhibit gaze following even for forthright counter-predictive gaze cues (Driver et al., 1999; Bayliss and Tipper, 2006).

Gaze following with the aim of establishing JA constitutes a very simple though effective mechanism allowing for the inference of the attentional focus of other persons. The ability to adopt the attentional focus of another person is a prerequisite for reinforcement learning, from infants to adults (Vernetti et al., 2017). Infants at 6 months of age are already able to follow the eyes of other persons, in particular in a communicative context (Senju and Csibra, 2008). Accordingly, early proficiency in gaze following in infants predicts the development of mentalizing and emergence of language (Morales et al., 1998; Charman et al., 2000). JA and gaze following facilitate social learning, social competence, self-regulation, intelligence, and depth of information processing (Mundy and Newell, 2007).

In this state, person A takes the lead within the interaction by initiating JA. While gaze following in RJA reflects person A’s understanding that B’s perception and actions are goal-directed or have communicative intent, the initiation of JA is considered to require elaborate processing and insight (Tomasello and Carpenter, 2005). To initiate JA, A has to acknowledge (1) the dual function of social gaze (Gobel et al., 2015; Jarick and Kingstone, 2015) i.e., that gaze does not only serves her in perceiving but also that her gaze informs B about her focus of attention and, (2) sharing of attention is a desirable aim for mutual interaction (Tomasello et al., 2005). Whereas first elements of RJA are already evident at 6 months of age, IJA does not emerge before the second year of life (Mundy and Newell, 2007; Mundy et al., 2007). Chimpanzees followed the experimenters gaze on a frequent basis but did not try to initiate JA (Tomasello and Carpenter, 2005). Interestingly, differential development of both RJA and IJA can be observed in brain systems from childhood to adulthood (Oberwelland et al., 2016), as well as during atypical development in disorders such as autism (Oberwelland et al., 2017). In autism, IJA is typically more impaired than RJA and emerges much later than in typical development (Mundy, 2003). These empirical findings clearly point toward separate underlying cognitive systems of RJA and IJA (Mundy and Newell, 2007).

The innate tendency to expect other humans to follow their gaze (Pfeiffer et al., 2011) corresponds to the perception of successful initiation of JA as rewarding (Schilbach et al., 2010; Pfeiffer et al., 2014; Oberwelland et al., 2016). A successfully initiated instance of JA alters the consecutive interaction by increasing the tendency to look at and dwell upon the partners face (Bayliss et al., 2013).

Having defined the basic states during triadic JA, the picture becomes more complex when considering that each of the two participants can adhere to any of these states during a triadic interaction unfolding in time. In theory, a dual social state may be one of 25 possible combinations (representing varying degrees of “interactivity”), spanning a two dimensional SGS (Figure 2; see McCall and Singer, 2015 for an alternative concept of a 2D gaze space). Some of these combinations might be more ephemeral than others: e.g., a person A might soon lose the motivation to initiate JA if person B does not respond to him adequately, person A might switch to PO very soon subsequently (‘stability’ of states is indicated by cell color in Figure 2, with gray cells indicating unstable and ephemeral states; red arrows represent subsequent shifts from unstable to stable states).

Furthermore, it is conceivable that mutual attention (PO/PO) might facilitate transitions from non-interactive to interactive states (indicated by green arrows in Figure 2). These transitions have yet to be empirically investigated. Only non-interactive states (blue box in Figure 2) can be understood on the basis of single persons whereas the study of interactive situations (purple box in Figure 2) requires a complex dynamic concept and experimental setup, based on the idea that the basic unit of analysis is the interaction between both interactants.