Introduction
Traditionally, there have been two main views on how humans recognize faces: the analytical view and the holistic/configural view (Ellis, 1975; Sergent, 1986). According to the analytical view (e.g., Haig, 1984; Gosselin and Schyns, 2001; Sadr et al., 2003), observers explore a face by scanning local features in order to extract the most diagnostic information to individualize the face. According to the holistic/configural view, the features are not perceived and represented independently of each other. Instead the face is perceived as an integrated whole (Sergent, 1986; Tanaka and Farah, 1993; Farah et al., 1998; Maurer et al., 2002; Rossion, 2008). The holistic/configural view has been mainly supported by behavioral studies showing that the perception of a given facial feature (e.g., a local element such as an eye, a distance between two elements, or even half of a face) is influenced by the position and the identity of other facial features (Sergent, 1986; Young et al., 1987; Tanaka and Farah, 1993; Farah et al., 1998).
Holistic/configural processing matures early on (e.g., Tanaka et al., 1998; de Heering et al., 2007) and its integrity depends on early visual experience (e.g., Maurer et al., 2007). This is illustrated by studies performed with patients who missed early visual inputs because a dense and opaque cataract blocked any patterned visual input to reach the retina at birth, until the day of surgery. Specifically, when tested at adulthood, these cataract-reversal patients perform worse than age-matched controls when they have to discriminate faces that differ in term of the relative distance between the features (Le Grand et al., 2001). Contrary to age-matched controls, their matching of half of a face is also not influenced by the identity of the other half (absence of composite face effect; Le Grand et al., 2004).
An unresolved issue is whether the absence of early inputs in another modality than vision, audition for instance, can modulate holistic/configural processing. There are a few reasons why this could be the case. First, it has been suggested, in agreement with some experiments performed on animals, that enhanced visual performance is found in specific regions of the auditory cortex of congenital deaf cats (e.g., Lomber et al., 2010; for a review, see Rauschecker, 1995). The same phenomenon has also been observed in humans (for a review, see Bavelier and Neville, 2002). More specifically, the absence of audition, sometimes together with the use of sign language, has been identified as a potential factor favoring intermediate and high-level vision: deaf are better than hearing controls to perform mental rotation (Emmorey et al., 1993) and gestalt completion (Siple et al., 1978), as well as to detect (Loke and Song, 1991; Stivalet et al., 1998; Armstrong et al., 2002) and discriminate (Neville and Lawson, 1987) moving stimuli appearing in periphery but not in the center of the visual field (Bavelier et al., 2000, 2006; Bosworth and Dobkins, 2002; Buckley et al., 2010; although, see Hauser et al., 2007).
Regarding face recognition abilities, deaf individuals’ superiority over hearing, as well as the exact role of early exposure to sign language is still debated. Using a card memory game, Arnold and Murray (1998) found that deaf signers were better than hearings signers to match faces, but not objects. In turn, hearing signers performed better than hearing non-signers. Given these results, the authors attributed deaf superiority to their use of sign language and raised the appealing possibility that deafness and the long use of sign language might have additive effects. However these authors did not control for the age of sign language acquisition, which makes strong conclusions about the importance of signing difficult to make. Later on, Arnold and Mills (2001) compared the performance of a group of deaf signers, hearing signers, and hearing non-signers in a task where they had to memorize the location of objects, faces, and shoes. They found deaf signers to perform like hearing signers, both of whom were better than hearing non-signers on the face and shoe task. Along the same line, Bettger et al. (1997) reported that deaf individuals using American signed language (ASL) performed significantly better than hearing non-signers at discriminating face photographs presented under different views and lighting. In another experiment, the same authors showed that hearing signers born from deaf parents also show better results than hearing non-signers in this task, suggesting indirectly that the enhanced performance of deaf signers is linked to their experience with ASL rather than to their auditory deprivation. Interestingly deaf signers’ expertise with upright faces does not extend to inverted faces (Bettger et al., 1997), Mooney faces or the recognition of the faces of the Warrington test (1984) in which participants have to recognize previously memorized faces (McCullough and Emmorey, 1997).
Although McCullough and Emmorey’ (1997) study shows a superiority of deaf people to detect subtle manipulations introduced at the level of the mouth, the studies described above focus more on the global level of performance of this population with faces than on how they process faces. Here we addressed this issue by testing a group of deaf participants and a group of hearing participants in an inversion-matching task (Experiment 1) and a composite face task (Experiment 2). In Experiment 1, we used picture-plane inversion as a manipulation because it disrupts the ability to process faces to a greater extent than what is observed for non-face stimuli (Yin, 1969). Specifically, it has been suggested that whereas upright faces are encoded as integrated wholes, inverted faces are rather processed feature-by-feature, in a piecemeal manner (e.g., Yin, 1969; Sergent, 1984; Farah et al., 1995; Maurer et al., 2002; Rossion, 2008, 2009). Here, we hypothesized that, independently of whether the stimuli are presented simultaneously on the screen (Experiment 1A) or with a delay between the target and the probes (Experiment 1B), if deaf participants focus more than hearing participants on the details of the faces such as the mouth for example, as previously suggested by McCullough and Emmorey (1997), they could show a reduced or abolished inversion effect. Indeed it has been shown that individuals with acquired prosopagnosia, who rely more on some facial features than controls without being able to integrate them into a coherent template, show an abolished face inversion effect (see Busigny and Rossion, 2010 for a recent review). Alternatively, if deaf individuals integrate facial features to a greater extent than hearing controls because they are used to process a largely distributed range of visual information (by simultaneously processing the mouth and the eyes to understand the syntactic structure of a sentence, for example), they could show an equally large or even enhanced face inversion effect compared to hearing controls. In Experiment 2, we used the composite face effect originally reported by Young et al. (1987). In the context of a matching task with unfamiliar faces (Hole, 1994), it refers to the observation that two identical top parts of a face are perceived as slightly different if their respective bottom parts belong to different facial identities. This perceptual illusion is abolished or strongly reduced if the top and the bottom parts of the face are laterally offset. Here we used the paradigm as it is generally used, asking participants to pay attention to the top parts of the faces while their bottom halves are different (Experiment 2A). We expected hearing participants to show a strong composite effect on accuracy and/or correct response times (for empirical demonstrations, see for example, Hole, 1994; Le Grand et al., 2004; Goffaux and Rossion, 2006; Michel et al., 2006; Rossion and Boremanse, 2008). Consistently with the predictions of Experiment 1, we also hypothesized that deaf individuals would show a larger composite effect than hearing controls in case of enhanced holistic/configural face processing. Conversely, their composite effect would be reduced if they rely more on facial features than hearing controls. Participants were also asked to judge the bottom parts of another set of composite faces (Experiment 2B). As previously shown (Young et al., 1987; Ramon et al., 2010), we expected hearing participants to show a smaller composite effect than in Experiment 2A. We also hypothesized that deaf participants would be less affected than hearing participants in this part of the experiment because of their everyday use of lip-reading.
General Discussion
The aim of this study was to examine how early deaf participants process socially relevant visual stimuli such as faces given the evidence that reorganization of the cortical functions is observed in humans in case of sensory deprivation such as deafness (for a review, see Bavelier and Neville, 2002). To our knowledge McCullough and Emmorey (1997) were the only authors who investigated this topic. They made the observation that deaf individuals were significantly better than controls to detect featural manipulations introduced at the level of the mouth. To date no study had focused on deaf ability to process faces holistically/configurally despite that this type of processing is thought to be at the heart of hearing individuals’ expertise with faces (e.g., Farah et al., 1998; Maurer et al., 2002; Van Belle et al., 2010a). Specifically we conducted two experiments to assess whether deaf individuals would show enhanced or reduced holistic/configural face processing compared to hearing controls.
The results of the inversion-matching task (Experiment 1) were threefold. First, deaf participants were slightly better than hearing participants, but also generally slower, to match visual stimuli such as cars and faces except when they were simultaneously presented on the screen. They also showed an enhanced face inversion effect in response times compared to hearing participants, both during the simultaneous and the delayed matching of faces. Finally, they took longer than controls to process inverted faces, suggesting that they were more dependent on holistic/configural processing than non-deprived observers. In line with the perceptual field hypothesis of the face inversion effect (Rossion, 2008, 2009) suggesting that the inversion of a face induces a reduction of the size of the perceptual field to a single local feature, we would hypothesize that hearing participants were able to directly focus on the most diagnostic feature of the face to match inverted faces, namely the eyes. Conversely, we think that deaf participants needed more time to do the same because their representation of a face is probably not as biased towards the diagnostic eye region as it is for hearing participants. In other words, we believe that deaf face individualization capabilities rely on cues that are more evenly distributed on the superior and inferior parts of the face than what is observed in hearing individuals due to long-term experience with lip-reading and in discriminating grammatical facial expression used with sign language (McCullough and Emmorey, 1997; Letourneau and Mitchell, 2011; but, see Watanabe et al., 2011 for different results on Japanese deaf participants).
The results of the composite task (Experiment 2) indicated a composite face effect of the same magnitude in deaf and hearing participants, independently of whether the locus of attention was the top or the bottom of the face. Both populations could therefore rely on a holistic template to simultaneously extract information from the whole face configuration. This observation is compatible with Experiment 1 because the inversion-matching paradigm and the composite face paradigm do not measure the exact same thing. Specifically the inversion paradigm is a measure of participants’ dependency on holistic/configural processing. That is, the more a participant needs holistic/configural face processing to match inverted faces, the more his/her performance drops because of his/her inability to recruit this kind of processing. In contrast, the composite face paradigm provides an index of how strongly the different parts of the face are integrated into a holistic representation when one part of the face (e.g., the bottom of the face) does not provide additional diagnostic information and is only there to interfere.
In sum, the current study suggest that early deafness does not enhance or reduce the amount of holistic/configural processing devoted to faces but rather increases the dependency on this mode of processing. Future studies recording eye–gaze fixations during upright and inverted face individualization could help clarifying this issue. For example, gaze-contingency stimulation (Van Belle et al., 2010a,b) could be used to test a group of early deaf individuals as well as a group of hearing participants with faces. If, as predicted by McCullough and Emmorey (1997), the absence of one sensory modality such as audition leads to the enhancement of the visual representation of the mouth in an upright face, then deaf individuals should be less impaired than hearing individuals in a condition where a gaze-contingent window only reveals this internal feature (central window condition). The same experiment with inverted faces would confirm or infirm that deaf individuals’ first fixation on an inverted face is closer to the mouth than what is observed in hearing individuals. Furthermore, if deaf individuals rely on a larger area than hearing controls when they process inverted faces and if, consequently, their dependency on holistic/configural face processing is particularly salient with this face category, they should be more impaired than controls in a condition where the central features of inverted faces, but not upright faces, are masked (central mask condition) that forces the observers to rely on the whole face (Van Belle et al., 2010a,b).