INTRODUCTION
In neurology and psychiatry the detailed study of illusory own body perceptions (Lhermitte, 1939; Hécaen and Ajuriaguerra, 1952; Critchley, 1953; Ionasescu, 1960) has suggested close links between bodily processing and self-consciousness. Such illusions include a variety of alterations in perceptual bodily experience such as the experiences of the absence of a body part (Hécaen and Ajuriaguerra, 1952; Critchley, 1953; Frederiks, 1969), body part transformations (Lhermitte, 1939; Hécaen and Ajuriaguerra, 1952; Lippman, 1952; Ionasescu, 1960), body part displacement (Lippman, 1952; Nightingale, 1982), disconnection of one body part from the body (Lippman, 1952; Blanke et al., 2003; Heydrich et al., 2010), the delusional misidentification of one’s own body part (i.e., somatoparaphrenia; Gerstmann, 1942; Vallar and Ronchi, 2009), as well as phantom limbs (Lhermitte, 1939; Hécaen and Ajuriaguerra, 1952; Brugger, 2006), and supernumerary phantom limbs (Vuilleumier et al., 1997; Khateb et al., 2009; for reviews, see Blanke et al., 2008; Blanke, 2012). Dramatic forms of illusory own body perceptions are autoscopic phenomena, such as the out-of-body experience and the doppelganger experience (also known as heautoscopy) of neurological origin (Brugger, 2002; Blanke et al., 2004). During autoscopic phenomena patients not only report altered perceptual bodily experience, but a breakdown of the spatial unity between the physical body and the self (Brugger, 2002). Thus, patients suffering from an out-of-body experience perceive the self and the world as if their self was localized outside their physical bodily boundaries (abnormal self-location and first person perspective) and no longer identify themselves with their physical body but with a spatial position where the illusory autoscopic body is perceived (abnormal self-identification; Blanke and Metzinger, 2009). While out-of-body experiences are disturbing experiences, patients continue to experience the self as singular, that is, they experience only one self that is abnormally localized, but at one single position and vantage point (Brugger, 2002). This is not the case in many patients suffering from heautoscopy, where the self may be experienced as reduplicated – existing at two or even more locations simultaneously (for review, see Brugger et al., 2006). Thus, in heautoscopy the self may be experienced at the location of the physical body, at that of the autoscopic body, or self-location may be at both locations at the same time (Brugger, 2002; Blanke et al., 2004; Heydrich and Blanke, 2013).
How can we study the mechanisms of experienced singularity of self-consciousness in healthy participants? Virtual and augmented reality techniques have enabled scientists in recent years to study many aspects of human behavior and multi-sensory integration. Scientists have used virtual reality (VR) techniques to investigate visual capture of the arm or the entire body (Slater et al., 2009; Sanchez-Vives et al., 2010; Slater et al., 2010), and for a broad range of research topics such as phobias [fear of public speaking (Slater et al., 2006), heights (Usoh et al., 2003), tunnels (Mühlberger et al., 2007), social interaction (Bailenson et al., 2003), proxemics (Llobera et al., 2010), visual control of locomotion (Warren et al., 2001; Mohler et al., 2007), and space perception (Thompson et al., 2004; Mohler et al., 2010)]. Use of VR techniques has also enabled scientists to carefully control stimuli that are otherwise difficult or even impossible to control in the real world.
Virtual reality can be used in combination with video technology (often referred to as augmented reality) or can be used with a 3D graphical rendering of a surrounding visual world. Different research groups have recently developed experimental paradigms to study several aspects of bodily self-consciousness – such as full body ownership or self-identification with the hand or body, as well as self-location (i.e., the experience of where I am in space) – by exposing participants to multisensory conflicts of bodily stimuli (Botvinick and Cohen, 1998; Ehrsson, 2007; Lenggenhager et al., 2007; Ionta et al., 2011; Blanke, 2012; Aspell et al., 2013). Initiated by seminal work on illusory ownership for a fake hand (e.g., rubber hand illusion, Botvinick and Cohen, 1998; Schmalzl and Ehrsson, 2011) more recent work has targeted illusory ownership for a virtual body (e.g., the full body illusion, see below, Ehrsson, 2007; Lenggenhager et al., 2007), a virtual face (Tsakiris, 2008; Sforza et al., 2010) or a virtual voice (Zheng et al., 2011).
Thus, it has been found that during the full body illusion participants self-identify with a virtual body and show drifts in self-location towards the virtual body if their back is stroked while they see their own virtual body on a head mounted display (HMD) being stroked in synchrony (Lenggenhager et al., 2007, 2009; Ionta et al., 2011). These findings were strengthened by Aspell et al. (2009), who showed that this experimental scenario is associated with changes in visuo-tactile perception as quantified by the crossmodal congruency effect (CCE). Despite the importance of these studies for the understanding of self-identification and self-location, these experimental setups did not alter the experienced singularity of the self, as they were mostly inspired by studies of out-of-body experiences (Ehrsson, 2007; Lenggenhager et al., 2007). However, the related neurological condition of heautoscopy is associated not just with altered self-identification and self-location, but frequently with a loss of experienced singularity of the self leading to self-identification with two spatially distinct bodies and self-location at two distinct spatial locations (bi-location of the self; Lenggenhager et al., 2007, 2009; Aspell et al., 2009). As this latter important aspect of bodily self-consciousness has not been studied in earlier experimental work, we here modified the full-body illusion based on a recent experiment that induced illusory ownership for two fake hands that are being stroked synchronously with the hidden hand of the participant (Ehrsson, 2009; Guterstam et al., 2011). While these rubber hand studies were able to demonstrate that supernumerary extremities can be experienced as “mine,” as in the clinical condition of supernumerary phantom limbs (13, 14), we here investigated whether we could experimentally induce self-identification with more than one body.
Running two studies using two different techniques, e.g., video (study 1) and computer generated graphics (study 2), also allowed us to compare the two approaches. On the one hand, 3D computer generated graphics enables the experimenter to control the representation of participants. The experimenter can manipulate, for example, body size (Yee and Bailenson, 2007), identity (Yee et al., 2009), or the motion characteristics of the body (Ries et al., 2009). Video, on the other hand, preserves the visual cues from the real world, including the identity of the participants.
Moreover, in experiments investigating multi-sensory stimuli, precise timing is likely important. Typically in video the delay between the seen image in the HMD and what one would see without wearing the HMD is low and well controlled, whereas in 3D computer generated graphics, this time additionally depends on the tracked data and is typically between 25 and 60 ms. Thus, it might be useful for the field of VR to compare 3D computer generated and video display of humanoid avatars according to new criteria, oriented towards the subjective feeling of self-identification with and self-location towards a virtual body rather than the usual criteria of visual realism.
To summarize, the present studies aimed to investigate whether healthy participants may experience a breakdown of the experienced singularity between the body and the self as quantified through self-identification with more than one body in our different VR setups. This allowed us to compare the two VR techniques that are most widespread today (video-based VR and computer generated-based VR) using the same paradigm. We first extended a previous setup (Lenggenhager et al., 2007) using a video HMD (study 1) and investigated self-location and self-identification while participants saw two fake bodies that were stroked either synchronously or asynchronously with their own body. We hypothesized that self-identification with two fake bodies would be stronger during synchronous stroking and that self-location would be significantly shifted towards the two bodies in the body synchronous (BS) condition only. Furthermore, we hypothesized that synchronous stroking will be associated with the breakdown of the singular self and the feeling of having multiple bodies. In a second follow-up study we adapted the set-up to 3D computer generated graphics in order to compare video with computer generated technology (Study 2) using a between subject design maintaining the same hypotheses as in study 1.
DISCUSSION
We report that self-identification with two virtual bodies was stronger during synchronous stroking as compared to asynchronous stroking. This was accompanied by the sensation of having more than one body and a change in self-location towards the virtual bodies that was body-specific and depended on the synchrony of stroking in study 1. We discuss (1) the implication of our findings for the study of bodily self-consciousness, (2) possible explanations for the differing results in study 1 and 2, and (3) the implications for the use of two widely used VR techniques to investigate visual capture illusions.
EXPERIENCE OF HAVING TWO BODIES
Using a modified version of the full body illusion paradigm in which we presented two virtual bodies, we demonstrate that it is possible to self-identify simultaneously with two virtual bodies. This is in contrast to the classical setup, where participants are exposed to a single second body (the virtual body) in addition to their own physical body, but do not report having more than one body (Lenggenhager et al., 2007). The present data are a first step towards the investigation of double body ownership, e.g., to feel being touched on two visually presented bodies and to identify with the two bodies simultaneously. Interestingly, in study 1, this was further associated with the subjective feeling of having multiple bodies.
Similarly, the rubber hand illusion has been recently extended to more than one fake hand: whereas in the classical rubber hand illusion setup it has been argued that the feeling of illusory ownership is limited to the fake (seen) hand (Botvinick and Cohen, 1998), it has been shown that the use of two fake hands results in the sensation of having an additional hand (Newport et al., 2010), as can be observed in patients suffering from a supernumerary phantom limb of neurological origin (Khateb et al., 2009). Furthermore, Guterstam et al. (2011) showed that healthy participants experience a second additional right hand (“supernumerary limb illusion”) if the real hand was visible during the rubber hand illusion. Questionnaire and skin conductance response data (SCR) provided further evidence that ownership was equal for the real and the fake hand, in contrast to the significant disownership for the real hand which accompanies the traditional rubber hand illusion (Moseley et al., 2008). The authors suggest that this is achieved through multisensory integration processes in premotor and parietal cortices representing two equally probable locations for the seen somatosensory stimulation and therefore identifying with the two hands at the same time. One might argue that our findings could be interpreted in a similar way. However, there are some important differences between the present setup and the one used by Guterstam and colleagues. First, while being exposed to two virtual bodies through the HMD, the physical body was not directly visually perceived during our experiment. We argue that the visual presentation of two virtual bodies is necessary to induce “double” body ownership, as earlier work using the traditional full body illusion (with one virtual body) did not induce the sensation of having two bodies (Lenggenhager et al., 2007). Therefore, our study is closely related with the experimental setup of Ehrsson (2009) investigating the “supernumerary limb illusion,” where also two fake hands are presented while the physical hand is covered. We speculate that the experience of self-identification with both virtual bodies is due to the extension of the large and often bilateral receptive fields in bimodal visuo-tactile neurons in the parietal cortex to include both virtual bodies (i.e., Duhamel et al., 1998; Blanke, 2012). Such bimodal visuo-tactile neurons have been shown to integrate visual stimuli in peripersonal space and tactile stimuli applied to the upper body and face. Secondly, there is an important difference between the sensation of a supernumerary body part and the sensation of double body ownership: while one can experience having an additional limb, the singularity of self-awareness might still be conserved, e.g., there remains a singular self that seems to have a total of three limbs during the illusory state.
The identification with more than one body and the sensation of having more than one body, as observed in study 1 and not confirmed in study 2, should, however, be regarded as a preliminary experimental step towards understanding the mechanisms involved in the experience of the singular self and its loss. We note that double body ownership has been reported clinically in cases of heautoscopy for several decades (Brugger, 2002; Blanke et al., 2004; Heydrich and Blanke, 2013). The present experiment failed to evoke behavioral measures of such strong duplications of the self as behavioral drift in self-location towards the virtual bodies (in study 1) was similar to the effect of viewing a single body (Lenggenhager et al., 2007; Ionta et al., 2011). More work, likely using additional interoceptive manipulations (i.e., Aspell et al., 2013; Suzuki et al., 2013), may be necessary to achieve stronger distortions of the self, associated with double body ownership as in heautoscopy (Brugger, 2002; Heydrich and Blanke, 2013). Although we argue that the drift measure reflects the position in space (self-location) where the subjects felt that they were standing during the illusion and note that it has been replicated across several studies (Lenggenhager et al., 2007; Aspell et al., 2009) and using different techniques (e.g., mental imagery, Ionta et al., 2011), the drift measure by itself is not an objective measure for double body ownership. However, future research will have to address the question of how viewing two bodies affects changes in self-location, also in the right–left axis, more systematically, e.g., by stroking one body synchronously while stroking the other body asynchronously, or by using eye-tracking to observe whether participants’ attention could be biased towards one body.
There are also several differences and limitations in our findings if compared with those obtained in previous experiments using the full body illusion. First, in contrast to the findings of Lenggenhager et al. (2007), modulation of illusory self-identification by synchronous visuo-tactile stimulation in the present study was not limited to the body condition; thus participants reported that they also self-identified with the two objects (question 3). This is in line with findings from Armel and Ramachandran (2003), who showed that participants showed increased SCR and ownership ratings when the “RHI” was performed with a table. This occurred when the table was stimulated synchronously with the subject’s hand, and in absence of a fake hand. We note that, since participants in our studies were required to fixate on a cross between the two bodies (or two objects), this might have also reduced the body specificity of the illusions (e.g., self-identification) as reported previously (Lenggenhager et al., 2007). In line with this argument is the observation that in a recent experiment by Aspell et al. (2009) using the full body illusion and the CCE, the specificity of the full body illusion as measured by questionnaires was also not limited to the virtual body, e.g., synchronous visuo-tactile stimulation also modulated self-identification with an object. It could be argued that the CCE, an attention demanding discrimination task (i.e., a dual task next to the full body illusion), may reduce the body-specificity often observed in the absence of such dual tasks. As neither Ehrsson (2009), Guterstam et al. (2011), nor Newport et al. (2010) reported an object control condition or another attention demanding task during their experiments with an extra hand, it is not known whether self-identification for the two objects in the present experiment was due to experimental differences or due to the fact that in our studies two bodies/objects were presented simultaneously.
The overall findings from study 1 were replicated to some extent in our second study, using 3D computer generated graphics. Self-identification with the two virtual bodies was positively modulated by visuo-tactile synchrony, irrespective of whether two bodies or two objects were presented (question 3). Likewise, illusory touch and illusory causation (questions 1 and 2) were induced by visuo-tactile synchrony. Importantly, we found that participants identify with the two bodies at the same time, e.g., we specifically asked participants in study 2 if they identified with the two virtual bodies simultaneously or in an alternating fashion, and 76% of responses indicated that this double self-identification was happening simultaneously. However, we found that in the 3D computer generated setup participants showed a forward drift in self-location (towards both the avatars/objects) regardless of whether visuo-tactile stimulus was synchronous or not. Also, no sensation of having multiple bodies was reported in study 2.
Statistically, there were significantly stronger results on the underlying components of the questionnaire, namely illusory touch and illusory drift, in study 1 compared to study 2. The effect of body was in particular important to illusory drift. This was consistent with the findings from the overall behavioral drift: an effect of body was found on the behavioral drift in the synchronous conditions. The between-groups comparison also revealed a rightward bias present in both studies, that was observed in the questionnaire results (study 1) and in the behavioral drift measure (study 2), so in this sense the overall left–right drift results were similar in both studies.
The partial discrepancy between the results of study 1 and study 2 leads to the second part of the discussion, namely the comparison between the two VR techniques, which might explain the diverging results of study 1 and 2.
COMPARISON BETWEEN VIRTUAL REALITY TECHNIQUES
The two studies used video and computer generated HMD virtual environments respectively, with the second study using motion tracking and 3D computer generated graphics to render the virtual bodies and stick.
In the following we would like to discuss several issues that may be of relevance when using VR techniques to study bodily self-consciousness, namely distance estimation, visual fidelity, latency, visual realism, and the self-location measure.
Distance estimation has been shown to be veridical with real-world expectations in carefully calibrated video HMD setups, but underestimated in computer generated HMDs (Swan Ii et al., 2006). An underestimation could have caused a change in our self-location measure. We note, however, that in this case self-location results should have been influenced globally across all conditions. In addition, both Mohler et al. (2010) and Ries et al. (2009) have demonstrated that seeing a virtual avatar in HMDs for a short time leads to no underestimation of distances in computer generated HMDs. For these reasons we do not think that distance underestimation played a role in our experimental results, or the differences between the results in study 1 and 2
Further differences between study 1 and study 2 were visual fidelity, visual realism and latency. Here, visual fidelity refers to whether the virtual body resembles the body of the participant. The virtual bodies in study 1 were video images of the participants, and in contrast study 2 used generic 3D avatars.
Although it seems that in study 2 no high visual fidelity was needed to identify with a virtual body or to some extent even with an object (question 3), it has been shown that top–down processes such as visual identity might influence the rubber hand illusion (Tsakiris and Haggard, 2005) and the full body illusion or related paradigms. This could explain the fact that we did not find a change in self-location and no illusory experience of ownership of more than one body in study 2.
Numerous object controls in previous studies have emphasized the relationship between appearance and illusory response, e.g., Lenggenhager et al. (2007) did not detect drift with an object control, and Petkova and Ehrrson (2008) found that greater visual fidelity mattered for their measure of skin conductivity in response to a visual threat, when they compared a human mannequin to an object control. Taking our results in the context of related work suggests that fidelity is more important for certain measures. From our results, those measures are self-location drift (consistent with Lenggenhager et al., 2007) and the subjective sensation of having more than one body, where the effects were only detected using the high fidelity visual bodies from study 1.
Object control conditions reinforce the principle that the objects to be embodied must look human-like. However, our results raise the question of whether there is a fidelity scale, in which higher fidelity objects produce stronger illusions of identification and ownership. Tsakiris et al. investigated this for the RHI with a range of hand-like objects from low (block) to medium (hand shape) to high fidelity (rubber hand). In general the principle was shown to hold, where increasing fidelity appeared to increase the illusion for a variety of measures. Interestingly, consistent with our findings, the effect of drift was particularly sensitive, and only the highest fidelity synchronous condition showed significant drift from the baseline measure (Tsakiris et al., 2010). Similarly, while participants reported ownership over two hand images in a study by Newport et al. (2010) this did not transfer to a motor response task. Thus, when visual fidelity is higher (study 1), we see a positive drift towards the virtual bodies and a significant effect of double body ownership in the body-synchronous condition (question 5), compared to the body-asynchronous control. However, it remains speculative why self-location and double body ownership (question 5) are more sensitive to visual fidelity as compared to illusory touch (question 1 and 2) and self-identification (question 3). For self-location, this may be because the effect can dissociate from self-identification, as suggested by Rohde et al. (2011) in the RHI. Their results for the RHI show that drift can be driven with vision only, results which in our context would predict the importance of visual fidelity to the drift measure. They found that drift can also occur in the asynchronous condition even when self-identification is low, i.e., the results we obtained in the body condition in study 2, which we discuss further below with regards to latency. Regarding double body ownership (question 5), further studies are needed to investigate the relationship with visual fidelity by directly comparing different visual renderings with the same methodology.
Another aspect of the computer generated setup is visual realism. Although full-body and body part illusions have been performed in other VR setups without high visual fidelity, there were still many differences between our study and previous implementations. For example, the plain background in study 2 is in stark contrast to the visual realism in a study from Slater et al. (2010), who created visual capture with a virtual environment consisting of a detailed scene, complete with fireplace, television, sound, virtual mirror, and interaction, i.e., when participants moved their head they could look around, which was not the case in the studies presented here. Any one of the additional details could be used by computer generated VR to increase visual capture, presence, and other factors. Thus, it may be necessary to add more details to the computer-generated environment to elevate the illusion quality and strength to those obtained with video (or to create an equal illusion of presence to that achieved by video). However, understanding the importance of each component in the VR setup, e.g., sound, 3D model, interaction, would require future, more systematic, work. The work would build on the findings here, suggesting that the similar experimental conditions using two different VR technologies show stronger results in video-based VR.
The final major difference between our two studies was latency. Although the delay between the stick physically touching the body and the visual stick touching the virtual body was only approximately 25 ms longer in study 2 than study 1, this increased latency could have reduced the difference between the synchronous and asynchronous conditions. Importantly, according to a recent review on intersensory synchrony, delays of >20 ms do become noticeable (Vroomen and Keetels, 2010). Rohde et al. (2011) showed a positive drift can be also found during the asynchronous condition in the RHI, i.e., changes in perceived hand position still occurred with asynchronous stroking. If we analyze, for example, the self-location measure in study 2 without the asynchronous conditions and compare the BS to the OS condition, then we do in fact find a significant drift towards the virtual bodies in the body condition as compared to the object condition [t(16) = 2.37, p = 0.03]. This would support the hypothesis that a synchrony-dependant change in self-location was not found in study 2 due to a weaker contrast between synchronous and asynchronous conditions and that mostly a difference between the body and the object condition was observed. A solution to this weaker contrast in study 2 would be to create a more delayed asynchronous condition, or alternatively to use a more sensitive measure of self-location. Already Lenggenhager et al. (2007) have speculated on the importance of their asynchronous control condition, and their first experiment did not use a delay in the asynchronous condition, but a pre-recorded video, which they note would be perceived as even more asynchronous and less predictable, thus creating a stronger contrast between synchronous and asynchronous experimental conditions. Similarly, Sanchez-Vives et al. (2010) used pre-recorded animation in their asynchronous condition, in a computer graphics version of the RHI. In addition, more robust behavioral measure of self-location might be needed, such as the CCE (Aspell et al., 2009), distance estimation measures (Ionta et al., 2011; Pfeiffer et al., 2013), or motor imagery (Ionta et al., 2012) as these are based on repeated reaction time measures (Aspell et al., 2009).
Additionally, we suggest that the illusion may be harder to detect with two bodies. Related work with lower fidelity setups or similar VR setups with equivalent latency used one body or part of the body. A combination of higher fidelity, higher realism, and lower latency may be demanded in the case of study 2 to detect the effect of self-location with more than one virtual body.
Finally, we note that a within-subject design would have given more statistical power in a comparison between the two techniques used. Unfortunately this was not possible, given that the technique for study 1 was only available in Lausanne and the technique for study 2 only in Tübingen. Although we tried to replicate the setup as close as possible, the above outlined comparison of the two techniques is therefore based on a between-subject design and some statistically significant differences may have been missed.