This paper presents the development of a virtual climbing application using commercially available virtual reality (VR) equipment. The application enables users to simulate climbing a virtual wall with their hands and feet while remaining physically grounded. In contrast to real climbing, where climbers support their bodies by gripping and stepping on holds, the virtual environment requires users to maintain postural balance with one foot suspended in midair. This discrepancy highlights a fundamental difference in the motion dynamics between real and virtual experiences. This study analyzed this inconsistency and proposed methods to enhance the sensation of natural climbing in VR. The experimental results revealed that the application provided an accessible and enjoyable experience, allowing users to perform climbing-like movements using both hands and feet. However, their sense of climbing realism and feelings of strangeness varied individually. The mismatch in motion coordination did not significantly impair realism and enjoyment; however, the absence of tactile feedback—specifically, the sensation of force through the hands and feet—resulted in perceptual gaps for users. This paper also describes climbing movements that cannot be performed during virtual climbing in place and discusses potential solutions. These findings offer valuable insights into improving realism and enjoyment in VR-based climbing simulations.
boulderingclimbinginteractive entertainmentperceptual mismatchvirtual realityThe author(s) declared that financial support was received for this work and/or its publication. This study was supported by JSPS KAKENHI (grant number: JP25K15201).section-at-acceptanceTechnologies for VRIntroduction
Virtual reality (VR) enables users to navigate and explore virtual spaces freely while remaining physically stationary at a location such as a living room. Experiences such as running across various fields, competing with other players, and walking around a famous tourist spot using a head-mounted display are attractive to many users. Several VR applications allow users to navigate a virtual space using a joystick or buttons on a handheld controller. These interfaces can be operated while seated, making them suitable for applications where users are in a relaxed state.
To enhance the realism of movement in VR, interfaces have been developed that allow users to walk or run by swinging their arms (Coutinho, 2022; McCullough et al., 2015) or stamping their feet (Hirao et al., 2024; Nilsson et al., 2013). Swinging the arms or stamping the feet imposes a physical load suitable for exercise. While treadmills can offer a realistic walking or running experience, they require substantial space and raise safety concerns when used with a head-mounted display (Birnstiel et al., 2022). Recently, commercially available interfaces featuring slippery floors or specialized shoes have been developed to enable walking or running in VR (Diaz et al., 2024). These interfaces allow users to change direction quickly, making them ideal for games.
Although various interfaces have been developed for walking and running, research on climbing interfaces remains limited. However, climbing typically requires access to specialized facilities or natural mountainous terrain. Recent studies have explored using mixed-reality technologies to simulate climbing in diverse environments (Kosmalla et al., 2017; Kosmalla et al., 2020; Schulz et al., 2019; Tiator et al., 2018). However, these approaches depend on indoor climbing walls. While commercially available climbing wall treadmills offer climbing experiences in limited spaces, they depend on mechanically operated vertical movement, making them costly and impractical for casual home use.
To provide accessible home entertainment, VR game applications that allow users to experience climbing at home are available on the market (Crytek, 2021). In this application, users hold controllers in both hands, reach for holds on the virtual wall, press control buttons to grab the holds, and simulate climbing by moving their hands downward—all while remaining in a standing position. Climbing in this application relies only on hand movements. However, in actual climbing, it is important to use both hands and feet according to the wall and body positions.
If VR climbing allowed users to climb with both hands and feet, it would enhance the realism, fun, and exercise effectiveness of the experience (see the reviews by Gonçalves et al., 2022; Mouatt et al., 2020). Accordingly, this study aimed to develop a VR climbing application that allows users to climb using both hands and feet at home, utilizing commercially available VR equipment. The immersive virtual experience will primarily contribute to home entertainment but may also support exercise.
In this paper, after presenting related work, we present the technical challenges in enabling virtual climbing while standing on the floor and its design. The proposed system was evaluated through user experiments, and its effectiveness, limitations, and potential future developments are discussed.
Related work
As mentioned in the Introduction, VR applications that enable users to experience climbing with both hands and feet while standing on the floor have not yet been developed. Salzman et al. (2024) introduced a VR system that enables users to grasp holds while standing on the floor. This system was designed to rehearse the hand and body positions for training. However, full-body climbing involving both the hands and feet has not been implemented.
In contrast, some studies have attempted to alter the user experience of climbing a real wall using VR or MR. For example, Kosmalla et al. (2017) proposed a VR system in which users wear a head-mounted display (HMD) to experience climbing on a large mountain while climbing a physical wall. A similar MR-based system was developed by Tiator et al. (2018). Schulz et al. (2019) assessed stress and anxiety in VR climbing, where users experienced climbing a tall wall while climbing a low physical wall. Kosmalla et al. (2020) showed that inclusion of feet in the VR climbing experience is more important than hand visualization in terms of perceived hand and foot movement and climbing enjoyment, which highlights the importance of including foot operation in the VR climbing system developed in the current study.
One technical problem in developing a VR system in which users experience climbing while standing on the floor is the discrepancy in the body posture between the real and virtual spaces. Because no support exists in the real world, some climbing postures become impossible. Therefore, an appropriate transformation of body movements from real space to virtual space is required to avoid a feeling of incongruity. This problem also arises in virtual walking by stamping feet in place, which is referred to as walk-in-place, because the leg motion of natural walking is not identical to that of stamping feet in place. No unique definitive way exists to determine virtual walking speed from leg motions during stamping feet. Therefore, appropriate methods have been investigated in many studies (Hirao et al., 2024; Nilsson et al., 2013).
Slater et al. (1994) applied the walk-in-place technique to virtual stair climbing and ladder climbing. However, vertical movements, such as stair climbing, pose additional challenges. Because of the absence of steps in the real world, users receive no force feedback for stepping. In approaches to virtual stair climbing proposed by Asjad et al. (2018) and Nagao et al. (2018), users step on small bumps to receive force feedback during stepping. However, this method requires users to walk forward to step over bumps, which necessitates a larger space. Hirao et al. (2024) developed a system in which resistance is added by affixing a rubber band to the feet of users, which allows them to experience walking on a slope. This system, however, requires additional equipment.
Lai et al. (2015) developed a VR system that enables users to experience climbing ladders by moving their hands and feet. Their system is similar to the system developed in the current study because users climb in place using both hands and feet, relying solely on commercially available VR products. However, climbing a straight ladder involves simple cyclic movements. In contrast, virtual climbing in the current study requires more complex movements, which pose additional challenges that must be addressed, as described in the next section.
Challenges of virtual climbing in place
Climbing with both hands and feet while standing on the floor (i.e., climbing in place) can introduce a discrepancy between user posture in virtual and real spaces, which is the main issue discussed in this study. Figure 1 illustrates an example of a user placing their foot on a virtual hold, while the corresponding foot in the real world is suspended above the ground. Lowering the floating foot in real space (i.e., the right foot in the figure) simulates stepping onto the hold and results in a climbing motion in the virtual space. After this movement, both feet rest on the floor (see the right part of Figure 2). When the foot positions in the virtual space correspond to those in the real space, stepping down with the right foot requires the left foot to float simultaneously, as shown in the left part of Figure 2. Therefore, the user appears to stand only on their right foot in the virtual space while standing on both feet in real space, creating a discrepancy between the virtual and real spaces.
Climbing in place: a player is experiencing climbing by wearing a head-mounted display.
Graphic showing a person in a rock climbing pose on the left with dark climbing holds and, on the right, the same pose with blue hand controllers and a blue headset, illustrating a virtual reality climbing setup.
Climbing movements with right foot in real and virtual space. Gray and blue silhouettes represent the posture before and after the movement, respectively.
Illustration showing a person rock climbing on the left, with a zoomed-in progression on the right depicting a person in a virtual reality headset using hand controllers, shifting from an active climbing pose to a standing position.
Matching the relative positions of the hand and foot presents another challenge in simulating realistic climbing motion. In real climbing, climbers rely on their hands to maintain contact with the wall and support their body. Therefore, during climbing, the user typically grips the hold with at least one hand. To align hand positions in both the virtual and real spaces in this example, the user must lower the hand simultaneously while stepping down with the foot, as shown in Figure 2. If the user does not move the hand in this motion, the virtual hand will appear to float away from the hold, resulting in an unfeasible action of supporting the body only with feet. As shown in this example, discrepancies between the relative positions of the limbs in virtual and real spaces are likely to occur when climbing with both hands and feet. This discrepancy may result in unfeasible motions and induce an unnatural feeling in the users. This study addresses this issue by developing an algorithm that enables users to climb naturally using both hands and feet. The effectiveness of the algorithm was evaluated through user experiments.
DesignDevice
In this study, a VR climbing system was implemented using an HTC VIVE Pro headset and VIVE Trackers within the Unity3D game engine. The system tracked the head position via the head-mounted display (HMD), hand positions using handheld controllers, and shin positions with trackers attached to the user’s legs. Foot positions were estimated using shin positions. Figure 3 shows a player climbing using the developed system.
A player wearing a head-mounted display climbing the virtual wall using both hands and feet.
Two-panel photograph showing a person in a black outfit using a virtual reality headset and controllers in an indoor setting, raising one arm while lifting a leg equipped with motion sensors.Algorithm of basic climbing
The developed VR system displayed a climbing wall, along with the player’s hands and feet, through the HMD; however, the legs and torso were not visually rendered (see Figure 4). The players perceived climbing on the virtual climbing wall as an ascending scene displayed through the HMD. Therefore, the virtual climbing algorithm is a method of moving the player’s head in a virtual space in response to hand and foot motions.
Climbing wall and a player’s hands displayed through a head-mounted display (a), a player placing the right foot on a hold (b).
Screenshot from a virtual reality environment showing a digital climbing wall with color-coded holds and simulated gloved hands gripping the wall at the top. Below, panel (b) displays two side-by-side perspectives of the wall: the left shows one virtual foot and climbing holds, the right shows the same scene after a movement with both virtual feet on climbing holds separated by a rightward arrow.
The players supported their bodies with their hands and feet on virtual holds fixed to the wall. The players in the developed system could grip a hold by pressing a trigger button on the handheld controller when touching the hold with their hand (see Figure 4a). Furthermore, they could release their hand from the hold by releasing the trigger button. This control method is used in climbing games in the market (Crytek, 2021). The player could place their foot on a hold by moving the foot downward into contact with the hold (see Figure 4b). This interaction was limited to occur only when moving the foot downward, thereby avoiding the unnatural effect of the foot sticking to the hold from below while moving upward. To release the foot from the hold position, the player must move the foot upward by more than 0.015 m. The foot placed on a hold changed color to green to notify players of the status of their feet. Sound effects were played when the foot was placed or released.
When the players gripped a hold on the virtual climbing wall or place their foot on the hold, the hand or foot was fixed to the hold in the virtual space. In this paper, we refer to the most recently fixed hand or foot on a hold as the reference extremity. The positions of the other three extremities (hands and feet) and the head in the virtual space were determined based on the positions of the extremities relative to the reference extremity in the real space, as expressed in the following equation:pv=prefv+pr−prefrwhere prefv and prefr represent the position of the reference extremity in virtual and real spaces, respectively. pv and pr represent the positions of the other three extremities or the head in virtual and real spaces, respectively. The positions of the extremities and the head in real space were collected by the VR system at 90 Hz. For example, when players gripped a hold with their right hand, maintained a stable hand position, and moved the other hand and feet, they saw the hand and foot motions through the HMD reflecting the motions in real space while the right hand was on the hold. When the players gripped a hold with their hand and moved it downward, the head and other extremities moved upward in the virtual space, resulting in climbing. When the players moved their hand fixed on the hold to the right, the hand remained stable; however, the head and other extremities that were not fixed to the hold moved to the left in the virtual space, resulting in the player moving to the left on the climbing wall (see Figure 5). The reference extremity was not labeled visually to distinguish it from the limbs, as players could readily identify it from their movement in virtual space.
Moving the right hand (attached to a hold) in real space results in body movement in the opposite direction in the virtual space. Gray and blue silhouettes represent the posture before and after the movement, respectively.
Graphic showing a person engaged in rock climbing on the left with hand and foot holds depicted, while on the right, the same person wears a head-mounted device and interacts with virtual climbing holds, suggesting virtual reality rock climbing.
When multiple extremities were fixed to holds, the most recently fixed extremity was designated as the reference extremity. If the reference extremity was released, the next most recently fixed extremity assumed the role of reference extremity. Accordingly, the players’ motions on the climbing wall were determined by their hands and feet. However, this basic climbing algorithm occasionally allowed players to perform physically implausible movements. Therefore, additional rules were introduced to determine the positions of the extremities and select a reference extremity, thereby addressing the inconsistency between the hand and foot positions in the real and virtual spaces.
Additional rules for impossible motionsWeight transfer to a foot in the air
As described in the Introduction, when players climb by stepping on a virtual hold, the opposite foot appears to float in the virtual space while remaining grounded in the real world. In this state, players can move sideways by transferring their weight to the foot in the air because the real foot can be on the floor. However, weight transfer to the foot in air should be impossible. Accordingly, when the real feet were on the floor and weight transfer to the foot in the air was detected, the player fell off the virtual wall. As an implementation of this rule, in the developed system, the player falls from the virtual wall when the Euclidean distance from their head to the floating foot is less than that to the other foot (see Figure 6). The weight on each foot under static conditions can be theoretically estimated based on the relative position between the feet and the center of mass of the player. If the two distances are equal, that is, the player is standing symmetrically, the weight on each foot will be the same. If the distance between the center of mass and the floating foot becomes smaller than the other distance, that is, if the center of mass of the player moves to the floating foot (we note that the foot is on the floor in reality), the weight on the floating foot increases, whereas the weight on the other foot decreases. The proposed system uses the head position rather than the center of mass to estimate the relative weight distribution between the two feet because the center of mass depends on the body shape and posture of the player. According to the detection algorithm, the player falls from the wall if they place more than half of their body weight on a foot floating in the virtual space.
Placing weight on the left foot on the floor in real space but on air in virtual space. Gray and blue silhouettes represent the posture before and after the movement, respectively. If the distance from the head to the floating foot (D1) is less than that to the other foot (D2), the player falls from the wall.
Illustration showing a person rock climbing on the left and a person with a virtual reality headset and controllers on the right. Arrows indicate limb movement directions, a red X and arrow highlight a foot misplacement, and lines labeled D1 and D2 represent distance measurements from the headset to a flat surface.
In the developed system, placing weight on a foot positioned in midair was not strictly prohibited. This design choice reflects real-world climbing scenarios, where climbers can shift their bodies sideways while hanging from holds using their hands. Prohibiting weight placement on a foot floating in virtual space may limit natural movements. Therefore, the algorithm seeks to balance motion flexibility with a reduction in unnatural sensations.
Display of the hand and foot fixed on a hold
The positions of the hand and foot fixed on a hold were maintained until they were released, regardless of their movement in real space. Players may unintentionally move their hands due to their real hands being raised. This can cause the virtual hand gripping the hold to shift unnaturally within the virtual environment. Accordingly, the developed system allowed players to move their hands in the hold state for less than a determined distance, while the virtual hands and feet were stable during the holds. When movements exceeded a defined threshold, the avatar was released from the hold and repositioned relative to the reference extremity. The foot positions were handled similarly.
The inconsistencies between the positions in the virtual and real spaces were mostly caused by unintentional movements. As in the example described in the introduction, when players step on a hold while gripping a different hold with their hand, they must move their hand on the virtual hold downward in union with the stepping motion. However, this movement was challenging for novice users. If players failed to move their hands fixed on holds while stepping, they would fall from the wall because the system did not permit supporting the body solely with their feet for longer than a predefined duration. To prevent unintentional errors, the application provides an auditory alert whenever a limb is released from a hold.
Climbing using only hands or feet
When players climb by pulling a hand downward on a hold, the corresponding foot placed on a hold may lift off the surface in response to the hand movement. In real climbing, this motion requires a heavy load because players lift their bodies using only their hands. However, in a VR climbing system, this motion requires no load, resulting in an unnatural feeling. Accordingly, the developed system employs the hand energy for each hand to determine the possible hand movements. Players can climb using their hands only when they have sufficient energy. Hand energy was consumed according to the climbing motion and recovered over time. A similar system is employed in a game in the market (Crytek, 2021). Players in the game rely solely on their hands, using upper body strength to climb. In the proposed system, players can use their feet to lift their bodies. Therefore, hand energy consumption is adjusted based on foot usage during climbing. When players climb using their feet, no hand energy is consumed, even if the hands move downward. This system offers players a wide range of strategies.
The rule regarding hand energy consumption can adjust the difficulty of virtual climbing. In the experiment described in Section 5, the participants had no prior experience with the VR climbing application. Therefore, the experiment was conducted without applying the hand energy rule. Similarly, in real climbing, players must grip a hold with at least one hand to support their body. This regulation can also affect the difficulty of the climb. In the experiment, the system allowed players to maintain contact with the wall without gripping any hold for up to 6 s. If the duration exceeded this limit, the players fell from the wall.
Passing through the wall
The floating hand and foot in virtual space were displayed based on their positions relative to the reference extremities in real space. Therefore, when players moved their hands or feet across a wall, their corresponding virtual hand or foot became embedded in the wall, creating an unnatural appearance. To avoid this problem, the positions of the displayed hands and feet were limited to avoid passing through the walls. This limitation caused a mismatch between the positions of the reference and other extremities. However, this mismatch did not result in an unnatural experience, as the discrepancy remained minimal unless players deliberately pressed their hands and feet into the wall.
Experiment
To assess the developed system, we conducted an experiment where participants experienced VR climbing and answered a questionnaire. The participants experienced two types of virtual climbing—using only their hands, and using both hands and feet—then compared the two approaches.
MethodsParticipants
Twenty-four male students aged 18–24 years voluntarily participated in the experiment. All the participants provided written informed consent before participating in the study. The study was conducted in accordance with the principles of the Declaration of Helsinki. The participants had no prior experience with the commercially available VR climb application.
Climbing wall
The climbing wall in the virtual space was created using a commercially available model of a climbing wall measuring 6 m wide and 10 m high (see Figure 7). The holds on the wall were arranged such that the adjacent distances were larger than 0.5 m. There are three types of holds, each distinguished by color: green holds are for hand grips, orange holds are for foot placement, and pink holds can be used for either hands or feet. This regulation was implemented to increase the difficulty of climbing, as players could climb easily using only their hands.
The virtual climbing wall used for the experiment.
Illustration of a climbing wall with green holds designated for hand grips, orange holds for foot placement, and pink holds for both hands and feet, each type distributed across the wall surface.Procedure
The participants engaged in the virtual climbing task described above, which involved climbing using both hands and feet, and compared it with a version allowing hand use only. Before the experiment, the participants practiced gripping holds with their hands and placing their feet on them for 1 minute. Fixing the VIVE trackers to the player’s shins and practicing climbing with both hands and feet required approximately 5 min. First, the participants freely experienced virtual climbing using only their hands for 5 min and then answered the questionnaire. In the questionnaire, the participants rated, on a scale of 1–7, their level of enjoyment and the ease of understanding the operation. Furthermore, they rated, on a scale of 1–5, their sense of climbing realism and any feelings of strangeness. If they felt strange, they were asked to describe the details thoroughly. Next, the participants experienced virtual climbing using both hands and feet for 5 min and answered the questionnaire. In the questionnaire, they rated the following on a 7-point scale: enjoyment, perceived change in realism compared to hand-only climbing, and ease of understanding hand and foot operations. They also rated their sense of climbing realism and any feelings of strangeness on a 5-point scale. The Likert scale response anchors for each question are shown in Figure 8, along with the results. If they felt strange, they were asked to describe the details thoroughly. Finally, the participants were asked to report their opinions, discoveries, and impressions of the system. A 7-point Likert scale was employed when responses required indicating both neutrality and directional tendencies. For questions without a neutral option, where the presence or absence of experience was to be answered, a 5-point scale was used to reduce variation due to subjective differences in respondents’ criteria. To enable an effective assessment of the validity of the foot interface, the order of experiencing the climbing with hands only and climbing with both hands and feet was not counter-balanced. Climbing with hands and feet encompasses hands-only climbing. Therefore, if participants first experience climbing with both hands and feet, this prior experience may introduce a significant bias and learning effect in the hands-only climbing condition. In addition, a preliminary experiment showed that climbing with hands only was much easier than climbing with both hands and feet; additionally, the participants climbed much more quickly and had much less fun after experiencing the climbing with both hands and feet. Accordingly, this experiment was designed to assess the impact of introducing the foot interface and to identify issues for future development.
Participants’ responses to virtual climbing experiences. (a) Did you enjoy the virtual climbing? (b) Was the hand/foot operation easy to understand? (c) Did you perceive the sense of climbing? (d) Did you perceive any change in realism compared to climbing with hands only? (e) Did you experience any perceptual mismatch?
Grouped horizontal stacked bar charts labeled panels a through e display survey responses for “Hands only” and “Hands and feet” or “Foot” conditions. Colored segments represent responses on seven-point Likert scales, with axes annotated from zero to one hundred percent. Each panel measures different subjective experiences such as enjoyment, perceived difficulty, sense of climbing, change in realism, and perceived discrepancy, with corresponding legends indicating the scale points using distinct colors.Results
The results of the questionnaire used to evaluate the virtual climbing experience are shown in Figure 8. The ratings were compared between climbing methods with the Wilcoxon signed-rank test at a 5% significance level using IBM SPSS Statistics ver. 28.0.1.1.
Regarding the enjoyment of the climbing activity (see Figure 8a), 79% of participants rated the hands-only climbing as “Enjoyed it very much” or “Enjoyed it.” The rest of the participants rated it as “Enjoyed it slightly,” and no participant rated it as neutral or negative. Regarding the climbing activity with both hands and feet, 75% of participants rated it as “Enjoyed it very much” or “Enjoyed it” and 21% of them rated it “Enjoyed it somewhat” or responded neutrally, and one participant rated it as “Did not enjoy it very much”. The binomial test indicated that, for both climbing methods, the number of participants who rated this activity positively (scores 5–7) was significantly larger than the number of remaining participants (p < 0.001). The average rating of the climbing activity only with hands (M = 6.2, SD = 0.8) was slightly higher than that of climbing with hands and feet (M = 5.9, SD = 1.1); however, the Wilcoxon signed-rank test showed no significant differences between the ratings of these climbing methods (z = 1.31, p = 0.190, r = 0.27).
Regarding the understanding of hand operations (see Figure 8b), all participants rated the hands-only climbing as “Very easy” or “Easy”. Regarding the hand operation while climbing with the hands and feet, all participants rated it as “Very easy” or “Easy”, except for one participant who rated it as “Slightly easy”. The average rating of the ease of understanding hand operations while climbing with both hands and feet (M = 6.3, SD = 0.6) was slightly lower than that while climbing with hands only (M = 6.6, SD = 0.5), and the difference was significant (z = 2.12, p = 0.034, r = 0.43). Most participants rated the ease of understanding foot operation positively, although three participants rated it as neutral or “Slightly difficult”. The rating (M = 5.6, SD = 1.0) was significantly lower than that of the ease of understanding hand operations while climbing with hands and feet (M = 6.3, SD = 0.6, z = 3.2, p = 0.001, r = 0.65).
Regarding the sensation of climbing (see Figure 8c), when climbing with hands only, most participants rated it positively, although one participant rated it as “Felt almost no sense”. For the climbing activity using both hands and feet, 54% of the participants rated it as “Felt the sense slightly”, and the rest of the participants rated it as “Felt the sense” or “Felt the sense very much”. There was no significant difference between the rating of the climbing sensation with the hands only (M = 3.7, SD = 0.6) and that with both hands and feet (M = 3.6, SD = 0.7, z = 0.58, p = 0.56, r = 0.12).
According to 63% of the participants, the realism of the climbing was increased by adding foot operations, whereas 25% of participants reported that it decreased, and 13% of participants reported that it did not change (see Figure 8d). The average rating for the increase in realism was 4.6 (SD = 1.3), and the one-sample Wilcoxon signed-rank test showed that the median was significantly greater than neutral (score = 4, z = 2.1, p = 0.036, r = 0.43). This indicates an increase in perceived realism, though large individual differences were observed.
Regarding the perceptual mismatch during hands-only climbing (see Figure 8e), 42% of the participants reported little to no discrepancy, whereas 58% of the participants felt a slight to moderate discrepancy. In contrast, when climbing using both hands and feet, 25% of participants reported little to no discrepancy, whereas 75% reported a slight to moderate discrepancy. The Wilcoxon signed-rank test showed no significant difference in the perceptual discrepancy reported for the hands-only climbing (M = 2.67, SD = 0.96) relative to that reported for climbing using both hands and feet (M = 2.83, SD = 1.13, z = 0.467, p = 0.641, r = 0.10). In both types of climbing, more than half of the participants reported a slight or greater discrepancy in perception, whereas some reported no inconsistency.
Comments provided varying reasons regarding these ratings. For hands-only climbing, comments by 10 participants attributed the perceptual mismatch to the absence of force feedback, noting the ability to climb with minimal effort or use hand positions that would not be feasible in real climbing. Two participants reported the sensation of moving the wall down instead of moving their bodies, whereas one participant found it strange to climb without using their feet. Six participants commented on the unrealistic visuals, particularly noting a lack of realism in the appearance of a hand gripping a hold.
For climbing using both hands and feet, two participants reported observing a discrepancy in terms of having to support their body while standing on one foot. Another two participants observed a discrepancy between having both feet on the floor and standing on one foot in the virtual space. Regarding the inconsistencies in the hand and foot positions between the real and virtual spaces, two participants reported a discrepancy in the position of their feet, floating in the virtual space, while climbing with their hands. Other comments highlighted the lack of force feedback. Two participants commented that they could climb using only their feet or hands, with minimal effort. Three participants experienced a discrepancy regarding the foot sensation of stepping on a hold. Additionally, three participants observed a discrepancy in the sensation of moving their hands while gripping holds and simultaneously using their feet to climb. One participant demonstrated a discrepancy in that the gripping sensation was consistent across holds of different sizes.
To analyze the relationships among ratings of experiences, we conducted a correlation analysis for each climbing method. Tables 1, 2 show the Pearson’s correlation coefficients for these ratings for the two climbing methods. Enjoyment was significantly correlated with the perceived realism of the experience in both climbing methods. Specifically, for the climbing with both hands and feet, enjoyment was also correlated with the ease of foot operation and negatively correlated with the sense of discrepancy. A weak negative correlation was observed between the ease of foot operation and sense of discrepancy. Realism was not correlated with the ease of hand or foot operation but was negatively correlated with the sense of discrepancy.
Pearson correlation among ratings for hands-only virtual climbing experience.
1
2
3
1. Enjoyment
-
2. Reality
0.478*
-
3. Ease of operation
0.302
0.226
-
4. Sense of discrepancy
−0.277
−0.402+
−0.120
*Correlation is significant at the 0.05 level (2-tailed).
+Correlation is significant at the 0.1 level (2-tailed).
Pearson correlation among ratings for experience of virtual climbing with both hands and feet.
1
2
3
4
1. Enjoyment
-
2. Reality
0.560**
-
3. Ease of hand operation
0.327
0.143
-
4. Ease of foot operation
0.491*
0.276
0.460*
-
5. Sense of discrepancy
−572**
−0.519**
−0.182
−0.351+
**Correlation is significant at the 0.01 level (2-tailed).
*Correlation is significant at the 0.05 level (2-tailed).
+Correlation is significant at the 0.1 level (2-tailed).
In summary, virtual climbing was enjoyable and easy to perform, but discrepancies were reported, mainly owing to the absence of force or relative lack of body movement. Adding foot operations increased the sense of climbing realism for approximately two-thirds of the participants. However, approximately one-fourth of participants reported a decreased sense of realism and increased perceptual discrepancy, indicating substantial individual differences in perceived realism during climbing.
Discussion
In virtual climbing using both hands and feet, inconsistencies in the hand and foot positions between the real and virtual spaces are unavoidable without using any specific equipment. Although there were concerns that these inconsistencies might reduce realism and enjoyment, experimental results showed that approximately two-thirds of the participants felt that adding foot operations enhanced the sense of realism. Most participants did not comment on the discrepancy in hand and foot positions between the real and virtual environments. When people climb with their hands or feet, they usually focus on the destination of their hands or feet (i.e., a hold for grabbing or stepping on) and the hands or feet with which they are moving. Accordingly, the current positions of the other hands and feet obtained from visual information may not receive attention because people can perceive the positions of their hands and feet through proprioception (Proske and Gandevia, 2012; van Beers et al., 1998).
In contrast, many participants commented on the ability to climb with minimal effort or use hand positions that would not be feasible in real climbing. In hands-only virtual climbing, players could climb while standing upright. However, in reality, most individuals cannot move along a wall using only their hands due to limited upper body strength. This is only possible in the case of virtual climbing. This discrepancy may have decreased the realism and caused the feeling of pulling down the wall, rather than climbing. This problem also exists in the virtual climbing with hands and feet. The authors were concerned that the absence of force feedback might make the climbing feel less realistic and enjoyable than hands-only climbing. However, despite notable individual differences, most participants enjoyed virtual climbing using both hands and feet. During the climbing activity, the players had to keep their feet suspended, requiring them to balance their bodies to avoid falling. The need to maintain balance may increase the difficulty and nature of climbing, thereby contributing to enjoyment and realism.
Although most participants found the virtual climbing using both hands and feet to be enjoyable, this enjoyment did not exceed that of the hands-only climbing. The evaluation of the two types of climbing was not counterbalanced in this experiment. Therefore, the experience of these types of climbing could not be compared rigorously. However, it is noteworthy that enjoyment was rated as negative or neutral by some participants only for the case of climbing using both hands and feet. There was a strong correlation between enjoyment and perception of realism; on average, participants reported increased realism when foot operation was added, though some participants rated the realism as low. Therefore, such large individual differences in perceived realism must be addressed in the future development of virtual climbing with both hands and feet. Negative correlations were observed between the sense of discrepancy and realism, and between the sense of discrepancy and ease of foot operation; therefore, making it easier to place one’s foot on a hold by modifying the detection algorithm could increase the enjoyment of virtual climbing. Moreover, some movements are possible in reality but impossible in virtual climbing. As described in Section 4.3.1, the developed system prevents players from primarily putting their weight on their floating foot in the virtual space (see Figure 6). However, for players with actual climbing experience, this movement—grabbing a hold with one hand while their body is unsupported and reaching out with the other to grab another hold—not only poses a challenge but also adds to the enjoyment of climbing. Realizing this movement while standing on the floor requires specialized facilities that support the user’s weight, often entailing significant space and cost. One likely solution that does not require large facilities is adding force feedback equipment to the hand controller. Simulating the grip force required for this movement increases the effort needed and may reduce the problem of players easily performing this movement while keeping their feet on the floor. This presents an interesting avenue for our future work.
Another impossible movement in the virtual climbing arises owing to the rule that the reference extremity (the hand or foot last fixed to a hold) determines movements on the wall. Players must move their reference extremity to move on the wall. Therefore, to move using a hand or foot other than the reference extremity, the player must either release the reference extremity from the hold and switch to the previous reference extremity, or release the desired hand or foot from the hold and reattach it so that it becomes the reference extremity. This action is not necessary in real climbing. Player movement on the wall can also be determined based on the movements of all hands and feet fixed to holds. However, as described previously, when moving the body with multiple hands and feet fixed to holds, players must move all of those extremities in the same way. This is particularly difficult and requires practice, especially when using both hands and feet. The reference extremity rule makes it easier for players to understand how to move on the wall.
The difficulty of synchronizing the movements of hands and feet that are fixed to holds may also be reduced by adding force feedback equipment to the hand controller. For example, when a player is gripping holds with both hands, if the player moves the reference extremity hand while keeping the other hand stationary, the current system will cause the other hand to move off of the hold. If the force required for the other hand to grip the hold increases in synchronization with the movement of the reference extremity, the player may perceive the other hand to be gripping the hold, leading to more natural movements.
Regarding the unnatural sensation felt when climbing with minimal force, applying force feedback via the hand controller may also increase the effort required for climbing and mitigate this problem. However, a hand controller with force feedback only creates the sensation of force on the hands. Implementing real force feedback to the arms and legs of players requires specialized facilities. However, wearable force displays employing resistance (Achibet et al., 2015; Al Maimani and Roudaut, 2017; Mitsuda et al., 2002) or pseudo-force feedback (Mitsuda, 2013) techniques may offer a solution to mitigate perceived strangeness. These approaches will be explored in future work.
Another important aspect of future work will be investigating the individual differences observed regarding the perceptual mismatch and enjoyment during the virtual climbing experience using hands and feet. All participants in this study were young men (students), which is a limitation of this study. To understand the sources of these individual differences and enhance the applicability, future studies should involve a more diverse group of participants with varying ages, genders, and levels of experience with VR and climbing. Furthermore, no participant claimed any burden during the experience of virtual climbing; however, standing on one foot and maintaining balance may be challenging for elderly people. Climbing using hands and feet is likely to produce more fatigue than hands-only climbing. However, given the popularity of VR-based fitness applications (Mouatt et al., 2020), climbing using hands and feet could be advantageous, depending on the application and the users.
Conclusion
In this study, a virtual climbing application was developed using commercially available VR equipment. In the application, players climb a virtual wall using their hands and feet while standing on the floor. In real climbing, climbers support themselves by gripping holds and placing their feet on them. In contrast, virtual climbing requires players to maintain body stability while one foot is suspended in the air. This study addressed the inconsistencies between hand and foot motions in real and virtual spaces and suggested solutions that provide a natural climbing experience. The experimental results indicated that virtual climbing, which incorporates the need for balancing on one leg, offered an easy and enjoyable experience as well as the sensation of climbing when using both hands and feet. However, larger individual differences in the experience were observed compared to the case of hands-only climbing. While the inconsistency between hand and foot motions in the real and virtual spaces did not cause a significant sense of discrepancy, the absence of force feedback from the hands and feet led to a perceptual mismatch for players. Additionally, this paper described movements that are possible in real climbing but impossible in virtual climbing in place, and discussed potential solutions for investigation in future studies.
Data availability statement
The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.
Ethics statement
Ethical approval was not required for the studies involving humans because the experiment involved negligible risk and was conducted in accordance with the principles of the Declaration of Helsinki. The studies were conducted in accordance with the local legislation and institutional requirements. The participants provided their written informed consent to participate in this study.
Author contributions
TM: Methodology, Supervision, Conceptualization, Validation, Data curation, Writing – review and editing, Investigation, Resources, Formal Analysis, Writing – original draft, Funding acquisition, Project administration, Visualization. SK: Investigation, Software, Data curation, Writing – review and editing, Methodology.
Acknowledgements
The authors would like to thank Ryota Yamamoto for assisting in the development of the software.
Conflict of interest
The author(s) declared that this work was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Generative AI statement
The author(s) declared that generative AI was not used in the creation of this manuscript.
Any alternative text (alt text) provided alongside figures in this article has been generated by Frontiers with the support of artificial intelligence and reasonable efforts have been made to ensure accuracy, including review by the authors wherever possible. If you identify any issues, please contact us.
Publisher’s note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
ReferencesAchibetM.GirardA.TalvasA.MarchalM.LécuyerA. (2015). “Elastic-arm: human-Scale passive haptic feedback for augmenting interaction and perception in virtual environments,” in 2015 IEEE virtual reality (VR), 63–68. 10.1109/VR.2015.7223325Al MaimaniA.RoudautA. (2017). “Frozen suit: designing a changeable stiffness suit and its application to haptic games,” in Proceedings of the 2017 CHI conference on human factors in computing systems (CHI '17), 2440–2448. 10.1145/3025453.3025655AsjadN. S.AdamsH.ParisR.BodenheimerB. (2018). “Perception of height in virtual reality,” in Proceedings of the 15th ACM symposium on applied perception, 1–8. 10.1145/3225153.3225171BirnstielS.OberdorferS.LatoschikM. E. (2022). “Stay safe! safety precautions for walking on a conventional treadmill in VR,” in 2022 IEEE conference on virtual reality and 3D user interfaces abstracts and workshops (VRW), 732–733. 10.1109/VRW55335.2022.00217CoutinhoC. (2022). “Arm-swinging movement,” in Unity® Virtual Reality Development with VRTK4 (Berkeley, CA: Apress), 171–176. 10.1007/978-1-4842-7933-5_11Crytek (2021). The Climb2. Available online at: https://www.meta.com/experiences/the-climb-2/2617233878395214/ (Accessed July 24, 2025).DiazD.VernonL. G.SkerrittE.O’NeilJ.DuchowskiA. T.VolonteM. (2024). “A comparative study of omnidirectional locomotion systems: task performance and user preferences in virtual reality,” in The 17th ACM SIGGRAPH conference on motion, interaction, and games, 1–11. 10.1145/3677388.3696326GonçalvesG.CoelhoH.MonteiroP.MeloM.BessaM. (2022). Systematic review of comparative studies of the impact of realism in immersive virtual experiences. ACM Comput. Surv.55 (6), 1–36. 10.1145/3533377HiraoY.NarumiT.ArgelaguetF.LécuyerA. (2024). Revisiting walking-in-place by introducing step-height control, elastic input, and pseudo-haptic feedback. IEEE Trans. Vis. Comput. Graph30 (7), 3210–3223. 10.1109/TVCG.2022.322817137015485KosmallaF.ZennerA.SpeicherM.DaiberF.HerbigN.KrügerA. (2017). “Exploring rock climbing in mixed reality environments,” in Proceedings of the 2017 CHI conference extended abstracts on human factors in computing systems, 1787–1793. 10.1145/3027063.3053110KosmallaF.ZennerA.TaschC.DaiberF.KrügerA. (2020). “The importance of virtual hands and feet for virtual reality climbing,” in Extended abstracts of the 2020 CHI conference on human Factors in computing systems, 1–8. 10.1145/3334480.3383067LaiC.McMahanR. P.HallJ. (2015). “March-and-Reach: a realistic ladder climbing technique,” in 2015 IEEE symposium on 3D user interfaces (3DUI), 23–24. 10.1109/3DUI.2015.7131719McCulloughM.XuH.MichelsonJ.JackoskiM.PeaseW.CobbW. (2015). “Myo arm: swinging to explore a VE,” in Proceedings of the ACM SIGGRAPH symposium on applied perception, 107–113. 10.1145/2804408.2804416MitsudaT. (2013). Pseudo force display that applies pressure to the forearms. Presence22 (3), 191–201. 10.1162/PRES_a_00150MitsudaT.KugeS.WakabayashiM.KawamuraS. (2002). Wearable force display using a particle mechanical constraint. Presence11 (6), 569–577. 10.1162/105474602321050703MouattB.SmithA. E.MellowM. L.ParfittG.SmithR. T.StantonT. R. (2020). The use of virtual reality to influence motivation, affect, enjoyment, and engagement during exercise: a scoping review. Front. Virtual Real.1, 1–2020. 10.3389/frvir.2020.564664NagaoR.MatsumotoK.NarumiT.TanikawaT.HiroseM. (2018). Ascending and descending in virtual reality: simple and safe system using passive haptics. IEEE Trans. Vis. Comput. Graph24 (4), 1584–1593. 10.1109/TVCG.2018.279303829543176NilssonN. C.SerafinS.NordahlR. (2013). “The perceived naturalness of virtual locomotion methods devoid of explicit leg movements,” in Proceedings of motion on games (MIG ’13), 155–164. 10.1145/2522628.2522655ProskeU.GandeviaS. C. (2012). The proprioceptive senses: their roles in signaling body shape, body position and movement, and muscle force. Physiol. Rev.92 (4), 1651–1697. 10.1152/physrev.00048.201123073629SalzmanJ.LiJ.YangB.FeinerS. K. (2024). “VHard: an XR UI for kinesthetic rehearsal of rock climbing moves,” in IEEE international symposium on mixed and augmented reality adjunct (ISMAR-Adjunct),638–639. 10.1109/ISMAR-Adjunct64951.2024.00188SchulzP.AlexandrovskyD.PutzeF.MalakaR.SchöningJ. (2019). “The role of physical props in VR climbing environments,”Proc. 2019 CHI Conf. Hum. Factors Comput. Syst. (CHI '19), 183, 1–13. 10.1145/3290605.3300413SlaterM.UsohM.SteedA. (1994). “Steps and ladders in virtual reality,” in Virtual reality software and technology, 45–54. 10.1142/9789814350938_0005TiatorM.GeigerC.DewitzB.FischerB.GerhardtL.NowottnikD. (2018). “Venga! climbing in mixed reality,”First international symposium on amplifying capabilities and competing in mixed realities (SHS ’18) in Proceedings of the first superhuman sports design challenge, 9, 1–8. 10.1145/3210299.3210308van BeersR. J.SittigA. C.Denier van der GonJ. J. (1998). The precision of proprioceptive position sense. Exp. Brain Res.122 (4), 367–377. 10.1007/s0022100505259827856
Edited by:Patrick Bourdot, Université Paris-Saclay, France
Reviewed by:Razeen Hussain, University of Genoa, Italy
Brigida Bonino, Istituto di Matematica Applicata e Tecnologie Informatiche Enrico Magenes Consiglio Nazionale delle Ricerche Sezione di Genova, Italy