Front. Virtual Real.Frontiers in Virtual RealityFront. Virtual Real.2673-4192Frontiers Media S.A.152656710.3389/frvir.2025.1526567Virtual RealityOriginal ResearchVisual assistance may impede sensorimotor learning during gamified rehabilitation exercisesHuang et al.10.3389/frvir.2025.1526567HuangYuxuan1*ElangovanNaveen2KonczakJürgen2InterranteVictoria1*1Department of Computer Science and Engineering, University of Minnesota, Minneapolis, MN, United States2School of Kinesiology, University of Minnesota, Minneapolis, MN, United States
Edited by:Adriana Salatino, Royal Military Academy, Belgium
Reviewed by:Stephanie Elena Crowe, University of Canterbury, New Zealand
This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
Regular exercise is critical to post-stroke recovery, but can be frustrating and difficult for patients with limited motor capabilities. Robotic assistance devices are being used to support this process, but due to cost and accessibility concerns some researchers have drawn inspiration from virtual reality and proposed the use of what we call “visual-only assistance”, in which a patient’s physical movements are mapped into game-oriented visual feedback modified towards greater success in the gameplay objectives. Our concern is that the motivational benefits it provides may come at the cost of reduced sensorimotor learning, which could ultimately be counterproductive to the recovery process. To explore these concerns, we conducted a between-subjects study with 24 participants to examine how two types of visual-only assistance affect short-term proprioceptive skill learning in a motor training game involving airplane steering with wrist rotation. One group experienced “attractor assistance”, in which the airplane was continuously displaced toward an ideal position and orientation. The other experienced “tunnel assistance”, in which direct user control was maintained unless straying too far, triggering an invisible barrier. We hypothesized that motor learning would be more impeded with attractor assistance due to the constant mapping variation between physical movements and visual feedback, but our experiment found that motor learning did not materialize in either condition, suggesting that substituting intermittent visual assistance for constant visual assistance is insufficient to guarantee superior motor learning outcomes and that, pending further investigation, the use of visual assistance for proprioceptive training should continue to be regarded with caution.
serious gamesaccessibilitysensorimotor rehabilitationmotor trainingvisual feedback manipulationNational Science Foundation10.13039/100000001section-at-acceptanceVirtual Reality and Human Behaviour1 Introduction
Motor rehabilitation is essential for the functional recovery and independence in people after a stroke (Langhorne et al., 2011; Dobkin, 2004). Research has shown that the repetition of movements can effectively assist in the re-establishment of the motor control pathway (Bütefisch et al., 1995; Kawahira et al., 2004) through reinforcement of sensorimotor feedback and proprioception, which makes it a core component of motor rehabilitation therapies.
However, the plain repetition of movements that are out of context can be tedious and unengaging. Many existing studies suggest that nonadherence to rehabilitation therapy is one of the main factors limiting the effectiveness of rehabilitation training (Dishman and Ickes, 1981; Sluijs et al., 1993). Therefore, there has been a growing trend to incorporate gamification elements to rehabilitation therapies to improve engagement and adherence (Burke et al., 2009; Mubin et al., 2020), usually with adjustable difficulty to accommodate the varying ability of stroke patients (Karime et al., 2013). Nevertheless, acute stroke patients can still find it too challenging to complete the gamified tasks unassisted due to their very limited motor capabilities, and making the tasks too simple can defeat the purpose of an engaging and motivating experience.
To account for this, dedicated robotic devices are designed and developed to assist the patients in executing the movements they intend to make by sensing and magnifying their own efforts towards a static or dynamic goal state, defined in the context of an appropriately designed gamified task (Marini et al., 2017; Veerbeek et al., 2017; Hussain et al., 2020). The use of robotic assistance is proved to be effective and reliable in facilitating motor recovery (Veerbeek et al., 2017; Hussain et al., 2020) but many complicated factors remain to be addressed, most notably the access to these devices being limited to onsite therapy sessions.
Since regular motor exercise is key to motor rehabilitation, researchers have been considering the deployment of gamified recovery support software (Rand et al., 2014; Baranyi et al., 2014), including games that provide what we call “visual assistance” as a potential compliment to the use of robotic assistance so that patients can perform motor exercises from home while allowing the therapists to track gameplay duration (adherence) and performance (skills improvement) remotely to support them in optimally steering the recovery process (Dukes et al., 2013; Ferdous et al., 2022). With visual assistance, a patient’s physical movements are mapped into game-oriented visual feedback in a way that is either overtly or covertly modified towards greater success in the gameplay objectives, which achieves motivational benefits similar to robotic assistance but without requiring actual mechanical devices.
While such visual manipulation can be implemented in different forms and is commonly used in virtual reality (VR) applications for various purposes (Poupyrev et al., 1996; Frees and Kessler, 2005; Wentzel et al., 2020) to improve usability, in rehabilitation exercises the ultimate goal is sensorimotor learning, which requires the patients to process sensory information and learn how to control their movements based on the sensory feedback. Our concern is that a method of visual assistance that applies a constantly-varying manipulation to the mapping between a user’s physical movement and the visual feedback they receive about that movement could be counter-productive to sensorimotor learning, because the same movement of the patient can lead to different outcomes depending on what the assistance is doing, which, in spite of motivational benefits, makes it harder for them to appreciate how they are controlling their physical movements. Therefore, in this study we hypothesize that a visual assistance method that preserves the movement-to-feedback mapping to greater extent can better accomplish the goal of avoiding patient frustration while enabling better motor training outcomes by affording more robust sensorimotor learning opportunities.
2 Related work2.1 Proprioceptive training and sensorimotor rehabilitaion
Neurological conditions such as stroke can cause motor impairment as well as proprioceptive dysfunction (Carey et al., 1996; Kenzie et al., 2014). It is well-established that training of repetitive movements with volitional effort is an effective intervention to aid in stroke rehabilitation (Bütefisch et al., 1995; Kawahira et al., 2004; Tacchino et al., 2016), and sensorimotor training can improve proprioceptive functions, which further leads to improved motor performance (Elangovan et al., 2017; Elangovan et al., 2018). The rationale is that the relevant neuro-connections can be reorganized through the proprioceptive training and the brain can relearn the mapping between the motor inputs and outcomes. It is worth noting that measurable improvement in proprioceptive performance can not only be observed in stroke patients but also in healthy populations even after short-term training (Aman et al., 2015; Ostry and Gribble, 2016; Cuppone et al., 2018), so as a matter of ethics, researchers avoid using clinical populations for preliminary investigations, where results from healthy participants can be sufficient to inform the next steps in the work (Elangovan et al., 2017; 2018; Wang et al., 2021).
2.2 Robotic assistance for rehabilitation
In cases where the participants are unable to complete the training tasks due to limited motor capability, the standard solution is to assist their movement using exoskeletons or other robotic devices (Veerbeek et al., 2017; Hussain et al., 2020). Many different robotic systems have been developed for this purpose and proved to be effective. Masiero et al. created a simple 3-DOF wire-based robot which uses motors to control the lengths of the wires and pull the patients’ upper limbs to assist their motion (Masiero et al., 2007). Studies have demonstrated that early sensorimotor training using this device can speed up the recovery. Montagner et al. developed a force feedback exoskeleton for the right arm, and achieved positive results in terms of rehabilitation outcomes and user feedback in a pilot study consisting of various motor training tasks (Montagner et al., 2007). A wrist rehabilitation robot developed by Masia et al. can apply torques to either assist or resist the rotation of the wrist and is used in both motor control studies and rehabilitation (Masia et al., 2009). In Borboni et al. (2016), designed a hand rehabilitation glove capable of assisting personalized combinations of finger movements and showed its efficacy with experimental results on 126 stroke patients. However, a major constraint of robotic assistance is that the robotic devices are only available in hospitals or clinics, which significantly limits their accessibility.
2.3 Visual feedback manipulation
In recent years, many elements of VR have also been incorporated into motor training (Laver et al., 2017; Hao et al., 2022; Demeco et al., 2023; Khokale et al., 2023), including the introduction of visual manipulation in an assistive manner to support motor training when robotic assistance is unavailable.
In Virtual Reality, things are not constrained by the laws of physics, and many interaction techniques have been proposed that manipulate the visual feedback of users’ spatial input in a way that assists with the completion of specific tasks in the virtual environment. For example, “go-go” (Poupyrev et al., 1996) and its variants (Wentzel et al., 2020) allow users to work with objects out of reach by scaling up the displayed movements when they exceed 2/3 of the user’s arm length. Precise and rapid interaction through scaled manipulation (PRISM) is a method that scales down the displayed movement to improve stability when performing fine movements (Frees and Kessler, 2005; Frees et al., 2007). There are also studies that implicitly modify users’ task performance in VR towards greater success, such as in the ball throwing task described in (Murakami et al., 2024), where the trajectory of the ball is pushed towards the target. The methods discussed in these studies share a similar goal to robotic assistance in that they aim to improve engagement and motivation by enabling users to perform certain tasks more easily or accurately than what they are actually capable of.
Some work has also incorporated the idea of visual assistance into the context of stroke rehabilitation and motor training. In the “duck duck punch” rehabilitation game developed by Dukes et al., the movements of stroke patients are visually amplified to account for their limited range of motion (Dukes et al., 2013). Similarly, Ferdous et al. used scaled movement to improve the reach of Parkinson patients in a VR rehabilitation game of smashing balloons (Ferdous et al., 2022). The same idea has also been used in lower-extremity rehabilitation training, such as in (Chowdhury et al., 2019), where rotational gain is applied to patients’ leg movement to assist them in a rehabilitation game involving kicking a ball. However, all the above-mentioned studies focused on task completion and motivational benefits without explicitly considering the introduction of visual assistance affects training outcomes. In Ross et al. (2023), demonstrated that the “duck duck punch game” combined with aerobic exercise can improve the upper-extremity function of stroke patients, but that training was targeting range-of-motion, i.e., how far one can move, rather than sensorimotor learning, i.e., how accurate one can perceive and execute movements. To the best of our knowledge no study has yet specifically considered the impact of visual assistance on the acquisition of sensorimotor skills.
Therefore, in this study we aim to examine the sensorimotor learning in a motor training game when visual assistance is present, and explore whether a variant of visual assistance that introduces manipulation in an “as-needed” fashion could effectively reduce the potential disruption of sensorimotor learning and achieve a superior training outcome compared to traditional visual assistance that is “always on” and employs a constantly varying mapping between action and outcome.
3 Methods
We conducted a between-subject study to compared the proprioceptive training outcomes and user experience between two implementations of visual assistance, one that constantly manipulates the visual feedback towards greater in-game success, and one that becomes active only when the user is doing exceptionally poorly. Based on our concern that visual feedback manipulation may disrupt proprioceptive learning, we hypothesized that an “as-needed” variant of visual assistance might enable better motor training improvement due to the reduced amount of visual manipulation being introduced, but might also potentially sacrifice usability due to the users performing more poorly. We did not consider the need for a control condition in which no assistance was applied, because we were focused on addressing practical situations in which some sort of assistance, either visual or robotic, is needed. However, we recognize in retrospect that including such a condition would have been wise.
3.1 Apparatus and materials3.1.1 Experimental equipment
We used a wrist rehabilitation robot built by the Italian Institute of Technology (Masia et al., 2009) to track the user’s wrist movements in our study. This device has three degrees of freedom in movement, corresponding to pronation/supination, flexion/extension, and abduction/adduction of the human wrist. To use this device, participants sat on a stool, rested their arm on the device’s armrest, and grabbed the stick of the wrist robot with their dominant hand. In this study, the robot did not provide any mechanical assistance. The reason why we used a wrist robot as the tracker, as opposed to a smartphone or VR controllers was because in a future study we wanted to compare visual and robotic assistance with the hardware as a controlled variable, and this study served as a preliminary study to explore possible visual assistance design. The robot was connected to a desktop computer running Linux Mint (Version 18) and visual feedback was displayed on a monitor that was mounted at eye height on a narrow table approximately 40 cm in front of the participants. See Figure 1 for an illustration of the experimental setup.
A person playing the motor training game using the wrist robot and collage of the different scene variations in the game.
A person using a wrist-controlled interface to play a plane flying game, and different stylized virtual environments of the game.3.1.2 Motor training game
We developed a motor training game using Unity3D. The game was inspired by air racing, in which the player controls a virtual plane flying forward on its own at constant speed. The plane’s rotation is linked to the user’s wrist rotation. The goals are to steer the plane to fly through all gates and match the rotations visualized as outlines of plane’s rotational pose naturally maps to the 3-DOF of human wrist, and by setting the location of the gates and the orientation of the outlines we can define the required wrist rotations in an intuitive manner. The game has 20 procedurally generated levels, each consisting of 15 gates, where the forward displacement between adjacent gates is 15 units and the horizontal and vertical displacements are randomized within ±[1,3] and ±[0.5,1] units, respectively. The target rotation is randomized within [-45°, 45°]. The purpose of the randomization to ensure non-repetitive gameplay, yet consistency in difficulty across levels. The forward speed of the plane is 4.5 units/second, meaning that each level takes 50 s to complete. We also created five scene variations shown every four levels to reduce visual fatigue (see Figure 1).
Whenever the plane reaches a gate, a score is calculated based on the accuracy of its position and orientation with respect to the target outline. Table 1 shows the scoring system. The players were shown how many additional points they earned after passing each gate, and the total score for each level was shown after the level’s completion.
Table of all possible situations when the plane is passing a gate and their corresponding scores.
Performance
Pass without Collision
Pass with Collision
Miss
Roll Diff ≤ 10°
3
2
−1
10° < Roll Diff ≤ 20°
2
1
−1
20° < Roll Diff ≤ 30°
1
0
−1
Roll Diff ≥ 30°
0
−1
−1
The score is determined by whether the plane passes the gate with or without collision, and how accurate the plane matches the target rotation.
3.1.3 Visual assistance
We implemented visual-only counterparts of two commonly used mechanical assistance schemes (Basteris et al., 2014): 1) The moving attractor (Chang et al., 2007; Lambercy et al., 2011; Squeri et al., 2013), in which an assistive force constantly attracts the end-effector towards a target trajectory and the amount of assistance is modulated by varying a stiffness parameter, e.g., proportional to the distance between the end-effector and the target, and 2) the virtual tunnel (Casadio et al., 2009), in which assistive forces are introduced only after the error exceeds a certain threshold.
Specifically, the two types of visual assistance we implemented for the game are as follows,
• Attractor: in which the plane is constantly attracted towards a predefined optimal trajectory and its roll is adjusted toward the required roll for the upcoming gate.
• Tunnel: in which the plane is prevented from deviating beyond a certain distance from a predefined optimal trajectory or beyond a certain angular displacement from the target orientation at the next gate
To avoid over-assistance, the assisted movement for the attractor is proportional to: 1) the distance from the plane location to the closest point on the trajectory (d1), and 2) the distance in the forward direction from the plane to the next gate (d2). Specifically, the assisted movement increases as the plane deviates farther away from the optimal trajectory i.e., when the player is not doing well, and as the plane approaches the gates, i.e., when there is less time left to adjust. See Figure 2 for a visual illustration. Specifically, the attractive force f, i.e., how much the position/rotation of the plane is adjusted towards the target position/rotation in one unit of time is calculated as f=k×(d1+(d−d2)), where d is the distance between adjacent gates, and k is the scaling factor. Similarly, for tunnel assistance the threshold of allowable positional and rotational error gets smaller at a specific rate (s) until reaching a minimum value (minR) as the plane approaches the gate, allowing more freedom when the gate is far ahead and less room for error when the gate is near. The allowable error threshold at any given distance d2 to the next gate can be calculated as thres=minR+s×d2. See Figure 2 for visual illustration of tunnel assistance. In both cases of assistance, the optimal trajectory is displayed as a semi-transparent curve in the game for reference.
Illustration of “Attractor” and “Tunnel” visual assistance. The attractive force increases as the plane deviates from optimal trajectory (d1 increases) and approaches the gates (d2 decreases). Similarly, the radius of the tunnel decreases as the plane approaches the gates.
Diagram comparing two guidance strategies: “Attractor” using force vectors toward a trajectory and “Tunnel” using boundaries around a path with gates and pylons marked.
In order to optimize the comparison between the two assistance methods, we conducted a pilot study with four participants from our research group to tune the parameters of the positional and rotational assistance as they played the game. The participants were guided to adjust the strength of the assistance so that the game felt moderately challenging and rewarding enough that they would be willing to play it for a long time. We asked them to tune one parameter at a time, and they went through each possible permutation of parameter tuning order 3 times with different random starting values. During the process, the participants used their dominant hand to control the airplane’s position and orientation via the wrist robot, and used their non-dominant hand to press the up/down arrow keys on the keyboard to increase/decrease the amount of assistance being provided. For our formal study, we used the median of the final values chosen by the four participants in the pilot study.
3.2 Participants
We recruited 24 participants (13M, 11F) between the ages of 18–34 from our University community, using posters and online platforms. Half (6M, 6F) were randomly assigned to the attractor condition, and half (7M, 5F) to the tunnel condition. While consistent with the number of participants used in prior related work Elangovan et al. (2017), Elangovan et al. (2018) we note that this is significantly less than the 128 participants recommended by the power analysis conducted on G*Power 3.1.9.7 for an independent t-test with a medium expected effect size (0.5) and alpha = 0.05 to achieve a power of 0.8. Due to the preliminary nature of our work, we focused our investigations on healthy participants with no known impairment in upper limb motion and normal or corrected-to-normal vision, similar to prior related work (Elangovan et al., 2017; Elangovan et al., 2018; Wang et al., 2021). After the study participants were compensated with a $30 gift card from an online retailer. The study was approved by the University of Minnesota Institutional Review Board (IRB).
3.3 Proprioceptive Measurements
The goal of sensorimotor learning is to improve people’s ability ability to perceive and precisely control their physical movements. To measure the sensorimotor learning outcomes, we used the same instruments as in (Elangovan et al., 2017; Elangovan et al., 2018; Wang et al., 2021) were adopted, which consist of a proprioceptive acuity test and a motor performance test.
The proprioceptive acuity test measured the sensitivity with which a participant can discriminate the position of their wrist, i.e., the proprioceptive threshold (PT). Across 30 different trials, the participant’s dominant hand was passively moved from a starting angle, first to one one angle and back, and then to a second angle and back. One of the two angles always represented a 15° displacement because that was the maximum rotation required in the training game. The other angle was always less than 15°. The participant was asked to indicate verbally which displacement was greater. A correct response would cause the difference between the two angles to decrease for the next trial, and an incorrect response would cause it to increase. The final measurement of proprioceptive acuity was computed using the psi-marginal method (Prins, 2013).
The motor performance test measured the accuracy with which a participant could actively replicate a passively demonstrated wrist pose, called the movement accuracy error (MAE). Lower MAE corresponds to better motor performance. To obtain wrist MAE, the wrist robot rotated the participant’s wrist to a flexion angle of 10°, held that pose for 2s and then returned to the starting position. subsequently, participants were instructed to actively rotate their wrist to match the angle they just experienced. When they felt they had assumed the correct position, they held still and gave a verbal signal. The test consisted of 10 trials and MAE was computed as the mean of the errors across all trials.
3.4 Procedure
After obtaining informed consent, we collected baseline (pre-training) measurements of participants’ proprioceptive threshold (PT) and movement accuracy error (MAE). After this, participants performed the Proprioceptive Training task, which consisted of playing the airplane flying game for 20–30 min. They then filled out several short surveys regarding their user experience.
After a 10-min break, we asked participants to complete the Proprioceptive Measurements again. Following this, participants filled out the a demographics questionnaire.
Last came our behavioral test of participants’ gameplay enjoyment. We told participants that they would need to wait for an additional 5 min for us to “upload the data” from the experiment, and that they could feel free to continue playing the game while they waited. We then left the area, but surreptitiously monitored whether the participant chose to play the game for their own pleasure. Finally, we debriefed the participants and concluded the study. See Figure 3 for a flowchart of the study procedure.
Flowchart of the study procedure.
Flowchart of a study protocol including consent, pre- and post-training proprioceptive tests and separate paths for two training conditions, surveys, and post-study debriefing.3.5 Outcome measurements
We quantified proprioceptive improvement as the relative difference (prescore−postscore)/prescore for both PT and MAE was measured pre and post training. We also assessed participants’ in-game performance improvement by assessing the change in their in-game scores between successive levels.
We used the raw “NASA Task Load Index” (NASA TLX) survey (Hart and Staveland, 1988) to assess perceived effort and a customized usability survey to assess enjoyment and awareness of assistance on a five-point-likert scale. Specifically, enjoyment was measured as the extent of agreement with the statement “I enjoyed playing this game.” and awareness was measured with another three statements: (1) “Sometimes I felt that there were unseen forces affecting how the plane flew.” which aimed to capture participants’ awareness of manipulation; (2) “Sometimes I felt as if the plane drifted off-course for no reason.” which was intended as a decoy question; and (3) “Sometimes I felt as if the plane was staying on course all by itself.” which sought to assess the awareness of assistance. The usability survey is attached in Supplementary Material. Persistence (the number of participants who chose to keep playing the game for their own pleasure while waiting for the data upload) and total in-game score were also recorded as secondary measures.
We also quantified the extent of users’ reliance on the assistance to check if they were reducing their own effort over time, as it would be a concern if, in the presence of assistance, people developed a tendency to stop trying and just let the game play itself. We used different assistance metrics for the Attractor and Tunnel conditions due to the difference in their mechanisms. For the tunnel assistance, reliance was quantified as the proportion of the time that assistance was active during the total time spent in a level. For the attractor assistance, which was always active, reliance on positional and rotational assistance was quantified as the total distance and the total angle manipulated by the assistance in each level.
3.6 Hypotheses
We hypothesized that proprioceptive learning outcomes would be facilitated to a greater extent when the assistance preserves a deterministic mapping from physical movement to visual feedback over a longer period of time. We also hypothesized that in-game performance would show greater improvement when users are assisted less, because there would be more opportunity to learn. At the same time, we hypothesized that users would have a better subjective experience, stronger in-game performance and greater persistence in gameplay when using a mode of assistance that helped them perform consistently better in a surreptitious way. Finally, we aimed to validate that visual assistance is not counterproductive to the elicitation of effort by demonstrating that users are not increasing their reliance on the assistance over time. These hypotheses led to the following predictions:
• H1: Use of tunnel assistance will be associated with superior learning outcomes compared to attractor assistance, specifically in terms of greater proprioceptive improvement (H1.A) and greater in-game improvement (H1.B).
• H2: Use of attractor assistance will be associated with better user experience compared to tunnel assistance, specifically in terms of more positive usability survey responses (H2.A), higher in-game scores (H2.B) and more frequent persistence (H2.C) in gameplay beyond the required amount.
• H3: In both conditions, user reliance on assistance will decrease over time.
4 Results
For all of our analyses, we used parametric tests (T-tests) when Shaprio-Wilk tests showed that the data were normal and non-parametric tests (Mann-Whitney, Wilcoxon) when the assumption of normality was violated.
4.1 Proprioceptive and in-game improvement
Proprioceptive improvement is visualized in Figure 4a. A Mann-Whiteney Test showed no significant difference in relative improvement in PT between the two conditions (U=89.0,p=0.34), and an independent T-test showed no significant difference in MAE improvement (t=0.63,p=0.53).
(a) The relative post-training performance improvement of proprioceptive threshold (PT) and movement accuracy error (MAE) in the two assistance conditions. Values above zero correspond to improvement. Pre-post training performance comparison of (b)PT and (C)MAE for each participant. Smaller values correspond to better performance. (a) Relative Improvement in proprioceptive threshold and movement accuracy (b) Pre-post training performance of proprioceptive threshold (c) Pre-post training performance of movement accuracy.
Composite figure showing: A) bar plots of relative motor improvement for two strategies across two measures; B and C) scatter plots of how pre- and post-training results compare.
Figures 4b,c show the pre-post statistics of PT and MAE for each participant. Paired T-tests showed no statistically significant improvement in PT after training in either the attractor (t=0.58,p=0.57) or tunnel conditions (t=−0.27,p=0.79). A Wilcoxon signed test also showed no significant improvement in MAE after training in the attractor condition (W=30.0,p=0.52). However, a paired T-test indicated a significant increase in MAE after training in the tunnel condition (t=−2.49,p=0.03), corresponding to a worsening movement accuracy. In both conditions, we found that post-training MAE was strongly positively correlated with pre-training MAE. PT on the other hand showed a strong tendency to regress towards the mean, i.e., users who started out with the poorest scores improved while those who started out with the highest scores showed a decrease. Overall, the analyses of the proprioceptive measures do not support H1.A.
To compare In-game improvement between our two conditions, we tested the parallelism of the nonparametric ANCOVA and found h=1.64759,p<0.001, which rejects the null hypothesis that the two curves are parallel. In Figure 5 it can be observed that in-game scores increased more steeply in the tunnel condition, which is consistent with H1.B. However, this finding should be interpreted with caution because of the ceiling effects that can be seen in the in-game scores in the attractor condition, i.e., because those scores were high from the beginning, there was little room for further improvement.
Test of parallelism in the in-game improvement between the two assistance. Overlapping samples are illustrated with increased opacity.
Line chart showing user scores across twenty levels, with Attractor group maintaining higher performance than Tunnel group, indicated by blue and orange lines respectively.4.2 Factors affecting proprioceptive learning
The histograms of participants’ responses to the three awareness questions are shown in Figure 6. The responses from “strongly disagree” to “strongly agree” were mapped to one through five for the data analyses. Mann Whiteney U test found no significant difference between conditions for any of the three responses: U=74.0,p=0.93 for “unseen forces”, U=78.5,p=0.69 for “all by itself”, and U=58.0,p=0.39 for “drifting off-course”.
Participants’ response to the exit survey questions: “Sometimes I felt that there were unseen forces affecting how the plane flew”, “Sometimes I felt as if the plane drifted off-course for no reason.“, and “Sometimes I felt as if the plane was staying on course all by itself”.
Three grouped bar charts displaying user agreement with statements on how their control of the plane was affected, comparing Attractor and Tunnel groups across five Likert-scale categories.
Few participants in either condition indicated a strong awareness of the presence of visual assistance. However, a Pearson correlation analysis showed a significant negative correlation between awareness of assistance and MAE improvement in the attractor condition (p=0.04,r=−0.61), as well as a negative correlation between awareness of assistance and PT improvement (p=0.10,r=−0.50), although the significance of the latter did not reach the 0.05 threshold. This indicates that, in the attractor condition, users who were more likely to report the presence of attractor assistance were less likely to experience proprioceptive learning after gameplay. See Figure 7 for an illustration of this relationship.
Correlation between participant’s awareness of assistance and their relative improvement in (a) proprioceptive threshold and (b) movement accuracy. Overlapping samples are illustrated with increased opacity. (a) Correlation between assistance awareness and proprioceptive threshold improvement (b) Correlation between assistance awareness and movement accuracy improvement.
Two scatter plots showing negative correlation between assistance awareness and relative improvement for both PT and MAE, with linear regression lines and shaded confidence intervals for Attractor and Tunnel groups.
We also found a significant positive correlation in the tunnel condition between ratings of mental exhaustion on the NASA TLX and a sense of the presence of unseen forces affecting the airplane’s behavior (p=0.01,r=0.70) and a perception of the airplane drifting off course (p<0.01,r=0.82), as well as a significant positive correlation between sensing the presence of unseen forces and perceiving the plane to be drifting off course (p=0.01,r=0.69). This suggests an overall negative impact of the invisible tunnel assistance.
4.3 In-game scores and usability
Figure 8 gives a statistical summary of the an illustration of usability, enjoyment and in-game performance scores. Mean overall NASA TLX scores were 51.8±13.4 in the attractor condition, and 57.2±9.8 in the tunnel condition, where a smaller value corresponds to lower perceived workload. Enjoyment ratings (higher is better) were 4.91±0.3 in the attractor condition, and 4.50±0.5 in the tunnel condition, after the removal of a single outlier with a score of two in the attractor condition. Average total in-game scores (out of a maximum possible 900) were 876.9±30.5 in the attractor condition, and 778.7±57.3 in the tunnel condition.
Statistical summary of participants’ (a) Total in-game scores, (b) NASA TLX scores, (c) and histogram of responses to “I enjoyed playing this game” in the two assistance conditions (a) Total in-game scores (b) NASA TLX Scores (c) “I enjoyed playing this game”.
Three-panel figure: a) box plots of in-game scores by condition, b) box plots of task load index (TLX), and c) bar chart of user enjoyment ratings by condition, comparing Attractor and Tunnel groups.
A Mann-Whitney U Test showed significant differences in the total in-game scores (U=134.0,p<0.001) and enjoyment (U=93.0,p=0.04), between the two assistance conditions, with better in-game performance and enjoyment with the attractor assistance. These results support H2.A and H2.B. Despite the difference, the in-game performance and enjoyment were high in both conditions. The difference in TLX scores was not significant according to an independent T-Test (t=−1.13,p=0.27).
Only one participant in each condition chose to play the game while waiting for their data to be uploaded, so there was no difference in persistence between the two assistance conditions. The result does not support H2.C. During debriefing, most participants reported that fatigue was the main reason why they did not want to keep playing, even though they did enjoy the game.
4.4 Reliance on assistance
We performed a Mann-Kendall test for each participant to see if there was an increasing/decreasing trend in reliance on assistance over the training period. The results are shown in Table 2. For the majority of the participants, there was either no trend in reliance or the trend was decreasing. This result supports H3.
Number of participants with decreasing/increasing or no trend in their reliance on the assistance.
Reliance
Decreasing
No trend
Increasing
Attractor Pos
2
9
1
Attractor Rot
3
8
1
Tunnel
5
6
1
Attractor Pos. and Rot. refer to the positional and rotational components of the attractor assistance, respectively.
5 Discussion
The most notable finding from our study is the failure to see proprioceptive improvement after training in either of our visual assistance conditions. Given that prior related studies (Elangovan et al., 2017; 2018; Wang et al., 2021) had consistently observed some proprioceptive learning after a similar training process, we feel that the most likely explanation for this finding is that both the attractor and tunnel assistance, as implemented, interfered with motor learning. While we would need to do a follow-up study directly comparing proprioceptive learning after training with and without visual assistance to robustly confirm this hypothesis, we advise in the meantime that the use of visual-only assistance in proprioceptive training for stroke rehabilitation should be treated with caution. We had hypothesized that an intermittent assistance model like our tunnel method could better support proprioceptive training by preserving a consistent mapping from motor input to visual feedback over a greater period of time, but our data show that this hope was not borne out. Given that the tunnel condition was associated with less enjoyment, and a higher awareness of this assistance was associated with stronger perception of the plane “drifting off-course” as well as worsening movement accuracy error, it is possible that the experience of encountering invisible guardrails was non-intuitive and therefore unhelpful to the proprioceptive training. In future training development efforts, it might be better to explicitly show the guardrails rather than rendering them invisibly, as well as exploring other possible approaches to providing assistance that are less likely to interfere with the proprioceptive learning process.
Both assistance methods resulted in satisfactory in-game performance and high subjective enjoyment, successfully fulfilling their assistive purpose, which confirms the positive findings from prior research using visual assistance (Dukes et al., 2013; Ferdous et al., 2022; Chowdhury et al., 2019). However, neither method led to improvements in proprioception. This result highlights an important implication: relying solely on task performance and subjective measures, as commonly done in prior studies, may be insufficient for evaluating the effectiveness of visual assistance techniques in stroke rehabilitation or motor training. It is essential to include motor skill improvement as a key evaluation metric.
Our failure to elicit proprioceptive learning might also be explained by an overall dampening in motor performance after gameplay due to fatigue. The wristbot was heavy and required a lot of physical effort to manipulate. Because of this, few participants opted to continue playing after the end of the experiment. In the attractor condition, users who reported higher TLX physical demand had worse in-game performance and lower post-game PT. However, in the tunnel condition, higher TLX physical demand was associated with a greater improvement in gameplay performance, which is inconsistent with a negative impact of exhaustion.
It is also possible that the game we designed was somehow fundamentally ill-suited to supporting improved sensorimotor learning, independent of the presence of visual assistance. Unfortunately this possibility cannot be ruled out, as we did not include a control condition in our study. However, given the consistency with which motor improvements have been observed in the past after similar periods of engagement with a wide variety of similarly-designed games, we find this unlikely. We also note that there is limited practical value in considering proprioceptive training without some form of assistance. Nevertheless, we recognize that more robust followup work, including a control condition of no assistance, is needed to definitively establish the potentially deleterious impact to sensorimotor learning from providing visual assistance that surreptitiously varies the mapping between physical action and visual feedback in a constantly-changing way.
6 Limitations
Because the experiment did not contain a control condition, we cannot be certain that the lack of motor learning outcomes can be wholly attributed to the use of visual assistance. While we had intended that our experiment would compare a “high” visual assistance condition to a “low” visual assistance condition, our data suggest that our two visual assistance conditions may have each been detrimental in its own way, but we cannot prove that with the data we have and further studies would be needed to explore that possibility.
Also, due to the preliminary nature of our investigations and in accordance with prior norms for working with vulnerable populations, we tested both of our systems on healthy participants. It is possible that the introduction of visual assistance might impact such populations differently. Experiments with vulnerable/protected populations will be necessary in the future, after further preliminary hypothesis testing with healthy participants.
7 Conclusion
In this paper, we set out to investigate the extent to which visual-only assistance, i.e., manipulating the mapping from motor input to visual feedback towards greater in-game success, might potentially hinder sensorimotor learning during gamified proprioceptive training. We developed a motor training game and compared two different implementations of visual assistance: one that operated continuously and used a constantly-varying mapping to persistently manipulate the player’s visual feedback toward showing greater success in gameplay, and one that used an invisible tunnel paradigm to prevent the player from straying too far off course, thereby preserving an unmodified mapping from motor input to visual feedback throughout a greater proportion of the training session. Our hypothesis was that the “tunnel” type of assistance would better support proprioceptive learning, despite providing a potentially inferior user experience. However, our results showed a failure to achieve significant proprioceptive improvement after training with either method. Our results suggest that the use of visual assistance during motor training should be treated with caution, and highlights the need for further investigations to explore the potential impact of visual assistance in proprioceptive training for stroke rehabilitation.
Data availability statement
The datasets presented in this article are not readily available because according to the IRB protocol the dataset will not be made public. Requests to access the datasets should be directed to huan2076@umn.edu.
Ethics statement
The studies involving humans were approved by University of Minnesota Institutional Review Board. The studies were conducted in accordance with the local legislation and institutional requirements. The participants provided their written informed consent to participate in this study.
Author contributions
YH: Conceptualization, Data curation, Formal Analysis, Investigation, Methodology, Software, Visualization, Writing – original draft. NE: Conceptualization, Methodology, Resources, Writing – review and editing. JK: Conceptualization, Resources, Supervision, Writing – review and editing. VI: Conceptualization, Formal Analysis, Methodology, Resources, Supervision, Validation, Writing – review and editing.
Funding
The author(s) declare that financial support was received for the research and/or publication of this article. This project was supported in part by the Minnesota Robotics Institute (MnRI) and by a grant from the National Science Foundation (I-Corps: Robotic system for physical rehabilitation, Award Number: 1745306).
The authors would like to acknowledge the Italian Institute of Technology for providing the wrist robot.
Conflict of interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
The author(s) declared that they were an editorial board member of Frontiers, at the time of submission. This had no impact on the peer review process and the final decision.
Generative AI statement
The author(s) declare that no Generative AI was used in the creation of this manuscript.
Publisher’s note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
Supplementary material
The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/frvir.2025.1526567/full#supplementary-material
ReferencesAmanJ. E.ElangovanN.YehI.-L.KonczakJ. (2015). The effectiveness of proprioceptive training for improving motor function: a systematic review. Front. Hum. Neurosci.8, 1075. 10.3389/fnhum.2014.01075BaranyiR.ReiseckerF.LedererN.GobberM.GrechenigT. (2014). “Wristdroid-a serious game to support and motivate patients throughout their wrist rehabilitation,” in 2014 IEEE conference on biomedical engineering and sciences (IECBES) (IEEE), 786–791.BasterisA.NijenhuisS. M.StienenA. H.BuurkeJ. H.PrangeG. B.AmirabdollahianF. (2014). Training modalities in robot-mediated upper limb rehabilitation in stroke: a framework for classification based on a systematic review. J. neuroengineering rehabilitation11, 111–115. 10.1186/1743-0003-11-111BorboniA.MorM.FagliaR. (2016). Gloreha—hand robotic rehabilitation: design, mechanical model, and experiments. J. Dyn. Syst. Meas. Control138, 138. 10.1115/1.4033831BurkeJ. W.McNeillM.CharlesD. K.MorrowP. J.CrosbieJ. H.McDonoughS. M. (2009). Optimising engagement for stroke rehabilitation using serious games. Vis. Comput.25, 1085–1099. 10.1007/s00371-009-0387-4BütefischC.HummelsheimH.DenzlerP.MauritzK.-H. (1995). Repetitive training of isolated movements improves the outcome of motor rehabilitation of the centrally paretic hand. J. neurological Sci.130, 59–68. 10.1016/0022-510x(95)00003-kCareyL. M.OkeL. E.MatyasT. A. (1996). Impaired limb position sense after stroke: a quantitative test for clinical use. Archives Phys. Med. rehabilitation77, 1271–1278. 10.1016/s0003-9993(96)90192-6CasadioM.GiannoniP.MorassoP.SanguinetiV. (2009). A proof of concept study for the integration of robot therapy with physiotherapy in the treatment of stroke patients. Clin. Rehabil.23, 217–228. 10.1177/0269215508096759ChangJ.-J.TungW.-L.WuW.-L.HuangM.-H.SuF.-C. (2007). Effects of robot-aided bilateral force-induced isokinetic arm training combined with conventional rehabilitation on arm motor function in patients with chronic stroke. Archives Phys. Med. rehabilitation88, 1332–1338. 10.1016/j.apmr.2007.07.016ChowdhuryT.FerdousS.PeckT.QuarlesJ. (2019). “Virtual ability simulation: applying rotational gain to the leg to increase confidence during physical rehabilitation,” in ICAT-EGVE 2019-international conference on artificial reality and telexistence and eurographics symposium on virtual environments.CupponeA. V.SempriniM.KonczakJ. (2018). Consolidation of human somatosensory memory during motor learning. Behav. brain Res.347, 184–192. 10.1016/j.bbr.2018.03.013DemecoA.ZolaL.FrizzieroA.MartiniC.PalumboA.ForestiR. (2023). Immersive virtual reality in post-stroke rehabilitation: a systematic review. Sensors23, 1712. 10.3390/s23031712DishmanR. K.IckesW. (1981). Self-motivation and adherence to therapeutic exercise. J. Behav. Med.4, 421–438. 10.1007/bf00846151DobkinB. H. (2004). Strategies for stroke rehabilitation. Lancet Neurology3, 528–536. 10.1016/s1474-4422(04)00851-8DukesP. S.HayesA.HodgesL. F.WoodburyM. (2013). “Punching ducks for post-stroke neurorehabilitation: system design and initial exploratory feasibility study,” in2013 IEEE symposium on 3D user interfaces (3DUI) (IEEE), 47–54.ElangovanN.CappelloL.MasiaL.AmanJ.KonczakJ. (2017). A robot-aided visuo-motor training that improves proprioception and spatial accuracy of untrained movement. Sci. Rep.7 (1), 17054. 10.1038/s41598-017-16704-8ElangovanN.TuiteP. J.KonczakJ. (2018). Somatosensory training improves proprioception and untrained motor function in Parkinson’s disease. Front. neurology9, 1053. 10.3389/fneur.2018.01053FerdousS. M. S.MichaelA.ChowdhuryT. I.QuarlesJ. (2022). “Use of scaling to improve reach in virtual reality for people with Parkinson’s disease,” in 2022 IEEE 10th international conference on serious games and applications for health (SeGAH). IEEE, 1–7.FreesS.KesslerG. D. (2005). “Precise and rapid interaction through scaled manipulation in immersive virtual environments,” in IEEE proceedings. VR 2005. Virtual reality, 2005 (IEEE), 99–106.FreesS.KesslerG. D.KayE. (2007). Prism interaction for enhancing control in immersive virtual environments. ACM Trans. Computer-Human Interact. (TOCHI)14, 2–es. 10.1145/1229855.1229857HaoJ.XieH.HarpK.ChenZ.SiuK.-C. (2022). Effects of virtual reality intervention on neural plasticity in stroke rehabilitation: a systematic review. Archives Phys. Med. Rehabilitation103, 523–541. 10.1016/j.apmr.2021.06.024HartS. G.StavelandL. E. (1988). Development of nasa-tlx (task load index): results of empirical and theoretical research. Adv. Psychol.52, 139–183. 10.1016/s0166-4115(08)62386-9HussainS.JamwalP. K.Van VlietP.GhayeshM. H. (2020). State-of-the-art robotic devices for wrist rehabilitation: design and control aspects. IEEE Trans. Human-Machine Syst.50, 361–372. 10.1109/thms.2020.2976905KarimeA.EidM.Alja’AmJ. M.El SaddikA.GueaiebW. (2013). A fuzzy-based adaptive rehabilitation framework for home-based wrist training. IEEE Trans. Instrum. Meas.63, 135–144. 10.1109/tim.2013.2277536KawahiraK.ShimodozonoM.OgataA.TanakaN. (2004). Addition of intensive repetition of facilitation exercise to multidisciplinary rehabilitation promotes motor functional recovery of the hemiplegic lower limb. J. Rehabilitation Med.36, 159–164. 10.1080/16501970410029753KenzieJ. M.SemrauJ. A.FindlaterS. E.HerterT. M.HillM. D.ScottS. H. (2014). Anatomical correlates of proprioceptive impairments following acute stroke: a case series. J. neurological Sci.342, 52–61. 10.1016/j.jns.2014.04.025KhokaleR.MathewG. S.AhmedS.MaheenS.FawadM.BandaruP. (2023). Virtual and augmented reality in post-stroke rehabilitation: a narrative review. Cureus15, e37559. 10.7759/cureus.37559LambercyO.DovatL.YunH.WeeS. K.KuahC. W.ChuaK. S. (2011). Effects of a robot-assisted training of grasp and pronation/supination in chronic stroke: a pilot study. J. neuroengineering rehabilitation8, 63–12. 10.1186/1743-0003-8-63LanghorneP.BernhardtJ.KwakkelG. (2011). Stroke rehabilitation. Lancet377, 1693–1702. 10.1016/s0140-6736(11)60325-5LaverK. E.LangeB.GeorgeS.DeutschJ. E.SaposnikG.CrottyM. (2017). Virtual reality for stroke rehabilitation. Cochrane database Syst. Rev.2018. 10.1002/14651858.cd008349.pub4MariniF.HughesC. M.SqueriV.DoglioL.MorettiP.MorassoP. (2017). Robotic wrist training after stroke: adaptive modulation of assistance in pediatric rehabilitation. Robotics Aut. Syst.91, 169–178. 10.1016/j.robot.2017.01.006MasiaL.CasadioM.SandiniG.MorassoP. (2009). Eye-hand coordination during dynamic visuomotor rotations. PloS one4, e7004. 10.1371/journal.pone.0007004MasieroS.CeliaA.RosatiG.ArmaniM. (2007). Robotic-assisted rehabilitation of the upper limb after acute stroke. Archives Phys. Med. rehabilitation88, 142–149. 10.1016/j.apmr.2006.10.032MontagnerA.FrisoliA.BorelliL.ProcopioC.BergamascoM.CarbonciniM. C. (2007). “A pilot clinical study on robotic assisted rehabilitation in vr with an arm exoskeleton device,” in 2007 virtual rehabilitation (IEEE), 57–64.MubinO.AlnajjarF.Al MahmudA.JishtuN.AlsinglawiB. (2020). Exploring serious games for stroke rehabilitation: a scoping review. Disabil. Rehabilitation Assistive Technol.17, 159–165. 10.1080/17483107.2020.1768309MurakamiH.FiscaleV.SocciniA. M.InamuraT. (2024). “White lies in virtual reality: impact on enjoyment and fatigue,” in Proceedings of the 30th ACM symposium on virtual reality software and technology, 1–2.OstryD. J.GribbleP. L. (2016). Sensory plasticity in human motor learning. Trends Neurosci.39, 114–123. 10.1016/j.tins.2015.12.006PoupyrevI.BillinghurstM.WeghorstS.IchikawaT. (1996). “The go-go interaction technique: non-linear mapping for direct manipulation in vr,” in Proceedings of the 9th annual ACM symposium on User interface software and technology, 79–80.PrinsN. (2013). The psi-marginal adaptive method: how to give nuisance parameters the attention they deserve (no more, no less). J. Vis.13, 3. 10.1167/13.7.3RandD.GivonN.WeingardenH.NotaA.ZeiligG. (2014). Eliciting upper extremity purposeful movements using video games: a comparison with traditional therapy for stroke rehabilitation. Neurorehabilitation neural repair28, 733–739. 10.1177/1545968314521008RossR. E.HartE.WilliamsE. R.GregoryC. M.FlumeP. A.MingoraC. M. (2023). Combined aerobic exercise and virtual reality-based upper extremity rehabilitation intervention for chronic stroke: feasibility and preliminary effects on physical function and quality of life. Archives Rehabilitation Res. Clin. Transl.5, 100244. 10.1016/j.arrct.2022.100244SluijsE. M.KokG. J.Van der ZeeJ. (1993). Correlates of exercise compliance in physical therapy. Phys. Ther.73, 771–782. 10.1093/ptj/73.11.771SqueriV.MasiaL.GiannoniP.SandiniG.MorassoP. (2013). Wrist rehabilitation in chronic stroke patients by means of adaptive, progressive robot-aided therapy. IEEE Trans. neural Syst. rehabilitation Eng.22, 312–325. 10.1109/tnsre.2013.2250521TacchinoG.GandollaM.CoelliS.BarbieriR.PedrocchiA.BianchiA. M. (2016). Eeg analysis during active and assisted repetitive movements: evidence for differences in neural engagement. IEEE Trans. Neural Syst. Rehabilitation Eng.25, 761–771. 10.1109/tnsre.2016.2597157VeerbeekJ. M.Langbroek-AmersfoortA. C.Van WegenE. E.MeskersC. G.KwakkelG. (2017). Effects of robot-assisted therapy for the upper limb after stroke: a systematic review and meta-analysis. Neurorehabilitation neural repair31, 107–121. 10.1177/1545968316666957WangY.ZhuH.ElangovanN.CappelloL.SandiniG.MasiaL. (2021). A robot-aided visuomotor wrist training induces gains in proprioceptive and movement accuracy in the contralateral wrist. Sci. Rep.11, 5281. 10.1038/s41598-021-84767-9WentzelJ.d’EonG.VogelD. (2020). “Improving virtual reality ergonomics through reach-bounded non-linear input amplification,” in Proceedings of the 2020 CHI conference on human factors in computing systems, 1–12.