A search over English language peer-reviewed journal papers was conducted across five databases, including Scopus, IEEE, Web of Science, PubMed, and Science Direct, between the years January 2015 to December 2021. The following keywords were used: (electro* OR respiration OR “galvanic skin”) AND “stroke rehabilitation” AND “virtual reality.” The keywords were searched in the title or abstract of the articles. Mendeley reference manager was used to remove duplicate citations across yielded results. The term “physiological computing” was not used as a keyword as it is a very specialized term that few researchers in the VR space utilize at the moment. By including all possible modalities that begin with “electro” (e.g., electrocardiogram, electroencephalogram, electromyogram, electrooculogram, electrodermal activity) we are bound to encompass all studies that utilized physiological computing systems without referring to this terminology directly.

Inclusion criteria included articles that 1) used HMD-based VR experiences, 2) included at least one feedback modality to improve the VR experience, 3) included a valid clinical measure or questionnaire to gauge the effectiveness of the intervention, and 4) developed a rehabilitation tool but tested on healthy subjects. Exclusion criteria included: 1) review papers, 2) studies not relying on HMD-based VR, and 3) studies not focusing on stroke rehabilitation applications. Eligibility criteria according to population, intervention, comparison, outcome, and study design (PICOS) has been tabulated in Supplementary Table S1 and made available in the supplementary material section. We note that studies with healthy people have only been included if the goal of study was aligned with providing interventions to the population of interest (as described in Supplementary Table S1).

After searching the abovementioned databases and merging the duplicated results, the authors screened titles and abstracts to exclude unrelated articles. The full-text screening was then performed on the remaining papers and aligned articles were included in this systematic review. To collect detailed information from each study, a data extraction spreadsheet was developed encompassing details across three domains: 1) study design and demographics (targeted stroke group, number of subjects, control group, gender distribution, target problem, session description, trial design), 2) technological aspects (HMD device, VR engine used, physiological modality, type of feedback, equipment, and electrode placement), and 3) outcomes (clinical/non-clinical scales, clinical results, availability of baseline information, pre- and post-intervention comparisons, and follow-up findings).

To assess the risk of bias (ROB) in the included studies, we utilized the National Institutes of Health—National Heart, Lung, and Blood Institute (NIH-NHLBI) quality assessment tool (National Institutes of Health, 2022), as well as the checklist for quasi-experimental studies based on the Joanna Briggs Institute (JBI) critical appraisal tool Tufanaru et al. (2017). For the former, articles are evaluated based on the type of the study and overall quality is judged based on “good,” “fair,” or “poor” categories. Categories are chosen based on replies to a number of questions, where answers can take the form of yes, no, cannot determine, not reported, or not applicable. Both NIH-NHLBI and JBI tools examine studies across three aspects, including objectives, methodology, and report of outcomes.

From the 218 identified articles, 46 were from PubMed, 16 from ScienceDirect, 59 from IEEEXplore, 38 from Web of Science, and 59 from Scopus. Figure 1 depicts a flow chart of the study selection process. After eliminating duplicates, 182 studies were left to be screened based on title and abstract. From this first pass, 130 papers were eliminated as they did not meet the inclusion criteria, resulting in 52 articles included for full-length analysis. In the end, 12 articles focusing on HMD-VR, multimodal physiological computing, and stroke motor rehabilitation were included in this review.

Detailed information about the configuration, modalities used, study type, experiment duration, and demographics of the participants are available in Table 1. Detailed description of hardware used to acquire the physiological signals are described in Table 2.

All of the included papers followed a “pre/post study design,” which refers to measuring specific metrics prior to the experiment and comparing them with recorded measures after the intervention (Thiese, 2014). As can be seen, among the included articles, four studies recruited only healthy subjects (Vourvopoulos and I Badia, 2016; Trombetta et al., 2017; Juliano et al., 2020; Achanccaray et al., 2021), three relied on patients who were in chronic stage of stroke (Vourvopoulos et al., 2019a; Vourvopoulos et al., 2019b; Marin-Pardo et al., 2020), three studies used a mixture of healthy and stroke patients (Lupu et al., 2018; Nissler et al., 2019; Stanica et al., 2020), one study used a stroke patient in sub-acute stage (Huang et al., 2018), and one study used a mixture of stroke patients in the chronic stage and patients with developmental disabilities (Elor et al., 2018). The reported numbers in Table 1 are for participants who have completed all the sessions and satisfied all experimental protocols. Overall, 64.1% of participants were healthy subjects, and only 40% of all participants were females.

The reviewed studies relied on virtual environments to deliver rehabilitation exercises by means of gamified interventions. Ten articles focused on upper limb rehabilitation, while the remaining two studies focused on both upper and lower limbs (Juliano et al., 2020; Stanica et al., 2020). Table 3 details the characteristics of the study (i.e., the aim of the study, target limb, and virtual environment description). As can be seen, among the included articles, only three studies used more than two different virtual environments (Huang et al., 2018; Nissler et al., 2019; Stanica et al., 2020).

As one of the reviewed papers was a case report, the NIH-NHLBI quality assessment tool for case series studies and the JBI checklist for case reports was used. For the remainder of the studies, the NIH-NHLBI quality assessment tool for before-after (pre-post) studies with no control group and the JBI checklist for quasi-experimental studies was utilized. The detailed responses obtained by both of the assessment tools are presented in Supplementary Tables S2, S3, respectively, both available in the supplementary materials section. Table 4, in turn, presents the results of the quality evaluation using the NIH-NHLBI tool.

As can be seen, from the NIH-NHLBI analysis, the majority of the pre-post studies (n = 11) were rated as fair, while four studies were rated as good, and two as poor. The fair-rated studies showed clear objectives, eligibility criteria, selection of participants, targeting populations of interest, description of intervention, description of outcome measures, and multiple outcome measures, they reflected issues in terms of sample size, blinding of outcome assessors, and follow-up rate. Poor-rated studies, in turn, had issues concerning multiple outcome measures, statistical analysis, and group-level interventions. The case study, on the other hand, was rated good quality due to clarity of objectives, detailed description of population and intervention, valid and reliable outcome measurement, and well-described statistical method and results. In terms of JBI criteria, the issues flagged concerned lack of control group, receiving a similar treatment (except exposure to HMD-VR), and lack of follow-up plan. None of the articles showed issues with clarity of cause and effects of their study, similarity of participants, multiple measurement points of the outcome, or similar outcome measurements. Three articles, however, had issues with reliability of outcome measurements due to not using appropriate statistical analysis methods. For the case study, the JBI tool reflected “yes” responses to all of the checklist questions, thus corroborating its quality.

Overall, the included articles employed different types of technologies. Figure 2 demonstrates an alluvial diagram of the methods, their combinations, and target aim. As evidenced, different physiological modalities and technologies have been utilized in VR-based stroke rehabilitation interventions. The following subsections highlight these methods in detail.

The reviewed papers relied on multiple approaches to measure the efficacy of their proposed methods. For example, several different questionnaires, physiological signal analyses, and clinical measures were explored. Table 5, presents a description and evaluation domains of the most-utilized clinical/non-clinical scales reported in the papers. As interventions may have outcomes across numerous domains, it is common for studies to use more than one assessment method. As seen from the Table, seven clinical scales have been used most often, including the Fugl–Meyer assessment (FMA), action research arm test (ARAT), Montreal cognitive assessment (MoCA), motor assessment scale, modified Ashworth scale, range of motion (ROM), and stroke impact scale (SIS). Studies typically compared these metrics pre- and post-intervention to gauge the benefits of the VR therapy. Further, to assess detailed alterations in the brain and plasticity improvements, two studies relied on magnetic resonance imaging (MRI) and functional MRI (fMRI) (Vourvopoulos et al., 2019b,a). Table 6 includes the significant final outcomes observed by the studies, as well as the time points in which clinical/non-clinical measures were taken for comparisons.

In terms of clinical measures, various metrics are observational, meaning that a clinician (or a physiotherapist) determines a score after the patient performs a specific task. Five studies relied on a certified occupational therapist or rehabilitation specialist to carry out these observational clinical measures (Huang et al., 2018; Vourvopoulos et al., 2019a; Vourvopoulos et al., 2019b; Marin-Pardo et al., 2020; Stanica et al., 2020). FMA and SIS can be seen to be the most widely used measures. However, FMA has been shown to be subject to ceiling effects (Lin et al., 2004), whereas SIS potentially has ceiling effects in hand function, memory and thinking, communication, mobility and activities of daily living (ADL), and instrumental ADL domains (Richardson et al., 2016).

In terms of non-clinical measures, post-hoc physiological computing tools have been leveraged to extract changes in the measured biosignals. For example, Vourvopoulos and I Badia, (2016) reported differences in EEG activity of motor imagery in virtual environments relative to regular imagery. In fact, the ANOVA test with a Greenhouse-Geisser correction and pairwise Wilcoxon signed-rank test showed a statistically significant result in the alpha band (F _{(2.524,20.191)} = 4.800, p < 0.05), between “motor execution and VR (VRMP)” versus the control condition. In the theta band (4–7 Hz), the difference was reflected between motor-execution and VRMP (F _{(1.874,14.990)} = 7.615, p < 0.05). Interhemispheric interaction was also observed in several studies to be affected by visual, audio, or haptic feedback (Vourvopoulos and I Badia, 2016; Vourvopoulos et al., 2019a; Vourvopoulos et al., 2019b; Achanccaray et al., 2021). Vourvopoulos et al. (2019a) calculated ipsilateral ERD/ERS of the beta band using C3 and C4 electrodes. Subsequently, by running a t-test, a statistically significant difference was observed between the first and last session (t (199) = −16.921, p < 0.001). Moreover, it is known that treatments with greater engagement levels keep patients motivated and yield better outcomes (Brett et al., 2017). As there is a direct relationship between engagement and attention (Danzl et al., 2012) and attention deficits could result from strokes (Hochstenbach et al., 1998), leveraging VR tools to increase patient focus could be highly beneficial. Certain rhythms (e.g., alpha and beta bands over the parietal region for visual stimulus), for example, have been related to attention (Kamiński et al., 2012; Jurewicz et al., 2018; Thiele and Bellgrove, 2018; Magosso et al., 2019) and associated with motor function (Khanna and Carmena, 2015).

To this end, Vourvopoulos and I Badia, (2016) found a relationship between the ability to perform kinesthetic imagery and alpha-beta band modulation (ρ = 0.50, p < 0.05) and, thus potentially pointing towards improved attention levels. Furthermore, Achanccaray et al. (2021) noted higher attention levels in visual-electrotactile stimulation compared to visual-only condition, and higher attention levels were also shown to be correlated with haptic feedback and exoskeleton usage Li et al. (2021). In fact, a significant inverse Spearman correlation was found between attention level and difficulty level (ρ = −0.45, p = 0.04). However, a Spearman correlation between attention level and beta band of channels regarding somatosensory and prefrontal regions was observed in both grasping (e.g., AF3 channel: ρ = 0.58, p = 0.04) and flexion/extension (e.g., AF3 channel: ρ = -0.5, p = 0.02) MI tasks. Indeed, by adding haptic feedback, Vourvopoulos et al. (2019a) reported 9 points improvement in the FMA score at the end of the intervention (pre: 31, post: 40) in a case report on a patient with left hemiparesis. These insights suggest that haptic-based feedback could be utilized to boost attention levels in VR-based interventions and lead to improved outcomes Achanccaray et al. (2021); Stanica et al. (2020); Vourvopoulos et al. (2019a). Notwithstanding, Vourvopoulos et al. (2019b) noted that patients with more severe impairments showed the most significant improvement using motor imagery-based paradigms in a VR setting, while EMG-based feedback was shown more promising for patients with mild impairments.

VR-based solutions typically rely on their improved sense of realism (Read et al., 2021), immersion and presence (Slater, 2009), and sense of embodiment (Riva et al., 2019) to boost rehabilitation performance (Lin et al., 2001). To achieve these goals, Vourvopoulos et al. (2019b), for example, customized hand models in the virtual environment to match the patient’s skin tone and gender and Stanica et al. (2020) provided close-to-reality animations of outdoor exercises. Juliano et al. (2020), in turn, leveraged linear regression to examine the relationship between neurofeedback performance and overall embodiment and reported a significant relationship in the HMD-VR experimental condition (F _(1,10) = 8.293, p = 0.016, R ² = 0.399).

The gamification potential of VR-based solutions can also allow for reward mechanisms to be put in place to improve outcomes. For example, the reward has been shown to improve motor learning (Nikooyan and Ahmed, 2015; Chen et al., 2018) and rehabilitation outcomes (Quattrocchi et al., 2017; Widmer et al., 2017). Reinforcement feedback also induces motivation (Vassiliadis et al., 2021). This is important, as lack of motivation has been said to be one of the major barriers in conventional rehabilitation (Damush et al., 2007). As such, reward-based VR interventions improve outcomes (Abe et al., 2011), result in higher rates of satisfaction, and have increased adherence (Trombetta et al., 2017; Elor et al., 2018; Huang et al., 2018; Stanica et al., 2020). For example, Elor et al. (2018) encouraged patients to use their affected weaker limb by giving them higher scores when an action was carried out with such affected limb. Overall, only four of the reviewed articles relied on reward feedback as part of their intervention (Trombetta et al., 2017; Elor et al., 2018; Nissler et al., 2019; Stanica et al., 2020).

Notwithstanding, VR-based solutions relying on HMDs are also known to elicit in certain individuals discomfort and nausea (known as cybersickness). Trombetta et al. (2017), for example, reported a subtle improvement in the sense of comfort when screen-based tasks were given to the participants, relative to those given via an HMD. Other studies, in turn, utilized the simulator sickness questionnaire, and only very minor degrees of discomfort were reported when using HMD-VR (Vourvopoulos et al., 2019b; Marin-Pardo et al., 2020). More recently, cybersickness has been associated with activity in delta (1–4 Hz), theta, and alpha EEG frequency bands (Krokos and Varshney, 2021). Future studies could also explore the use of collected EEG signals to measure cybersickness levels; this was not performed in any of the evaluated papers. Moreover, it is known that stroke can result in a range of cognitive impairments, thus it is necessary to investigate the perception and interaction skills of patients (Sun et al., 2014) in virtual environments. To this end, Nissler et al. (2019) implemented a clinical assessment method called Box and Block test in a virtual environment to measure gross manual dexterity and compared the performance of the patients in both conventional and VR-based settings. The results showed better control of a virtual prosthesis by a patient compared to the healthy control group.

Lastly, one additional advantage of a VR-based intervention with physiological computing is that it enables automated, repeatable, and reliable interventions. Such factors are crucial for treatment fidelity, which has been directly linked with intervention quality (Hildebrand et al., 2012). For example, Resnick et al. (2011) proposed a treatment fidelity plan based on design, training, delivery, receipt, and enactment for stroke exercise interventions. In non-VR-based interventions, there is a great chance of inconsistency in delivering the treatment by a physiotherapist. VR-based treatments, in turn, are programmed, thus system functionality could be used as a proxy for treatment fidelity (Coons et al., 2011). While the study by Stanica et al. (2020) utilized a specific order for all of the participants to assure consistency in the treatment, none of the reviewed articles used a fidelity checklist explicitly.

The included articles show the potential of VR-based interventions, coupled with physiological computing, for improved rehabilitation outcomes for stroke survivors. Notwithstanding, none of the articles followed a “randomized control trial” study design, which is preferred to investigate clinical results. Moreover, studies frequently reported a small sample size as a limitation, and only two articles reported a gender-balanced participant pool, thus making generalization of the results difficult (Miller et al., 1993; Butler et al., 2009). Moreover, several studies investigated the efficacy of an HMD-based intervention received in parallel with conventional therapy, thus making it difficult to truly gauge the benefits of the VR intervention per se. Overall, nonetheless, the results suggest that an additional VR-based intervention (in addition to the conventional treatment) is better than having two conventional treatments. Moreover, patient involvement is crucial for successful rehabilitation outcomes (Kristensen et al., 2016). As the majority of stroke patients will be older in age, they may be strangers to burgeoning VR and physiological computing (e.g., wearables) tools. Future studies should assure that sufficient training sessions and/or tutorials are provided and that the tasks can be achieved even by those with cognitive impairments resultant from the stroke. Incorporating some principles from iterative designing (Nielsen and Molich, 1990) and gamer user experience into the virtual environment could be beneficial (Tondello et al., 2016; Rajanen and Rajanen, 2018). Integrating many of the physiological computing tools directly into the VR headset could also alleviate some of these issues; such instrumented HMDs are already emerging (Cassani et al., 2018; Cassani et al., 2020a; Moinnereau et al., 2020).

Moreover, the examined studies measured intervention efficacy pre and post-intervention, thus providing limited insights on the long-term effects of VR-based rehabilitation. Future studies should focus on the long-term effects of these interventions. Moreover, the reviewed articles relied on “passive” physiological computing tools, in the sense that signals were measured and used to either change the virtual environment or to gauge intervention outcomes. More “active” tools, such as non-invasive brain stimulation (e.g., transcranial magnetic stimulation and transcranial direct current stimulation, tDCS), have recently emerged and shown to provide some success in rehabilitation when coupled with VR (Cassani et al., 2020b). Future studies could couple passive and active physiological computing (e.g., via an EEG-tDCS headset) and explore the benefits that such hybrid architectures could bring. As observed herein, the majority of the studies utilized EEG sensors to measure neural responses and plasticity. Functional near-infrared spectroscopy (fNIRS) is also emerging as a useful tool for plasticity monitoring (Balconi, 2016) and hybrid fNIRS-EEG headsets are already available in the market. As such, future studies could explore the advantages of integrating the improved spatial resolution advantages of fNIRS with the improved temporal resolution of EEG for stroke rehabilitation. Alternate feature representations, beyond simple ERD/ERS and EEG frequency subband powers, could also be explored as proxies of brain plasticity and intervention outcomes (Papadopoulos et al., 2021).

Regarding the effects of feedback type, studies herein showed that as more senses were stimulated, improved outcomes and attention could be observed. Recently, olfactory feedback has emerged as an additional feedback modality, as tools such as the OVR ION (OVR Tech, Burlington, VT, United States) have appeared in the market. Integrating olfactory feedback may further boost attention, sense of embodiment, and presence, which could show improvements in outcomes, as was the case in other allied domains (Gim et al., 2015; Longobardi et al., 2020). Additionally, none of the studies used glove-based haptic feedback systems. Such systems could provide finger tracking capabilities and individual finger and full hand vibrations, thus potentially also benefiting interventions of the upper limbs (Caeiro-Rodríguez et al., 2021; Gast et al., 2021). Lastly, the reviewed studies did not report experiments with patients suffering from post-stroke visual impairment/disturbances. Such patients may be incapable of benefiting completely from audio-visual VR-based interventions, thus multisensory experiences should be explored. The work of Wedoff et al. (2019) and Real and Araujo (2020), for example, leveraged the use of auditory and haptic stimuli to make VR experiences more accessible to those with visual impairments. In fact, VR interventions could potentially be used soon after the stroke to identify the risk of visual after-effects (Besic et al., 2019).