For the past one decade, augmentation of sensory feedback has been widely used to improve peoples' task performance related to the activities of daily living (Sigrist et al., 2013). Recent studies have reported that people's task performance can be improved by applying various types of augmented sensory feedback such as visual feedback (Blandin et al., 2008; Huet et al., 2009; Ranganathan and Newell, 2009; Snodgrass et al., 2010; Sülzenbrück and Heuer, 2011; Franz et al., 2014), auditory feedback (Riskowski et al., 2009; Eriksson et al., 2011; Helmer et al., 2011; Secoli et al., 2011; Sigrist et al., 2011), haptic feedback (Reinkensmeyer and Patton, 2009; Chen et al., 2011; Powell and O'Malley, 2011; Marchal-Crespo et al., 2012), and multimodal feedback (Ronsse et al., 2011; Wang et al., 2011; Sigrist et al., 2015). There exist various approaches and practical applications related to the augmented sensory feedbacks. In these studies, the augmented sensory feedbacks were often referred to by several interchangeable names, e.g., augmented feedback, biofeedback, extrinsic feedback (Schmidt and Wrisberg, 2008; Utley and Astill, 2008).

Among the types of the introduced augmented sensory feedback, haptic feedback is the one that can deliver tangible feedbacks, e.g., force, stretch, or vibration, to users' body parts. A haptic lever is one of the most commonly-used interfaces providing a force feedback to attract the user toward the safer or the desired directions while driving a power-wheelchair (Crespo and Reinkensmeyer, 2008; Marchal-Crespo et al., 2010a; Yoon et al., 2017) or assisting target-pointing/hitting tasks (Powell and O'Malley, 2012; Fisher et al., 2015; Patton and Huang, 2016). Vibrators or skin-stretchers have also been used to provide tactile stimulations for postural sway improvement (Gopalai and Senanayake, 2011; Pan and Hur, 2016; Pan et al., 2017), trunk sway improvement (Davis et al., 2010; Lee et al., 2012), target acquisition and pointing (Bark et al., 2010; Hsieh et al., 2014; Kaul and Rohs, 2016), balance training (Spelmezan et al., 2009; Nanhoe-Mahabier et al., 2012), gait function learning (Shull et al., 2011; Sienko et al., 2013). Several devices have been developed to provide realistic three dimensional tactile sensation to the user, e.g., touching a flat surface, grasping a virtual object, and tipping a surface or an object (Chinello et al., 2012; Prattichizzo et al., 2013; Pacchierotti et al., 2015).

Although there exist many investigations about the effect of haptic sensory augmentation, the benefit from the haptic sensory augmentation is still debatable. It has been known that the performance improvement under haptic guidance depends on the user's age and initial skill level (Marchal-Crespo et al., 2010b). Specifically, haptic guidance is more suitable for the novice subjects than the skilled subjects, and the significant improvement could be observed mostly for the for young adults compared to the elderly adults (Cesqui et al., 2008; Milot et al., 2010; Marchal-Crespo et al., 2010b). A critical drawback of the haptic guidance that has been reported is that the haptic guidance can improve the user's performance mostly for the simple motor tasks, e.g., pointing or reaching tasks; in contrast, its benefits tend to degrade for more complex motor tasks, e.g., driving tasks, learning motions in sport activity (Todorov, 2004; Rauter et al., 2010; Marchal-Crespo et al., 2012). All of these issues have been observed in our previous research where subjects drove a virtual power-wheelchair without colliding against obstacles under haptic guidance (Yoon and Hur, 2016). Considering that elderly adults are usually novice drivers, the observed inconsistency of performance improvement across the elderly adults impeded extending our methods for power-wheelchair guidance to the elderly adults.

To overcome these issues and advance state-of-the-arts in both multiple haptic modalities and haptic guidance for the elderly adults, we propose a haptic joystick combined with a custom-designed wearable skin-stretcher to more effectively guide the elderly adults while driving a power-wheelchair. The objectives of this study are (i) to examine the synergistic effects between two haptic sensory feedback modalities, i.e., force feedback to the hand and cutaneous skin stretch feedback on the steering forearm, (ii) and observe if the synergistic effects are consistent throughout various scenarios. Our hypothesis is that these two simultaneous feedbacks can provide more reliable and informative sensory cues for the improved driving performance in the elderly population. This is plausible because the additional skin stretch feedback can provide more reliable guidance on the supination/pronation of the forearm, which plays an important role in the rider's power-wheelchair control. To the best of authors' knowledge, combining force feedback and cutaneous skin stretch feedback for elderly subjects to improve power-wheelchair control has not been considered.

To validate the proposed approach, an experiment with the elderly subjects was performed. In the experiment, the subjects were exposed to four different task scenarios, and their performance was recorded under four different conditions of the augmented sensory feedback: no feedback augmentation, force feedback augmentation by a haptic joystick, cutaneous skin stretch augmentation by a wearable skin-stretcher, and a combination of force feedback and cutaneous skin stretch by the two devices. Statistical analyses were performed to identify the effects of two factors: task scenarios and augmented sensory feedback. The rest of the paper is organized as follows. In Section 2, the materials and method for this study are introduced. Section 3 describes the human subject experiment. Results are presented in Section 4. Discussions related to findings throughout this study are provided in Section 5.

We present both the main and the interaction effects of two independent factors, task scenario and augmented sensory feedback, on the subjects' performance, quality of achievement (M1), minimum distance to obstacle (M2), mean deviation from reference trajectory (M3), and the summation of average task completion time (M4). Recall that we have denoted four augmented sensory feedback conditions as NA (no feedback augmentation), F (force feedback augmentation by the haptic joystick), C (cutaneous skin stretch feedback augmentation by the skin-stretcher), and FC (force feedback and cutaneous skin stretch feedback augmentations). Four task scenario conditions will be simply denoted by numbers as Task1, Task2, Task3, and Task4 in below.

The results show that both task scenario and augmented sensory feedback had significant main effects on safety-related metrics i.e., M1, M2, and M3. First of all, for the augmented sensory feedback, NA and FC were significantly different for M1, M2, and M3, consistently. To be more specific, when both sensory feedbacks were simultaneously provided, the quality of driving (M1) improved, the minimum distance from the obstacle became greater with more safety margin (M2), and the deviation from the reference trajectory became smaller (M3). However, force feedback alone (F) was not enough to enhance the safety-related measures with statistical significance. Interesting observation is that a combined sensory augmentation of both force feedback and skin-stretch feedback effectively enhanced the safety-related metrics (e.g., M1, M2, and M3). This can be interpreted as follows: FC had more distinct effects on the improvement of the subjects' performance compared to the other augmented sensory feedback conditions, i.e., NA, F, and C. Even though each of F and C may not effectively enhance the safety-related metrics, the combined sensory feedback FC synergistically enhanced the safety-related metrics. This result is consistent with the findings from Shull and Damian (2015); Pan and Hur (2016); Pan et al. (2017) that skin is a good receptor; therefore, cutaneous stimulations have the great feasibilities to serve as an efficient augmented sensory feedback. Similarly, for gait and balance rehabilitation trainings, the feasibility to improve learning efficiency was attested by stimulating multiple skin locations so that the subjects can perceive different haptic stimuli more accurately (Jirattigalachote et al., 2011; Lurie et al., 2011).

Further investigation on M3 (i.e., deviation from the reference trajectory) under FC shows the feasibility of positive synergistic effect between force feedback and cutaneous skin stretch feedback. For the proposed guidance approach, the artificially-generated (software-generated) reference trajectory was invisible to the human subjects. In M3, C, and F separately didn't show any effect even though F seemed to have smaller mean value compared to C. However, the combined FC could effectively minimize the deviation of their driving trajectory from the reference, suggesting synergistic sensory augmentation from both F and C (see Figure 11C).

There were significant main effects of the task scenarios on the performances. The pairwise comparison found significant differences between Task2 and Task3 for all of the safety-related metrics (i.e., M1, M2, and M3). Task2 and Task3 have different road curvatures; the former is a smooth right turn whereas the latter is a sharp right turn. Therefore, this geometrical difference (or, difficulty) of the road was explicitly reflected by the pairwise comparison of subjects' performance. For the definition of each task, please refer to Section 3.2 where we defined four task scenarios as Task1: smooth left turn with a rock on a road, Task2: smooth right turn with a horse on a road, Task3: sharp right turn with a cart on a road, and Task4: sharp left turn with trash cans on a road.

For M1 and M2, performance of Task3 was better than the performance of Task2, suggesting that subjects performed better when the tasks were more challenging (e.g., Task3) compared to the easier task (e.g., Task 2). Similar observation was made for M4 (Task 2 vs. Task 4). One possible postulation can be provided via the relation between task performance improvement and the subject's alertness about the environment. It has been reported that people's motor performance may enhance when the motor tasks become more challenging with an appropriate amount as per their performance limit because people become more alert and cautious about the motor tasks when they encounter challenges. For example, it was reported that firefighter's functional balance enhanced significantly when they became more alert and cautious about the firefighting environment (Hur et al., 2013).

However, for M3, interesting results were found. First of all, less challenging tasks seemed to yield better performance for M3 (i.e., deviation from the reference trajectory). For example, Task1 was less challenging then Task2 since Task1 involved inward rotation (left turn) of the forearm (i.e., pronation) and Task2 involved outward rotation (right turn) of the forearm (i.e., supination) even though both Task1 and Task2 involve the same radii of curvature. It has been reported that control of inward arm movement is more dexterous than the control of outward arm movement. For example, forehand stroke is more accurate and powerful than backhand stroke in tennis or table tennis (Mavvidis et al., 2010). Also, it was reported that torque generation of the forearm in inward direction is more efficient and precise when outward direction (Seo et al., 2007, 2008). Also, please note that there was no effect of hand dominance since all subjects were right-handed. Similarly, trajectory deviated more for Task3 compared to Task1 and Task2, since Task3 is more challenging. For Task4, significant reduction in deviation was found even though Task4 involves sharp turn to the left direction. This can also be explained by the inward rotation of the forearm.

Please recall that there was a significant interaction effect between the task scenario and the augmented sensory feedback on M3 (Figure 13). The interaction effect is mainly due to the results that M3 was affected by the task scenario only when NA or C was applied as a sensory feedback condition. This suggests that NA or C does not affect the performance of M3; hence, the task characteristics (e.g., radius and direction of curvature) are directly reflected in the performance of M3. This also indicates that F or FC condition dominates the performance of M3 such that the task characteristics are not directly reflected in the performance of M3.

If we consider C and F as independent sensory augmentation modalities, the data in Figures 11–13 indicate that F dominates C and that C itself has no effect on the performance. However, when both F and C are combined, we could observe the synergistic effect of both F and C such that the overall performance (i.e., M1, M2, M3, and M4) became maximized with FC (Figures 11, 12). This synergistic behavior of FC can be explained with the effect of stochastic resonance for sensorimotor control systems. Stochastic resonance is a phenomenon in which the addition of unperceivable noise (e.g., vibrotactile noise) to a weak signal maximizes the detection and transmission of the weak signal through the neuronal network (Enders et al., 2013). When subsensory threshold vibration is applied to the skin, people do not perceive the vibration. Also, when a weak sensory signal is applied to the skin, people may or may not detect the signal, and the motor performance based on the weak signal would be suboptimal. However, when the weak sensory signal is combined with the subsensory threshold vibration, the detection of the signal enhances, and the motor performance based on the enhanced sensation is maximized (Hur et al., 2014; Seo et al., 2014).

In a broad sense, both cutaneous (or skin-stretch in this study) sensory feedback and force feedback can be regarded as a subset of haptic guidance. The difference lies in how human perceives each of the two augmented sensory feedbacks. Skin-stretch sensory feedback is perceived by Ruffini corpuscle and Meissner corpuscle, the mechanoreceptors underneath the skin, which are sensitive to skin stretch and skin slip, respectively (Johansson and Westling, 1987). In contrast, force feedback to the hand is perceived by Merkel corpuscle, the mechanorecpetor underneath the skin, which is sensitive to sustained pressure (Johansson and Westling, 1987). Force feedback is also perceived by Golgi tendon organ, a proprioceptor at each joint, that is sensitive to muscle tension and pull of tendons (Stephens et al., 1975). Therefore, skin-stretch feedback and force feedback are highly likely to have different mechanisms for sensing and processing sensory signals, and thus, the effects of C and F in the performance metrics (i.e., M1, M2, M3, and M4) seem to differ (Figures 11, 12). However, it's not well known how two sensory feedback signals are fused and processed to produce the synergistic effects, and thus deserves further investigations.

Even though it's not the scope of this study, it will be interesting to look into the effect of multiple modalities on the performance indices. It was reported that different modalities have different effects on the performance indices (Wang et al., 2011; Ronsse et al., 2011). For example, auditory alertness helped enhance the driving performance whereas visual feedback alone did not (Wang et al., 2011). Even though multiple modalities can be useful in enhancing the driving performance, haptic modality can have more direct effects on the improvement since visual or auditory modalities are already heavily used while conducting the tasks (Scott and Gray, 2008; Wang et al., 2011; Yoon et al., 2017). In our study, other modalities, e.g., vision or audition, were not included than haptic modalities. As reported in the result section, force feedback and skin stretch feedback have different effects in enhancing the performance indices even though both synergistically enhanced the performance. It will be interesting to investigate how other modalities, e.g., vision or audition, can influence the synergistic behavior. In addition, it will be worth examining the way both of force feedback and skin stretch feedback are integrated to maximize the enhancement.

In design perspectives, the future usage of the custom-developed wearable skin-stretcher is promising. Considering guidance hypothesis, we know that the augmented sensory devices should be worn persistently to improve the activities in daily living for users (Sigrist et al., 2013). The features of our skin stretchers – a small size, arm-band type, and wirelessly-controlled – show us that it can cope with the needs to assist the users in daily living. Existing research have reported the effectiveness of haptic guidance for power-wheelchair riders (Marchal-Crespo et al., 2010a,b; Morère et al., 2015; Foley et al., 2016). Therefore, in practical situation, we also expect that the proposed method support the power wheelchair riders successfully by combining with existing machine vision and sensor systems, such as introduced in How et al. (2013) and Simpson et al. (2004), to generate an artificial reference path based on the surroundings.

Although the balances in both age and gender were concerned when the subjects were recruited, the limitation of this study is that the subjects were not actual power wheelchair riders. Therefore, there exists a possibility of that the result might vary; however, we remained checking this possibility as future work. Also, the subjects performance while riding a practical system should be investigated as well.

In this study, we developed two augmented sensory feedback devices, a wearable skin-stretcher and a 2D haptic joystick, and provided elderly subjects with additional sensory cues via cutaneous skin stretch feedback and force feedback at their forearm and hand, respectively. The experiments were performed with four task scenarios in which the subjects were instructed to drive a virtual power-wheelchair along roads of various curvatures where obstacles were placed. The subjects' task performances were evaluated based on the following four metrics: quality of achievement (M1), minimum distance to obstacle (M2), mean deviation from a reference trajectory (M3), and the summation of average task completion time (M4). The results showed that M1, M2, and M3 were improved, compared to no feedback augmentation condition, when both a force feedback and a cutaneous skin stretch feedback were provided simultaneously. Especially, for M3, the cutaneous skin stretch itself did not seem to encourage the performance improvement; in contrast, the improvement was observed when it was combined with a force feedback. These findings substantiated that there existed the positive synergistic effects on elderly power-wheelchair riders' safety metrics when the two augmented sensory feedbacks were combined.

Future work should be followed to examine the efficacy of the proposed sensory augmentations–that is, the cutaneous skin stretch feedback on the forearm and the force feedback–for different tasks, e.g., hand writing, drawing shapes, learning motions in sports, will be of our interest. Next, further investigation should be conducted in physiological perspectives “why a cutaneous skin stretch and a force feedback encourages the performance improvement when they are combined in?” Accordingly, we should also check whether the encouraged improvement depends on stimulated locations–both sensory augmentations should be applied on ipsilateral or contralateral side, or whether the improvement will be consistent when one of those feedbacks works as a disturbance. Lastly, the various combinations of augmented sensory feedback should be considered to compensate or overcome degraded task performance in activities of daily living such as turning a door knob, spooning, waking up/down stairs, etc.

HY and PH have contributed to a theoretical concept, the design of the experiment, performing experiments, data analysis, interpretation, and drafting the manuscript. NA has contributed to design of the wearable skin-stretcher.