The experiment was conducted using a SensAble Phantom Omni (currently Geomagic Touch; Rockhill, SC, USA), a standard computer monitor (Dell UltraSharp^TM 2209WA) and a custom software program. The Phantom Omni is a three-degree of freedom, desktop haptic interface that can exert precise forces to the user through its end effector via a handheld stylus as well as measure the position of said end effector in space (Massie and Salisbury, 1994). The device allows users to feel and interact with virtual objects. Its maximum exertable force is 3.3 N and its position resolution is approximately 0.055 mm. The visual gain between displacements on the visual display and the haptic device was 1.48. The haptic device was programmed to operate in one of three feedback modes: none, spring assistance, and spring disturbance. The spring assistance mode produced a linearly increasing spring-like force that pulled the user back toward the target curve if the cursor deviated beyond a specified limit (bandwidth), while the spring disturbance mode produced a linearly increasing spring-like force that pushed the user farther from the target curve, augmenting errors, if performance deviated from the curve beyond a specified bandwidth (see below). For the assistance (i.e., error minimization) and disturbance (i.e., error augmentation) modes, haptic gain, magnitude, and channel bandwidth were established through pilot experimentations. The haptic gain represents the spring constant, k in the spring force equation, f = k × x, where f is the force exerted by the device, up to a maximum specified by the magnitude parameter, and x is the displacement between the cursor and its target position on the target curve. The programmable ranges for both gain and magnitude are [0, 1], representing values 0–0.5 N/mm for gain and 0–3 N for magnitude. The channel bandwidth parameter specifies the radius of the haptic feedback-free zone on either side of the target trajectory where no forces will be exerted on the user, regardless of the mode of operation. The gains for each of the experimental conditions were set to be equivalent in order to maintain the rate of change of feedback. During pilot studies, performing under the error augmentation condition with the same gain and magnitude as the error minimization condition made it very difficult to complete a trial. As a result, the haptic gain and magnitude for the error minimization condition were 0.3 N/mm and 1 N, respectively, while the gain of the error augmentation condition was kept the same (i.e., 0.3 N/mm) but the magnitude was reduced to 0.3 N. The target curve was 0.8 mm wide and the invisible channel bandwidth was 0.8 mm on either side of the target curve.

The position of the haptic device was adjusted for handedness and movements primarily took place in the vertical plane. Seated at a table, participants were required to grasp the device’s stylus to manipulate the corresponding cursor shown on the computer screen. The target curve comprised seven fixed control points, a start-point and an end-point, all of which were connected by sinusoidal curve segments (Figure 1). With respect to movement of the stylus, the amplitude of the curve was 155.6 mm from start-point to end-point, with path length 392.4 mm. With the visual gain, these measurements were 230.1 and 580.2 mm, respectively, on the computer monitor.

There were three practice/acquisition conditions: (i) a control condition with no manipulation of errors using the “none” feedback mode; (ii) an error minimization condition using the “spring assistance” mode; and (iii) an error augmentation condition using “spring disturbance” mode. Using the Research Randomizer website¹, participants were randomized to one of these three conditions for practice of the task, with the sole constraint that group assignments were equal in number, (i.e., nine participants in each group). Based on this random assignment, it was assumed that all groups were equal in skill prior to training; however, this was not measured with a pre-test to avoid providing additional practice, which could be problematic for this relatively simple task. After introductory explanation of the task, participants were allowed three familiarization trials with a curve different than the one to be learned. Once participants were comfortable with the device and the task, the experimenter explained, per their group assignment, what they should expect with respect to haptic feedback when practicing the task. Participants then began the practice phase: 100 trials organized as 20 blocks (five trials/block). Once the target curve appeared on the computer screen, participants started the trial by moving the cursor to the start-point near the bottom of the screen and ended the trial by moving the cursor to the end-point near the top of the screen. The target curve and the cursor were visible for the entire trial and participants were instructed to trace the curve as quickly and as accurately as possible. After each trial, feedback regarding the tracing accuracy (a red trace of their movement superimposed over the target curve) and the movement time (a numerical value displayed in seconds to the nearest decisecond) were provided onscreen. Participants also received a numerical summary tracing error score [a numerical value, in arbitrary units (AU) where 1 AU = 0.0798 mm] after each block indicating the average tracing error for a block of trials. Ten minutes after the end of practice, participants completed an immediate retention test in which they attempted five trials of the task without any augmented haptic feedback and without summary or terminal feedback regarding tracing error or movement time. Solely the cursor and the target curve were visible throughout these retention test trials. After approximately 1 day (M = 1.1, SD = 0.4), participants returned for a delayed retention test, which was identical to the immediate retention test, and a transfer test that solely differed from the retention test by having the target curve a mirror-reversal of the one that was practiced. Prior to skill acquisition, all participants were informed that the tests following skill acquisition would not contain any augmented haptic or visual feedback about performance.

Performance was evaluated by calculating the tracing error, movement time, and samples outside the bandwidth. The tracing error (in mm) for each trial, similar to the mean modulus error (Poulton, 1974), was calculated as follows:

Where there are N samples in a trial, for each sample, the distance, e[i], between the cursor and the next untraced portion of the curve was calculated; then, all these distances were averaged to produce the tracing error for that trial. Movement time for each trial was measured as the time (in seconds to the nearest millisecond) from when the cursor was moved to the start-point, to when the cursor was moved to the end-point. We then calculated a measure of overall performance efficiency, the Speed Accuracy Cost Function: Cost Function = Tracing Error ×Movement Time (Culmer et al., 2009; Raw et al., 2012). While we were primarily interested in tracing accuracy, the cost function variable allowed us to account for the speed-accuracy trade-offs that participants would inevitably make when performing this task. As such, we could determine if better tracing accuracy was a result of increased skill or simply a slower movement. A large cost function indicated less efficient and overall, poorer, task performance.

In order to determine whether any effects of haptic feedback were simply due to participants having different amounts of feedback about their performance, we measured samples outside the bandwidth, expressed as a percentage of samples in a trial, to ascertain the amount of augmented haptic feedback that was provided on a given trial. This data was only relevant for the error minimization and error augmentation groups during the acquisition phase of the experiment. All these data (movement time, tracing error, cost function, and samples outside the bandwidth) were averaged over each block of five trials to provide 20 data points for the acquisition phase and three data points for the retention/transfer phase of the experiment.

The primary outcome measure was the cost function and we conducted a mixed model ANOVA (3 group × 20 block in acquisition) with repeated measures on the block factor for the skill acquisition data and three separate one-way ANOVAs for each of the retention and transfer tests. The cost function provides a succinct and easily understood measure of performance. However, there are multiple combinations of the component factors (movement time and tracing error) that could lead to a particular outcome. For example, an increase in cost function (i.e., a decline in performance), could be due to increased movement time, increased tracing error or both. In order to fully understand the mechanisms leading to any significant effects on the cost function, we conducted similar analyses for movement time and tracing error, as appropriate. We also conducted a mixed ANOVA (2 group × 20 block) with repeated measures on the block factor on the samples outside the bandwidth data. Because we expected that performance would improve over the course of acquisition (i.e., cost function and samples outside the bandwidth would decrease), main effects or interactions involving block in acquisition were explored using contrasts with the first block as the reference category. Main effects of practice condition or test were explored using post hoc comparisons using Tukey HSD as well as the Games-Howell procedure when there was concern about the homogeneity of variances, as determined by Levene’s test (Field, 2009). When Mauchly’s test indicated that the assumption of sphericity had been violated for repeated measures factors, Greenhouse-Geisser corrections were applied (all 𝜀 < 0.75) and adjusted degrees of freedom were reported. Effects for all analyses were considered statistically significant at p < 0.05. Effect sizes associated with F-tests were estimated using partial eta squared values (η_p²).

Analysis of the proportion of samples that were outside the bandwidth (i.e., percent of the movement for which haptic feedback was received) resulted in a significant block by practice group interaction, F(19,285) = 2.5, p = 0.0001, η_p² = 0.14 (Figure 2). Neither the effect of practice group, F(1,15) = 3.6, p = 0.076, η_p² = 0.19, nor the effect of block, F(19,285) = 0.9, p = 0.614, η_p² = 0.06, were significant. To break down this interaction, we first performed simple contrasts with block 1 as the reference category for each practice group. This analysis did not yield any significant differences between blocks for either practice group. Subsequently, we performed a simple effects analysis to analyze the effect of group at each level of the block factor. This analysis yielded significant results for blocks 13 (p = 0.033), 15 (p = 0.008), 17 (p = 0.039), 18 (p = 0.024), 19 (p = 0.019), and 20 (p = 0.016). In all cases, the number of samples outside the bandwidth was greater for the error augmentation group. This suggests that in the last quarter of acquisition, the error augmentation group received significantly more haptic feedback than the error minimization group.

Representative traces from a participant in each group are shown in Figure 3, along with the corresponding tracing error, movement time and cost function for those trials. Figure 4 summarizes the results of speed accuracy cost function for each group during skill acquisition and tests and learning. Mauchly’s test of sphericity was significant, χ²(189) = 625.6, p < 0.001, so degrees of freedom were corrected using Greenhouse-Geisser estimates. There was a main effect of block, F(4.2,96.7) = 7.0, p < 0.001, η_p² = 0.23 (Figure 5) and contrasts revealed that cost function on acquisition block 1 was significantly higher than cost function on block 3 and all subsequent blocks of acquisition (all p < 0.05).

There was also a main effect of practice group, F(2,23) = 15.4, p < 0.001, η_p² = 0.57 (Figure 6), and Tukey HSD post hoc comparisons showed that the error augmentation group performed worse (i.e., higher cost function) during acquisition than both the error minimization group (p < 0.001) and the control group (p = 0.002). As might be expected, the error augmentation group also exhibited greater variance than the control and error minimization groups. Consequently, we sought to confirm our post hoc comparisons by using Games-Howell post hoc comparisons. These comparisons confirmed the group differences described above but also suggested that the control and error minimization groups were significantly different (p < 0.001).

Figure 6 shows group means of speed accuracy cost function for each of the retention and transfer tests. Analysis of cost function for the retention and transfer tests showed that there was a main effect of group for the immediate retention test, F(2,24) = 4.0, p = 0.031, η_p² = 0.25, and Tukey HSD post hoc comparisons indicated that the error minimization group performed significantly worse than the error augmentation group, p = 0.024. There were no statistically significant effects of group for either the delayed retention test, F(2,24) = 2.9, p = 0.076, η_p² = 0.19, or the transfer test, F(2,24) = 1.9, p = 0.173, η_p² = 0.14.

One primary reason for the inconclusive results regarding the benefits of haptic training is that researchers have not clearly differentiated or discussed the features of the various motor tasks or training programs that might impact their results (Powell and O’Malley, 2012; Heuer and Lüttgen, 2015). In addition to the basic difference between error minimizing and error augmenting types of haptic feedback, we have previously identified a subtle but important difference between implementations of haptic assistance, namely assistance as demonstration versus assistance as performance feedback (Williams and Carnahan, 2014), which likely invoke different mechanisms of motor learning (Heuer and Lüttgen, 2015). Researchers in (non-robotic) motor learning have long recognized the importance of these relationships among type of tasks, feedback parameters/practice conditions and motor learning. Singer (1977) noted that self-paced, closed skills that are performed in a stable environment, are generally targeted to automation. As such, reducing errors during practice, which encourages repetition of the correct performance, with minimal use of feedback, should be most beneficial for learning such tasks. In contrast, externally paced, open skills should be practiced for adaptability to a dynamic environment and should benefit more from the varied experience of errors during practice (Singer, 1977). However, our results do not provide support for this assertion, as learning of our self-paced, closed skill did not benefit from error minimization during practice. One explanation could be that the task was not strictly self-paced because participants were instructed to move as quickly and as accurately as possible (Kreifeldt, 1972). Outside the laboratory, very few skills are strictly self-paced; that is, most functional self-paced skills (e.g., driving) do have some implicit or explicit guidelines for movement speed. Thus, the instructions utilized in the present study may be more ecologically valid for real-world, closed skills.

In a more recent review, Heuer and Lüttgen (2015) take a neuro-cognitive perspective to address these relationships, specifically in the context of robot assistance for motor learning. Using the presented classification scheme for the products of motor learning, our task can be described as requiring spatial trajectory learning (Heuer and Lüttgen, 2015). While other tracking or path-following tasks, such as drawing circles (Viviani and Terzuolo, 1982) and steering vehicles (Marchal-Crespo and Reinkensmeyer, 2008), may also include dynamic features of trajectory learning, we did not require participants to adhere to any particular velocity or timing-based profile. We measured movement time in order to observe and account for the speed-accuracy trade-off inherent in this type of self-paced task. Importantly, Heuer and Lüttgen (2015) indicate that when a trajectory is demonstrated (visually or haptically), the primary mechanism of motor learning is probably observational learning, while error-based or reward-based learning mechanisms are invoked when error feedback is available, as it was in our experiment. This is along the same lines of the distinction we drew between haptic demonstration and haptic feedback (Williams and Carnahan, 2014). Based on the current literature, they go on to suggest that when error-based learning is the primary mechanism of motor learning, convergent (i.e., error minimizing) haptic training should inhibit learning while divergent (i.e., error augmenting) haptic training should facilitate learning. They also deduced that, based on current evidence, error augmentation may work best for tasks that involve simple paths and require precise hand movements. Our present results – the benefit of error augmentation for learning a simple path requiring precise hand movements – support both these conclusions.

Our data clearly indicate that the error augmentation group experienced significantly more errors during skill acquisition than both the error minimization and control groups. Consistent with the predictions made by Heuer and Lüttgen (2015) as well as error-driven models of motor learning, the error augmentation group exhibited the best performance on tests of learning. We propose that this group’s extensive experience with error-correction during skill acquisition strengthened the internal model for the task and led to greater learning (Thoroughman and Shadmehr, 2000; Wolpert and Ghahramani, 2000). Our results are also consistent with previous studies of physical guidance, which reported that practice conditions that provide frequent on-target experience (such as our error minimization condition) were detrimental for learning (Winstein et al., 1994; Sidaway et al., 2008). In the study by Winstein et al. (1994), participants who practiced an angular positioning task with a physical block reported that they moved quickly to the target without concentrating on stopping at the target location. The study authors proposed that this approach to practice disrupted information processing activities that are crucial for skill retention (Winstein et al., 1994). Similarly, it is possible that our error minimization participants simply traced the curve quickly without focusing much on accuracy because they knew that the device would keep them on target. This interpretation is compatible with theories suggesting that more effortful training conditions lead to better learning (Bjork, 1994; Linn and Bjork, 2006) as well as the notion of deliberate practice required to attain expertise in a given skill (Ericsson et al., 1993). It is also interesting to note that the control group exhibited longer movement times than the error minimization group during skill acquisition. This suggests that, despite their errors being similar in magnitude to that of the error minimization group during skill acquisition, the control group may have exhibited more effort during practice. Indeed, the longer movement times can be attributed to increased effort devoted to ensuring that there were very few errors in their movement. In other words, although the control group did not experience large errors, they were actively engaged in error detection and error correction that were reflected in longer movement times. This effortful but largely errorless approach to skill acquisition may have led to the observed trends for the control group’s better accuracy in comparison to the error minimization group on retention and transfer tests. However, since we did not measure attention, motivation or effort, we cannot confirm these suppositions.

Another potential reason for the observed benefits of error augmentation is that divergent force fields and random perturbations are known to cause limb stiffness which can result in more precise movements due to co-contraction of muscles (Heuer and Lüttgen, 2015). This is also supported by the observation that random perturbations (and not just feedback related error augmenting forces) can facilitate performance improvement (Marchal-Crespo et al., 2014a) and motor learning of spatial movement characteristics (Lee and Choi, 2010).

Comparing our results with other studies of haptic training is a little difficult given the differences between our experiment and others that targeted spatial features of trajectories and included evaluation of a haptic error augmenting training condition. Tasks in such studies included pursuit tracking around the outline of a figure (Lee and Choi, 2010; Powell and O’Malley, 2012), point-to-point movements (Burdet et al., 2001; Su et al., 2011), as well as steering a simulated vehicle (Lee and Choi, 2014) and a wheelchair with a joystick (Chen and Agrawal, 2013). Results for the studies utilizing pursuit tracking are mixed: Powell and O’Malley (2012) observed no group differences for performance immediately after practice while Lee and Choi (2010) observed that noise-like haptic disturbance as well as repulsive (feedback-based) haptic disturbance both resulted in better learning (as observed on a delayed retention test) when compared to progressive haptic guidance. The primary difference between these tasks and ours is that our participants did not have to follow a fixed target speed, they were free to trade-off speed for spatial accuracy. Both point-to-point movement tasks showed benefits of practicing in error augmenting force fields for producing straight paths after the force fields were removed (Burdet et al., 2001; Su et al., 2011). However, it could be argued that this ballistic task does not provide the same online visual feedback information that characterizes our task and other path-following tasks. Results of the steering tasks were not entirely consistent with each other: Lee and Choi (2014) reported no differences in tracking errors between their control, progressive guidance, haptic disturbance and hybrid guidance/disturbance groups on a delayed retention test while Chen and Agrawal (2013) reported that their assistive and resistive force groups both outperformed their control group, but did not differ from each other, immediately after training. One likely reason for this inconsistency is the difference in dynamics between the two steering tasks. Nonetheless, it is interesting to note that Lee and Choi’s (2014) simulated vehicle was steered at a constant speed while Chen and Agrawal’s (2013) participants were instructed to travel at the maximum speed but, ultimately, could control the wheelchair’s speed. Even though both steering tasks were evaluated only on spatial accuracy, both sets of authors acknowledge that in addition to the spatial, error-cancelling aspect of performance, a dynamic feature of the task, specifically the timely initiation of turns, was essential to success. Similar dynamic features are likely involved in our task but we did not explicitly train or measure these features. Overall, our contribution to this body of literature is evidence that error augmentation provides benefits over error minimization for learning spatial accuracy, even when participants can trade speed for accuracy.

It is known that feedback can provide motivation and information to guide future performance and, as such, more feedback can mean more information to enhance learning. However, studies have shown that learners can come to depend on increased amounts of feedback and ignore processing of other information (e.g., intrinsic feedback) that would contribute to learning and the development of error-detection capabilities (Salmoni et al., 1984; Schmidt, 1991). These findings are summarized by the guidance hypothesis which predicts that too much guidance will lead to enhanced performance during skill acquisition but worse performance during retention tests. The error augmenting condition, could not, by definition and design, lead to improved performance during skill acquisition (i.e., provide guidance). We will therefore consider only the control and error minimization groups when discussing the relationship between learning and the amount of feedback received in skill acquisition.

The amount of feedback experienced by participants in the error minimization group can be quantified via the proportion of samples/trial for which a participant received haptic feedback, i.e., the proportion of samples/trial that occurred outside the haptic feedback bandwidth. The amount of feedback that this group received was more or less constant throughout practice, averaging 45% of each trial in the first quarter of acquisition trials and 43% in the last quarter. Naturally, the amount of haptic feedback experienced by the control group was nil throughout skill acquisition. Other studies utilizing physical guidance have noted that the augmented information provided by physical guidance tends to be more guiding than feedback in the form of knowledge of results (as the control group received) because physical guidance has typically worked directly toward error reduction and successful task performance (Winstein et al., 1994). However, our results are inconclusive with respect to providing support for the guidance hypothesis: the control group was only significantly better than the error minimization on the transfer test, and only for tracing error, not speed accuracy cost function. Superior performance on transfer indicates that the skills acquired during training have generalized to benefit performance on variations of the task. However, there was no difference between these groups on the delayed retention test and the Games-Howell post hoc procedure indicated that these groups were not different on the transfer test.

Another potential explanation for the error minimization group’s learning effects is that their practice formed an internal model of the task that included the dynamics of error minimization (Armstrong, 1970; Schmidt, 1991). As such, their internal model for the basic, unguided task was undeveloped and when they were asked to perform the task without any augmented feedback, minimal learning was observed. In contrast, the control group was tested under the same conditions as practice and so their internal model of the task was perfectly suited to the testing conditions and it would have been difficult for the error augmentation group to incorporate the dynamics of haptic feedback into their internal model of the task because the feedback they experienced was so disruptive.

We have demonstrated that there is benefit to using error augmenting haptic feedback over error minimizing haptic feedback for tracing a two-dimensional curve. Ultimately, however, our results indicated that the control (no haptic feedback) group’s learning was not significantly different from either of the experimental groups. Similar results have been observed in other studies (Lee and Choi, 2010; Marchal-Crespo et al., 2014b). This could indicate that error-altering haptic feedback is unnecessary for this task, perhaps due to its simplicity. However, it is possible that what error-altering feedback, and more specifically error augmenting feedback offers is greater efficiency in learning a task. Most studies of haptic training are designed with a view to future applications to fields such a robotic rehabilitation or surgical training where saving time and money are key factors when introducing new training programs. As such, researchers should not only seek to determine which error-altering haptic feedback strategies facilitate learning but also whether there are differences in the time required to achieve and maintain the desired level of performance.