The wearable robotic device design shown in Figure 1 is based on the motorized wrist-driven orthoses (MWDO) detailed by McPherson et al. (2020) and Chang et al. (2022). We retain the four-bar linkage mechanism that actuates the MWDO, where robot and human motion contribute to gripper opening and closing (Figure 2A). The planar kinematics of this linkage is described in Chang et al. (2024a). In prior work, device software mapped motor motion directly to wrist movement, constraining the human and robot agents to move simultaneously and in constant collaboration. However, the software implemented on the Co-Grasping device in this work does not map either agent’s movement to the other, instead facilitating a range of interaction types. While collaboration between the human and robot is still possible in this device format, it is no longer required. In Figures 2B–D, we show how motion contributions from the human ( Δ α ) and robot ( Δ γ ) form the interior angle, β , where β = β t 0 − ( Δ α + Δ γ ) ; Δ α = α − α t 0 , Δ γ = γ − γ t 0 , and t 0 is the time at the start of the grasping phase. The grasping phase begins when the open hand is placed around an object and the first grasping action is initiated. Decreasing β drives the gripper closed, while increasing β drives the gripper open. When the gripper closes around an object, the human and robot must apply opposing static reaction forces to maintain the grip, regardless of the behaviors ( Δ α and Δ γ ) used to secure it. Note that α t 0 , γ t 0 , and β t 0 can vary between grasps, depending on how each grasp is initiated. In this study, we initiate grasps such that β t 0 always begins at the same value, as detailed in Section 2.1.1.

Participants have the physical ability to move their wrist as they wish at any time. This ability determines whether they collaborate or cooperate once the robot starts moving. Because the human and robot can move at different times after grasping is initiated, the moments at which the human and robot agents initiate and complete their own grasping actions, i.e., start and stop moving, are defined as t α i , t α f , t γ i , and t γ f , respectively. In the present work, user voice commands initiate the grasp such that the robot moves to close the gripper immediately (see Section 2.1.1), so we assume t γ i = t 0 . We additionally define the speeds, including magnitude and direction, at which each agent moves at any given moment as α ̇ and γ ̇ , and the resulting speed β ̇ .

Using these terms, we define the possible grasping behaviors. We consider grasping as an agonistic task, where both agents contribute only to the improvement of the shared activity (Jarrassé et al., 2012), such that grasping occurs when β ̇ < 0 and is constrained to α ̇ , γ ̇ ≥ 0 . The human and robot exhibit collaborative grasping whenever they move concurrently to reduce β , where α ̇ > 0 and γ ̇ > 0 . The matching of these movements by both agents in collaboration is shown in Figure 2D. Alternatively, the grasping behavior is cooperative if either α ̇ = 0 or γ ̇ = 0 . Figure 2B shows the participant cooperating with the robot by letting the robot move entirely while the wrist remains stationary ( α ̇ = 0 ) and Figure 2C shows the opposite. In a single grasp, both cooperation and collaboration can occur at different times. For example, if the robot begins to move before the human ( t γ i < t α i ) , the grasp starts in cooperation. Then, if the robot’s motion finishes after the human’s motion ( t α i < t γ f ) , a collaborative grasp phase can occur. The grasp may later return to cooperation if the agents do not complete their movements at the same time ( t α f ≠ t γ f ) .

Measuring how a person chooses to interact with the system when both collaboration and cooperation are possible, however, has yet to be quantified. Sensors integrated into the current test setup allow for this observation. Within the Co-Grasping device, a soft rotary potentiometer (Spectra Symbol, Salt Lake City, Utah, USA) measures the human’s wrist position ( α ) and an encoder measures the robot’s motor position ( γ ) . These sensors quantify the physical participation of each agent through their relative movements and effect on β . We also uniquely augment the original design of the MWDO to additionally measure forces exerted by and on the user. Unlike the original MWDO designs, which actuate the user’s own fingers, the linkage structure in the Co-Grasp device actuates a two-pronged artificial gripper. The user grasps a handle attached perpendicularly to the base plate with a 5 kg load cell (SparkFun, Niwot, Colorado, USA) to measure the amount of force exerted between the operator and the device (Figure 1). The user feels the reaction forces needed to maintain a successful grasp through this handle, providing inherent feedback on the state of the system that is typically not provided by motorized components of other robotic grasping technologies. When forces increase during grasping, relative motion of the human or motor indicates which agent leads force application. Because wrist muscular exertion is necessary to maintain a grasp, we estimate resulting muscle activity using a Myoware v1.0 electromyography (EMG) sensor (Advancer Technologies, Raleigh, North Carolina, USA) on the wrist extensors. These EMG measurements add a physiological perspective to the measured behavioral changes.

Participants grasped and released a set of test objects between two sets of elevated platforms. Nine right-handed individuals were recruited from the University of California, Berkeley (UCB), with a mean age of 22.56 ± 3 years. Six participants identified as female and three as male. Three participants ethnically identified as Hispanic/Latinx. Four participants racially identified as Asian, four as White, one did not provide their race. All work was performed under the UCB Institutional Review Board protocol #2020-02-12983.

Wrist angle, force, and EMG sensor data from each subtask was parsed into 3 segments: robot grasp, object transfer, and robot release. The robot grasp segment began when the robot began closing the gripper (start of Automated Close state in Figure 3) and ended when the robot stopped closing the gripper (start of Transfer state in Figure 3). Object transfer took place immediately after the robot finished its closing action, when the human had to lift and transfer the object to the next location, lasting the duration of the Transfer state in Figure 3. Finally, robot release began when the robot began opening the gripper and ended when it stopped opening the gripper, lasting the duration of the Automated Open state of Figure 3. During initial examination of the data, subjects typically remained passive during the robot grasp and robot release phases of each subtask. Participants only began interacting with the system during the object transfer phase, where they took on either a static and resistive role to hold the gripper steady while moving the object, or enacted wrist motion to recover from the robot’s error before moving the object. Therefore, we focused sensor-based analysis efforts on data from the object transfer segment.

In post-processing of the sensor data, a moving average filter with a window size of 5 data points (0.2 s) was initially applied to smooth sensor signals prior to subsequent calculations. Each signal was normalized to the lowest value of the current segment, such that the resulting processed time-series data represented relative changes in signal specific to the current subtask, instead of absolute changes in signal. Peaks were defined as the maximum value of the normalized signal. Additionally, since the duration of object transfer segments varied across subtasks and participants, we scaled timestamps to a common range, in order to maintain temporal data patterns independent of absolute time. Instead, each timestamp, s , was represented as a proportion of the total duration of the object transfer phase, s total , such that normalized time = ( s / s total ) ∗ 100 .

Objective metrics, taken from data collected during trials, included the following:

1. Total Wrist Motion: the area under the curve (AUC) of the normalized potentiometer data, in units of degrees * s / s total .

Subjective metrics taken from surveys in between trials included the following:

12. Trust: participants’ self-reported trust levels out of 100%, as measured by the Trust Perception Scale (Schaefer, 2016).

For objective metrics 1-9, each subject’s dataset was averaged to obtain a single value for each condition during the object transfer segment. From the Error condition trials, only data from the 12 subtasks where the robot error occurred were pooled together, excluding the remaining 15 subtasks during these trials where the robot performed normally. Objective metrics 10–11 and subjective metrics 12–13 were recorded for each trial and averaged by condition for each subject.

An omnibus Friedman’s test was performed to determine if the robot behavior condition significantly affected each metric, while accounting for individual subject differences. If the result of the omnibus test was significant (p < 0.05), a Nemenyi post hoc test was performed to determine which conditions differed from each other, while accounting for both subject-level differences and multiple comparisons. Paired comparisons were considered statistically significant when adjusted p-values from Nemenyi’s test were < 0.05. For specific comparisons between two trials, p-values were calculated using the Wilcoxon Signed-Rank test and considered statistically significant when p < 0.05.

Participants successfully recovered from most robot grasp errors, shown by one or fewer object drops per trial on average and no statistically significant difference in average drops between the Pre-error and Error trials (Figure 5A). Despite frequently recovering from these errors, errors impacted participants’ perceptions of the robot; Error condition trust levels dropped significantly from the Pre-error baseline (p = 0.003, Q = 3.30), shown in Figure 5B. Perceived workload and trial completion time, however, did not significantly change (Figures 5C,D).

Errors significantly affected many user behaviors. During the Pre-error condition, we consistently observed robot-led cooperation, with no human wrist motion detected at all (Figure 5E), meaning α ̇ = 0 . When errors occurred, total wrist motion significantly increased (p = 0.01, Q = 2.83), indicating a change to human-led cooperation with the robot ( α ̇ > 0 and γ ̇ = 0 ). When participants did move their wrist, their reaction time occurred around 20%–30% of normalized time after robot movement was completed. We did not observe any instances of collaboration, where human and robot movements overlapped.

Many exertion-related behaviors were also affected by errors. Total force, relative time to peak normalized force, peak normalized force, and initial force activity all showed increased during error trials, shown in Figures 5F–I. Furthermore, the increases in total force (p = 0.003, Q = 3.30), relative time to peak normalized force (p = 0.0001, Q = 4.01), and initial force activity (p = 0.003, Q = 3.30) were statistically significant (Figures 5F,G,I). On the other hand, most muscle activity metrics were not significantly affected by the errors (Figures 5J–L), with the exception of sustained peak normalized muscle activity (p = 0.008, Q = 3.00), shown in Figure 5M.

During the Pre-error condition, subjects chose to cooperate with the robot, holding their wrist stationary and allowing the robot to lead by taking complete control of actuating the gripper closed. In the Error condition, they also initially opted for robot-led cooperation, however, during the 44% of instances where the robot failed to accomplish the task alone, they utilized body power to reverse the roles and lead the remainder of the gripper actuation. This human-led motion against a stationary robot agent applying passive resistive force allowed the human to successfully recover the task from robot error.

Following errors, in the final Post-error condition, subjects reverted to constantly utilizing robot-led cooperation, without actively moving their wrists. Participants dropped very few objects (Figure 6A), and this number did not vary significantly. When robot behavior returned to normal in the Post-error trials, trust also returned to the original baseline levels, pictured in Figure 6B. No significant difference between trust levels was observed in the Pre-error and Post-error conditions.

Most behavior metrics paralleled trust and returned to their Pre-error baselines, shown by similar value ranges between the two conditions and a lack of statistical significance in the comparison. Wrist motion returned to the zero motion baseline (Figure 6E). Total force, relative time to peak normalized force, peak normalized force, and sustained peak normalized muscle activity also returned to similar Pre-error baseline measurements (Figures 6F–H,M).

Three metrics did not return to their baseline levels (Figures 6J–L). As pictured in Figure 6I, initial force activity showed a statistically significant increase between the Pre- and Post-error conditions (p = 0.048, Q = 2.36). Additionally, users reported significantly lower workloads (p = 0.006, Q = 3.06) and completed trials significantly faster (p = 0.04, Q = 2.47) toward the end of the study, compared to the beginning of the study (Figures 6C,D). A comparison of the first and last trials alone for these three metrics showed the same statistically significant trends. A similar comparison of the other behavioral metrics did not show statistically significant differences, except for sustained peak normalized muscle activity (p = 0.008, z = 2.52).

Changes in wrist motion clearly indicated increased human engagement during robot errors. At the beginning of the experiment, participants were reminded that they could move their wrist at any time to contribute to the grasping action, and we initially expected that they might collaborate with the robot by moving their wrist at the same time to close the gripper faster. Despite the added pressure of timed trials and experiences with robot failures, none of the participants engaged their wrist during the robot grasp phase, instead, waiting for the robot to fully complete its action before choosing whether or not to move the wrist. As such, when the robot performed grasps successfully on its own, users chose to cooperate by providing stationary resistance for the system and did not move their wrists, but a short period after the robot errors occurred, they successfully compensated for incomplete grasps by taking the cooperative lead and moving their wrist to finish closing the gripper. Although the exact cause for this behavior cannot be determined here, possible influencing factors could include prioritization of reduced workloads or hesitation to interact with the system during robotic actions, warranting further investigation of these collaborative and cooperative human-robot behaviors in the future.

Participants engaged more with the system, through wrist extension, but trusted it less when the robot exhibited errors. Reductions in trust observed in this work align with findings of other studies (Esterwood and Robert Jr, 2023; Schaefer, 2016; Esterwood and Robert, 2021) indicating the negative influence of robot errors on trust. Additionally, our results showed that trust was reduced even when the user could correct the robot’s error and still successfully complete their task goals. Similar perceived workloads, trial completion times, and object drops further indicated that errors did not significantly affect performance, despite the notable changes in trust during this same time period. This suggests that the overall performance of the wearable system did not correspond to the user’s trust in the system. It is therefore important to consider additional human-centric variables beyond device performance alone when evaluating new wearable technologies to reduce the risk of overlooking critical human factors that could impact device adoption and desirability (Motti and Caine, 2014).

The compensatory wrist movements after the robot had stopped moving caused the human operator to exert higher forces onto the device to correct for the robot’s mistake. In wearable devices where the applied forces from components controlled by the human and the robot are directed toward each other, notable human-applied force changes are an important consideration in the physical design of the system. The device structure must be able to withstand potentially increased forces during errors, which can influence material and electronic component selection. Related to user-applied force, extensor muscle engagement also increased slightly during robot error for all participants. However, these changes were not found to be statistically significant. Closer examination of the EMG signal indicated that muscle activity patterns depended more on the subtask (block position) than the robot condition. Since the positions of the forearm and wrist orient the hand, it is expected that grasping and moving objects from different locations would require varied upper limb orientations.

Despite insignificant changes in task completion (drops, workload, time) during the error cases, trust in our Co-Grasping device significantly declined when users faced these robot errors. Nonetheless, we found that trust was rebuilt to the original levels after the recoverable errors had ceased. This trust recovery ability supports the continued study of these devices with the potential to foster viable human-robot interactions that sustain human engagement. Similarly, findings from another study of unexpected errors in robotic prosthetic grasping showed that trust declined during operation of the system with errors, and increased back to baseline levels in subsequent trials where robotic behavior returned to normal (Abd et al., 2019). These two studied scenarios indicate that the negative impact of these error types on trust is not permanent. Subsequent studies on the effect of errors within wearable robotic grasping systems should therefore continue evaluating the impacts of other error types, like catastrophic errors (ex. if the robot prematurely re-opens the gripper), and longer interaction times (ex. frequent errors over the course of a day). Identifying potential and foreseeable problems that do not permanently damage users’ trust can be used to guide future human-robot interactive designs that are both effective and desirable. In addition to trust recovery, most behavior patterns returned to baseline, suggesting that users’ perceptions correctly matched their physical responses.

Lasting changes were observed in three metrics. Of the sensor-measured behaviors, the initial force activity in the Post-error condition remained significantly different than that in the Pre-error condition. These trends in initial force activity suggest that immediate behavioral reactions, whether due to learning effects or the unexpectedness of the robot error, should be investigated in the future. One perception metric and one performance metric did not return to baseline either: subjects reported significantly lower workloads and completed trials faster following errors than prior to errors, implying a learning effect. The tasks became easier for the users over time, and they began to complete the tasks more quickly. However, this learning trend was not observed in most behavioral metrics.

Wizard-of-Oz studies allow the creation of complex robotic interactions with less complex robots. Nevertheless, they can unintentionally influence behavior if users realize the robot is not autonomous. In our study, most participants seemed convinced of the robot’s voice-activated feature, evidenced by participants speaking louder or getting closer to the microphone during errors, as they believed some errors resulted from their lack of vocal accuracy. Additionally, participants expressed positive sentiments regarding the idea of a voice-activated robot. Future work should include post-study surveys to confirm these perceptions of complete automation.

We additionally did not collect any internal perception information of behavioral changes, like whether users believed they were altering their behavior or if their internal mood changed based on the robot conditions. Including these data points in subsequent works should be considered to obtain a more nuanced understanding of how users respond to robot error. Other sensory modalities such as EEG could be useful in exploring objective cognitive reactions to such errors and anchor other time-based physical reactions like those measured in this work. Future studies should also look to expand the demographic pool of participants to increase generalizability.

Researchers initially demonstrated the error and recovery method to the user in this study. Although we do not believe that this demonstration changed robotic perception based on increased trust after initial device-worn trials, future work should include unseen errors and lengthen the time of the study to allow participants to fully discover device functionality and potential robotic flaws, as well as how to organically recover from them.

Finally, this work presents an initial investigation into the human-led recovery and response to robotic error, however, many types of robotic errors may arise beyond the one evaluated here. We suspect that human responses may change with respect to different types of errors, particularly when considering the context of such errors. Studying other error types and varying the situations in which they arise will be very informative in the design and adoption of new wearable robotic tools.