Approximately 130,000 people suffer a Spinal Cord Injury (SCI) worldwide every year. Nearly half of these SCI cases are at the C6 level or above, resulting in significant paralysis, impaired quality of life, and need for self-care assistance (ICCP, 2017). Moreover, patients with C6 or higher cervical level of SCI lack the critical ability to grasp objects that prevents them from living independently (Nas et al., 2015). Indeed, several studies on SCI patient priorities have consistently reported that upper limb strength and dexterity restoration is the most desirable function to regain (Anderson, 2004; Snoek et al., 2004; Simpson et al., 2012; Collinger et al., 2013; Blabe et al., 2015). In a survey of individuals with tetraplegia following SCI, more than 75% indicated that Functional Electrical Stimulation (FES) neuroprosthetics for hand grasp would be “very helpful” to restore function that would positively impact quality of life (Collinger et al., 2013). However, the FES systems that have been demonstrated to date are either limited to providing only a few hand functions or lack the ability to enable dynamic motor control for performing complex functional tasks that require synergistic integration of paralyzed and non-paralyzed muscles.

Advances in Brain Computer Interface (BCI)-controlled FES technology offers a potential new way to reconnect the brain directly to the paralyzed hand/arm, restoring functional hand use. FES devices with control mechanisms other than BCI (e.g., myoelectric, sip-and-puff, eye trackers) have been proposed, but are less desirable due to increased cognitive load and non-intuitive mapping between thought and action (Ajiboye et al., 2017). Thus, BCI approaches are preferred for their ability to provide a more intuitive and “high-fidelity” control signal that can allow for more complex and clinically-relevant functional limb movements (Chadwick et al., 2011; Ethier and Miller, 2015). Indeed, in recent surveys a majority of paralyzed patients showed interest in using a BCI technology that can help restore lost hand/arm function (Collinger et al., 2013; Blabe et al., 2015).

Several groups have investigated BCI-FES neuroprosthetics for restoring hand grasp function in paralyzed humans with varied success. Some groups have coupled an electroencephalogram (EEG)-BCI with FES systems and showed that the paralyzed participants were able to use the systems to enable up to two functional hand movements by imagining hand/arm movement (Müller-Putz et al., 2005) or by imagining a non-intuitive motion such as foot (Pfurtscheller et al., 2003) or cursor movement (Lauer et al., 1999). However, the low dimensional control signals of the EEG as well as non-intuitive mapping of thoughts-to-action makes it unlikely that these BCIs could provide naturalistic continuous control for complex hand functions. An alternative approach, utilizing electrocorticography (ECoG)-based signals, can provide better spatial resolution compared to EEG and thus a potential neuroprosthetic control mechanism based on high quality neural signals. Indeed, a paralyzed participant using an ECoG-BCI controlled transcutaneous FES system was successfully able to perform three movements (hand open, palmer, and lateral grasps) (Márquez-Chin et al., 2009). However, this demonstration was done in an offline mode where ECoG signals recorded from an able-bodied participant were used to control FES-evoked movements of the paralyzed participant. Therefore, the applicability of ECoG-BCI for real-time control of multiple hand movements via FES orthotics remains to be demonstrated. To overcome the limitations of EEG/ECoG control, researchers have implanted intracortical microelectrode arrays (MEAs) that can allow for higher information transfer rate (Baranauskas, 2014) and a more precise detection of movements for decoding and controlling hand/arm FES systems. In a prior study, we showed proof-of-concept that a person with C5-level paralysis could use a MEA-BCI to control a transcutaneous FES system to enable six independent finger, wrist, and hand movements (Bouton et al., 2016) We also demonstrated that the system could be used to perform a functional grasp-pour-and-stir task, providing the user with simultaneous, differential control of Hand Open, palmar grasp, and lateral key grip. A similar study showed proof-of-concept that a person with C4-level paralysis could use a MEA-BCI to control a hybrid exoskeleton and implanted FES system to evoke upper limb reaching, Hand Open, and lateral key grip (Ajiboye et al., 2017). The participant in this study used these movements to perform functional feeding tasks. However, no prior study has provided careful quantification and characterization of MEA-BCI enabled FES upper limb motor control to allow for study reproducibility and comparison with other neuroprosthetic devices.

In this study, we show a critical step in the clinic-to-home translational path of BCI-FES neuroprosthetics by demonstrating that a patient with tetraplegia can achieve volitional control of seven hand functions using an easy to train, cortically-controlled, non-invasive, FES orthotic. We used a MEA, implanted in the motor cortex of a 26-year old study participant with a C5-level SCI, to record neural signals. We then used machine learning algorithms to translate the neural activity to intended movement commands. These commands were then used to control the transcutaneous FES orthotic wrapped on the participant's forearm which stimulated the appropriate muscles to evoke the intended movement (Figure 1). With the system, the participant was able to use a trained decoder to volitionally select up to seven distinct functional hand states and use them to manipulate multiple objects of varying size, shape, and weight. The participant's functional gains were assessed using the Grasp and Release Test (GRT; Stroh-Wuolle et al., 1994), a standardized test developed for evaluating neuroprosthetic performance by patients with SCI. We found more efficient grasp and transfer of objects using the BCI-FES compared to the participant's baseline. Our results also revealed important insights into the neural representation of different hand movements. In particular, we observed that a robust mapping of multiple hand movements can form under the implanted MEA in a very small area of the motor cortex. We found overlap between representations for objects of similar size and weight and we report a strong correlation between the discriminability in the neural representations of hand movements and decoder performance.

The cortically controlled FES system consisted of three main components: (1) an implanted 96-channel Utah MEA for recording neural signals, (2) a computer running data processing and a machine learning algorithm to decode the user's intended movement from the neural activity, and (3) a non-invasive FES cuff wrapped on the participant's forearm to stimulate the appropriate muscles to evoke the desired hand movement (Figure 1). Wavelet decomposition was used to process the raw cortical data into MWP neural features (see section Methods). These features were used as inputs to a SVM decoding algorithm that translated the neural activity to the user's intended movement, which was then used to control the electrical stimulation of the user's forearm (Figure 1). No device-related adverse events occurred during the duration of this study.

The ability to successfully manipulate multiple real-world objects encountered during activities of daily living remains a key challenge limiting the practical applicability of BCI-controlled FES devices for people living with tetraplegia. In our previous studies, we demonstrated proof-of-concept that implanted BCI-transcutaneous FES technology can achieve motor control of a paralyzed upper limb after SCI (Bouton et al., 2016). We focused on demonstrating that differential control of individual wrist, finger, and hand movements could be achieved, but did not attempt to quantify or characterize the behavioral or neural features of motor control. In this study, we advance prior knowledge by applying standardized tasks developed for neuroprosthetic studies (GRT object manipulation) to the evaluation of system performance. In this way, we not only allow for comparison between our BCI-FES technology and other neuroprosthetics but also develop a new understanding of the strengths and limitations of the BCI-FES system. We showed that the participant in our study could train to use the BCI-controlled FES system to perform functional tasks that required dynamic integration of FES-enabled paralyzed hand/arm muscles with non-paralyzed shoulder/elbow muscles. The system enabled the participant to select the desired hand movement, out of the seven possible trained movements as well as a rest state available to him, using motor intent. The BCI-FES also enabled the participant to manipulate objects of different sizes, shapes, and weights with skilled, forceful grasps. In addition, our study revealed insights into the neural representation of hand movements in the motor cortex. We showed that stable representations of different hand movements can form in a very small area of the motor cortex under the implanted MEA. Furthermore, we demonstrated that discriminability between these neural representations can affect decoder performance.

Because the test objects varied widely in size and shape, the FES system was calibrated for each object to evoke a unique hand shape/movement pattern that provided grip force and dexterity to enable palmar, lateral, and tip-to-tip type grasps. The FES system calibration for each grasp involved precise targeting of separate muscle groups in the forearm to evoke specific finger movements (see Figure 3 and Figure S1 showing the target muscle groups for each grasp). The use of MWP as neural features for decoding provided a high-fidelity spatiotemporal neural modulation signal that was strongly correlated with neuronal spiking activity, and could be used to discriminate between different hand movements in real-time without the need for thresholding or explicit spike sorting (Figure 4). During decoder training, the participant attempted to evoke the correct grasp for a particular object from seven movement states (plus rest) available to him. The results from decoder training show that the participant was able to use the decoder to control the system with high accuracy—the individual accuracy scores for each movement were all >96% (Table 1) during different neural decoder training sessions across 4 days. Several groups have observed that neuronal states associated with different imagined hand movements may be represented discriminably in the human brain (Klaes et al., 2015; Bleichner et al., 2016; Leo et al., 2016). Our finding that multiple hand movements can be decoded reliably from the motor cortex is further validation of these observations. The consistently high accuracy of the decoders in classifying individual hand grasps not only indicates the robustness of the neural representations in the motor cortex, but also suggests that this modulation can be reliably leveraged for precision control of a FES neuro-orthotic device that can restore multiple hand-grasp functions. The results also highlight that for our trained participant the decoder training time for the multiple object manipulation task was limited to 12–15 min. These results have implications as high accuracy and minimal training time are features that are desirable for potential users of neuroprosthetic devices (Collinger et al., 2013).

We used the standardized GRT to demonstrate the participant's ability to successfully use the trained decoder to manipulate multiple objects. The use of a standardized measure of functional outcomes not only helped us better assess the performance of our system but also provided standardized reportable scores that can facilitate objective comparison with other similar technologies, help identify areas of improvements, enhance reproducibility of research, and aid in decision making for clinicians and potential end-users. The results show that using the BCI-FES system, the participant was able to evoke the correct movement to manipulate each of the six test objects using naturalistic grips (Figure 5 and Movie S2). It should be noted that the participant is able to transfer some of the objects on his own without using the BCI-FES system (see Movie S2, right panel showing the participant manipulating the objects on his own without FES). For example, the participant used adaptive strategies (such as biceps-mediated forearm supination with shoulder abduction/adduction) to easily grasp and release the Peg and Block. However, heavier objects that required a more forceful grip were difficult to transfer without the system (Figure 5B). Using the BCI-FES system, the participant was able to transfer the heavier objects (VHS, Paperweight, Can, Fork) and also showed significant improvement in transfer times (Figure 5C).

The transfer speed using the BCI-FES system during the GRT compares favorably to those reported for other BCI-FES systems. For example, our participant transferred the Paperweight at a rate of 4.7 ± 1.2 s per transfer compared to ~36 s it took a participant to transfer the Paperweight using the EEG-Freehand system (Müller-Putz et al., 2005). It should be noted that the neurologic level of the participant in Muller-Putz et al. (C5 ASI A with residual shoulder and elbow movements) is functionally similar to the neurologic level of the participant in our study (C5 ASI A with residual shoulder and elbow movements). While the participant in our study had a twitch of wrist extension (yielding a zone of partial preservation to C6), he was not able to elicit the tenodesis grip of a person with a C6 neurologic level. The improved performance on the GRT with our BCI-FES system carries further significance as compared to the EEG-Freehand system where the participant only has a single grasp available, our participant had seven hand functions available to him and he can voluntarily choose the one which provides him the optimal grip for the target object. The transfer speed with our system also compares favorably with a BCI-controlled robotic arm used by paralyzed individuals—for example, a transfer rate of 6–10 transfers per minute for the Block vs. 0.1–1 transfers per minute reported for the BCI-robotic arm used to transfer similar sized blocks during the Box and Block (BBT) test (Wodlinger et al., 2015).

Analysis of the participant's neural modulation, as he imagined different hand movements, revealed interesting insights into the neural representations of hand grasps in the motor cortex. The PCA revealed overlaps in the MWP clusters of the grips for Paperweight–Block and the Can–VHS (Figure 7A). The similarities in neural representation for hand grasps were consistent with the spatial distribution of MWP on the cortical array where we observed a group of channels that appear to modulate similarly between these grasps (Figure 7B). The analysis of the differences in MWP modulation confirmed that the neural representation of the imagined Block and Paperweight as well as the Can and VHS grasps were, indeed, the most similar of all grasps (Figure 7C). This was consistent with hand morphology observed during performance of the grips, with Block and Paperweight grasp patterns being a synergy of lateral key and tip-to-tip precision grips and Can and VHS grasp patterns representing versions of a palmar power grip. This similarity in the neural representations for certain hand grasps may be one of the factors underlying the misclassification in decoding that we observed during the GRT test blocks. Other groups have made similar observations. For example, Leo et al. observed a clustering of neuronal representations based on postural differences in hand shapes (i.e., precision grasps and power grasps) which in turn affected the ability to correctly classify these hand shapes during decoding (Leo et al., 2016). Similarly, Bleichner et al. used an ECoG-based BCI to classify four different hand gestures and noted that the gestures that correlated strongest in neural representation were misclassified most often (Bleichner et al., 2016). The results not only expand on the total number of hand movements for which a stable representation could be observed in the motor cortex, but also show that it is possible to study and decode neural representations in a very small area under the implanted MEA. Regardless, our findings that there are overlaps between the MWP spatial patterns for some hand movements highlights that additional neural features (such as signal propagation or phase) and/or other decoding algorithms (such as deep learning algorithms) might need to be explored to expand the repertoire of hand functions that can be reliably decoded using a single MEA.

Interestingly, the neural representation for Hand Open in our experiments was the most distinct from the six other hand grasps (Figure 7C). We hypothesize that this is due to the Hand Open posture being morphologically distinct from all other grasps. In addition, the FES pattern for Hand Open was primarily over wrist and hand extensors, while the FES pattern for grasps often included both flexor and extensor muscle compartments. Therefore, somatosensory and muscle stretch receptor feedback from stimulation of sensate areas could have propagated to the motor cortex and differentially influenced neural activation patterns in Hand Open vs. grasp states. Not surprisingly therefore, during neural decoder training we observed the highest accuracy when the participant attempted a Hand Open movement (Table 1). Overall, there was a strong correlation between the discriminability in neural representation of different hand states and the corresponding decoding accuracy (Figure 7D). It should be noted that, compared to GRT test blocks, we rarely observed this misclassification among hand movements during the decoder training. This may be due to the differences in how the training and GRT test blocks are performed. First, in contrast to training blocks, the participant does not receive visual cues to initiate, sustain, and terminate the grasp during the GRT test blocks. Second, the decoder training is more structured and motor imagery is more consistent and deliberate as the participant must grasp and transfer the cued object once during the cue period. The GRT test block, however, may be more challenging for the participant because he must quickly and repeatedly activate and deactivate decoders to transfer objects as many times as he can in a 30-s test window. We believe that it is the combination of the lack of reinforcing visual cues, and the rapid task switching during the “beat the clock” condition of the GRT that increases the misclassification probability of the decoder for grasps with most similar neural representations.

Enabling grasp and manipulation abilities using BCI-FES technology is challenging compared to, for example, a 3-D reaching task or individual finger/joint movement as it not only requires high fidelity control signals and strategies (Schaffelhofer et al., 2015), but may also require additional sensorimotor information related to the shape of the target object that may be needed to preshape the hand correctly (Leo et al., 2016). In addition, a reaching task in space involves coordinating only three degrees of freedom (DOF) whereas control of an anthropomorphic hand requires control of 23 DOF (Vargas-Irwin et al., 2010) thereby increasing the complexity of the problem. Not surprisingly therefore, there are only a few reports of successful demonstration of BCI-enabled hand grasp, most of which were limited by the number of functional hand movements that could be enabled (Bouton et al., 2016; Sharma et al., 2016; Ajiboye et al., 2017; Friedenberg et al., 2017). Not only are the number of hand functions regained by our tetraplegic participant to manipulate objects substantially more than achieved by any previous study of FES devices, but we also show that this improvement did not come at the cost of accuracy, speed, or training time. Our results also have implications beyond reanimation in tetraplegia. The enhanced understanding of the neural representation of hand gestures in the human brain and the ability to accurately decode these movements can provide a novel control signal for the development of other BCI tools; for instance, communication based on sign language (Bleichner et al., 2016). Another advantage of using an intuitive BCI paired with real-time FES is the potential to promote synaptic neuroplasticity in the cortico-spinal tract (McGie et al., 2015) or to promote neuroprosthetic “learning” in the motor cortex (Ganguly et al., 2011).

It is important to note that this study is limited to one participant who had over two years of experience using the BCI to evoke hand and forearm states prior to performing GRT testing. Novice BCI users may take longer to achieve the same level of hand dexterity for object manipulation as described here. In addition, the transcutaneous FES cuff used in this study is designed to stimulate the paralyzed forearm muscles to control hand and wrist movements. It is therefore best suited to persons with C5 or lower levels of tetraparesis and who have some residual shoulder and biceps movements. Testing to assess whether the FES cuff can be used along with shoulder and triceps stimulation or a gravity assisting device in SCI patients with higher level of injury remains to be investigated. Our device was also limited by the lack of thenar (base of thumb) muscle stimulation, limiting the quality of hand grasps for small objects requiring precision grips (no objects of this type are represented in the GRT). The need for daily retraining of the decoders is another limitation of the current system that will need to be overcome in order to reduce setup time and facilitate translation of the device for daily use.

In summary, our BCI-FES neuro-orthotic device significantly improves upon the state-of-the-art for assistive devices capable of meeting tetraplegic individual's desired priorities of restoring multiple, voluntary, and naturalistic hand functions. We also demonstrate that our BCI-FES system can enable functional, skilled hand grasps that can generate adequate force to manipulate everyday objects with high-precision and practical speed. The fact that the participant could use the system to perform functional tasks ~900 days post-implantation further highlights the translational potential of our system. Future directions include addressing system limitations to make the next generation BCI-FES robust to daily neural signal variability, portable, wearable, with more electrodes and sensors, and less obtrusive to further facilitate clinical translation.

GS conceptualized the study; GS, HB, SC, and MB designed the scientific experiments; SC, MZ, GS, and MB performed research; DF and MS developed the decoding algorithms; NA contributed to developing the stimulation hardware and software; AR performed the surgery; SC, MZ, NS, and GS analyzed data; WM and MB were involved in participant recruitment; GS and SC wrote the manuscript. All authors contributed to editing the manuscript.