Ten Ottobock MyoBock 13E200 = 50 sensors were used to gather the sEMG signals. They provide on-board amplification, rectification and filtering. Sensors of this kind are a standard in clinical applications, especially in prosthetic sockets.

FMG signals were registered by ten FSR 400 Short force-sensing resistors by Interlink Electronics. Made of a robust polymer thick film, each FSR has a 5.6 mm-diameter sensitive area: when a force is applied to its surface, the electrical resistance of the FSR decreases correspondingly. These sensors are cheap (5€ apiece), but despite the specified remarkably large sensitivity range (0.2N–40N), they have non-negligible hysteresis at high forces, no guarantee of repeatability and a non-linear transfer function. Nonetheless, Castellini and Ravindra (2014) have shown that for small forces (0N–15N) their behavior is largely comparable and their transfer function is almost linear. In our setup, a small printed circuit board with a voltage amplifier (see Figure 2) provides the amplification of the FMG signals. The output of the sensor circuit is Vout=R2VCCR1+R2-R1R4VCCR1+R2×1RFSR, yielding a lowest admissible resistance of RFSR=R1R4R2=6kΩ, which corresponds to a theoretical maximum force observed on the FSR's surface of 3.33N (InterlinkElectronics, 2014).

In order to provide maximum flexibility in arranging the sensors on the subject's body, specifically on the forearm or stump, uniform 3D-printed housings have been designed for both kinds of sensors. The housings are made of flexible thermoplastic polyurethane and provide adherence to the subject's skin as they are tightened to the arm by a Velcro strap. Each housing provides braces to allow sliding on the strap, so that its position can be individually adjusted and maintained, regardless of the type of sensor (see Figure 3). The FSR sensor housing not only serves as a retainer for the sensor and the amplifier, but furthermore comprises a structured geometric body that is divided into two parts: approximately one half is shaped like a cone and pointing toward the FSR sensor's sensitive area, the other half, shaped like a hemisphere, is pointing toward the skin. This geometric shape has been specifically designed to concentrate the force exerted by the muscles on the FSR sensor's sensitive area. The bearing of this structure is realized by a surrounding, thin and flexible membrane with a thickness of 0.5 mm and a total area of 47.5 mm² (small diameter 6.3 mm, large diameter 10 mm), linking it to the housing. Even if the elastic and damping properties of the membrane have not been subject to further investigation so far, it is assumed that the membrane may increase the signal stability, i.e., it could hypothetically add a mechanical filtering to the bio-signal. This solution offers the capability to create any combination of FMG and sEMG electrodes. In order to gather signals from the complete circumference of the forearm, the sensors are placed evenly spaced around the forearm/stump. This arrangement, also called low-density surface electrode layout or uniform electrode positioning (Fang et al., 2015), has already been proven effective for robotic hand prosthesis control in a number of previous publications (Castellini and van der Smagt, 2009, 2013).

The typical voltage output of the FSR/amplifier/housing complex has further been characterized. The measurements have been performed with a Zwick Roell ZMART.PRO compression test device providing fixture of the sensor setup as well as controlled exertion of force with an accuracy of 100 mN. The corresponding output voltage of the device has been measured with a FLUKE 289 Multimeter with an accuracy of 1 mV. The relationship between voltage output and applied force, in a range from 0 to 5.2 N, is linear with a residual average error of about 7% (see Table 1). It is worth noting that, during the experiment (see below), no FSR ever reached the saturation point, meaning that the FMG signals have all been correctly captured. This matches to a large extent the results obtained in Castellini and Ravindra (2014) and Ravindra and Castellini (2014) for a similar device.

A Bluetooth ADC board (Figure 4), consisting of a Texas Instrument MSP430F5529 microcontroller and an on-board Bluetooth chipset, provides analog-to-digital conversion (ADC) of the signals of both the sEMG and FSR sensors, and their wireless transmission. As the microcontroller natively supports only 15 AD-channels, AD-conversion of up to 32 sensors is realized via analog multiplexing, providing a maximum sampling rate of 192.5 Hz for each channel. Since the sEMG sensors already provide rectified and filtered signals with an evaluable bandwidth limited to 10 Hz, the provided sampling frequency is an overshoot (see e.g., Castellini and van der Smagt, 2009). The same argument obviously holds for the FMG signals (Ravindra and Castellini, 2014).

The board employs two UARTs, communicating in turn with the smartphone (via a serial-over-Bluetooth connection) and the prosthesis (in our case, via a simplified RS232 protocol). Hence, the board can also relay control commands to a hand prosthesis. The final cost of the system is estimated to be below 150€.

The wearable myocontrol system is completed by a standard commercial smartphone (Huawei Honor 6 with a commercial value of about 300€), on which the data processing is performed. This device is equipped with a quad-core Cortex-A15 processor running at 1.7 GHz, 2 GB of RAM and a 3100 mAh Li-Po battery, claimed to keep the smartphone running for 2 full days at moderate usage. Its operating system is Android 4.4 KitKat. The smartphone weighs 132 g and easily fits in a pocket (7 × 0.8 × 14 cm) with its 5-inch display. A C# application similar to the one used in Strazzulla et al. (2016) was implemented, optimized and ported to the smartphone. The application enforces the following functionalities: (a) receiving and storing the data from the ADC board's serial-over-BT port; (b) displaying a visual stimulus, both on the smartphone screen and using the prosthesis; (c) building a prediction model for the control commands; and (d) sending them off to the prosthetic hand at a 10 Hz frequency, through the ADC board's serial-over-BT port. The machine learning method of choice was—coherent with our own previous work—Incremental Ridge Regression with Random Fourier Features (Gijsberts et al., 2014). As speculated in previous work and now proven, this method provides real-time capable non-linear multivariate regression while saving a lot of computational resources to the point that the maximum usage of the smartphone's CPU showed to be at 14%. Moreover, with the program running and the cell phone display activated and fully lighted, the battery endurance is at approximately 6 h, whereas an endurance of 11 h can be achieved with the display being switched off. For receiving the sample data and sending control commands to the prosthetic hand, the cell phone's internal Bluetooth peripheral has to be activated all the time.

Ten intact subjects, nine of which were right-handed (subject No. 9 being the left-handed one), joined the experiment (3 females and 7 males, 28±7 years old, weighing 72.4±9.91 kg, 177.8±12.14 cm tall). Each subject received a thorough description of the experiment, both in oral and written form. Informed written consent was obtained from all participants. Experiments with sEMG and FMG were approved by the Ethical Committee of the DLR.

The experiment consisted of performing ten times the following sequence of wrist and hand movements: (1) wrist flexion, (2) wrist extension, (3) wrist pronation, (4) wrist supination and (5) power grasp. To enforce the opening of the hand the relaxed stance was used, in order to mimic a more natural form of myocontrol. Each movement was visually stimulated on the screen of the smartphone (the name of the required motion would appear on the screen), while the experimenter was visually checking that the movement was actually being enforced, to ensure a correct execution. Each stimulation was administered as follows: the visual stimulus would appear for 2 s to allow the subject reach the full movement, then for 6 s data were captured representing the maximal activation for that particular movement (“activation phase”: only this phase was considered in the offline analysis), then the stimulus would disappear for 2 s to allow the subject return to the resting position. The sequence was administered in the same order to all subjects. The choice of this set of movements was motivated by the well-known importance of controlling at least the wrist pronation/supination (see e.g., Jiang et al., 2012b) together with grasping; for instance, pronation and supination of the wrist are operated by deep muscles (Biryukova and Yourovskaya, 1994), meaning that they are usually hard to detect using sEMG. It is worth mentioning that, to the best of our knowledge, there is so far no commercially available 2-active-DOFs prosthetic wrist, but a few prototypes are being studied [see e.g., the device embedded in the DEKA arm, https://www.youtube.com/watch?v=KCUwoxuAdYQ, and the prototype by Ottobock which appears for instance in Amsuess et al. (2016)].

For this specific experiment, the sensors were separated in two different bracelets, the first one with ten sEMG sensors on the left forearm and the second one with ten FSR sensors on the right forearm. The bracelets were located approximately 10 cm below the subject's elbows. This further choice, rather than that of placing twenty sensors on one single forearm, was motivated by the relatively small space available on the forearm of some of the subjects, which would have potentially limited the adhesion of each sensor to the subjects' skin. (A similar problem was reported of, e.g., in Castellini and Ravindra, 2014). Of course, this diminishes the comparability of the results from the two sets of sensors, but myocontrol literature has already presented cases in which, for instance, training of a machine-learning-based method has been performed bilaterally, i.e., gathering data and training from the intact forearm and predicting using data from the impaired limb (Castellini et al., 2009; Nielsen et al., 2011). In both these works, no difference was reported in performance whether the forearm to be used was the dominant or non-dominant one.

During the whole experiment, the subjects sat in a relaxed position with their forearms over their thighs and the hands in a lateral position (with the palms looking toward each other); they were advised to perform each movement bilaterally (Figure 5). The recorded data kept trace of the FSR and sEMG signals as well as of a numerical identifier univocally representing the stimulated movement. This index was used as the ground truth during the supposed maximal activation of the muscles—an instance of on-off goal-directed stimuli as already used in, e.g., Sierra González and Castellini (2013).

The signals recorded during the experiment were stored to the smartphone's internal memory, then analyzed off-line. Low-pass filtering was applied to both signals (3rd order low-pass digital Butterworth filter with a cutoff frequency of 1 Hz) in order to remove high-frequency disturbances. This is a standard procedure in the field (see e.g., Atzori et al., 2014). For the data processing and the subsequent statistical analysis, the following approaches have been chosen:

Additionally, at the end of the experiment, three surveys were administered to each subject, in order to complete the envisaged system assessment with respect to its usability: the System Usability Scale (SUS) (Brooke, 1996), the NASA Task Load Index (NASA, 1986) and a reworking of the Microsoft Desirability Toolkit by Travis (2008). The SUS consists of ten questions (Table 2) with answers represented on a 5 point Likert scale (1 - strongly disagree to 5 - strongly agree). The scoring in this survey is such that the answers to the strongly agree positive questions and to the strongly disagree negative questions generate a higher impact over the final score. The NASA Task Load Index provides an overall workload score on six subscales: Mental, Physical and Temporal Demands; Own Performance, Effort and Frustration. For each subscale, the answer could be in a range of 21 points, reaching from very low to very high (Table 2).

Lastly, Travis's survey consists in a series of “reaction cards” with adjectives that could be applied to the system to be tested; the user is asked to select the five cards that most closely match their personal reactions to the system. For the experiment, we used a list of 75 adjectives instead of the cards (most of them based on Travis's questionnaire), then the subject was asked to choose all adjectives he or she felt more related with the device. After that, in a more precise selection, the user had to choose only the 5 most important words and try to give a simple reason about his or her decision.