Comparing to traditional peripheral devices such as a mouse or a keyboard, finger gesture recognition (FGR) is much more convenient and natural for users to control an artificial limb and to interact with a computer (Rechy-Ramirez and Hu, 2015). As a result, FGR becomes more and more important during the past few years (Rechy-Ramirez and Hu, 2015). Currently, the most common methods of FGR are visual-based, voice-based, and surface electromyography (EMG)-based ones. Among them, surface EMG is the comprehensive photoelectrical signal of potential muscle action on the surface of the skin (Botros et al., 2020). It is a kind of non-stationary signal, and its strength is sensitively proportional to the degree of muscle activity, which makes it can accurately represent the gesture of fingers (Botros et al., 2020). Therefore, surface EMG-based is widely adopted to achieve FGR.

Surface EMG-based FGR has been researched for many years. Among existing approaches, machine learning-based approach is very popular and successful (Qi et al., 2020; Wong et al., 2021). For example, Phinyomark et al. (2011) applied the critical index analysis and fractal dimension to extract the characteristics of surface EMG signals, and seven kinds of gestures were recognized from eight-channel EMG signals. Ishii et al. (2012) divided hand motions into six movements and classified finger motions using two types of characteristics. Khushaba et al. (2016) proposed the mutual component analysis (MCA) by improving the principal component analysis (PCA) to deduct the noise and redundant features. The recognition accuracy reached 95% for 15 kinds of gestures by combining the feature selection and MCA from eight channels of the surface EMG signals. Ngeo et al. (2014) used the multi-output convolution Gaussian process to analyze the dependence of multi-joint gesture and to estimate the finger joint motion. Through the correlation between knuckles, the regression model was modified to improve the recognition rate of finger posture. AlOmari and Liu (2015) constructed a model by combining genetic algorithm, particle swarm optimization, and support vector machine (SVM). Arozi et al. (2020) identified the hand gesture through the single channel of the surface EMG signal with the time-domain feature extraction, PCA, feature dimensionality reduction, and neural network. The recognition accuracy is 86.7% for nine kinds of gestures.

Recently, since convolution neural network (CNN) was proposed by Krizhevsky et al. in 2012 (Atzori et al., 2016), it has achieved great success in many fields of image recognition, natural language processing, and language translation (Wu et al., 2019b; Yao et al., 2019). As it has much better performance of feature extraction and non-linear fitting than traditional machine learning models, many researchers employed CNN to classify hand gestures from surface EMG signals. For example, Atzori et al. (2016) and Geng et al. (2016) selected CNN to classify hand gestures using the original surface EMG signals as the input signal. A spectral map that was obtained by the short-time Fourier transform (STFT) from the original surface EMG signal was put into the convolution network (Du et al., 2017; Côté-Allard et al., 2019a). Zia Ur Rehman et al. (2018) constructed a simple network model consisting of one convolutional layer, one pooling layer, and two fully connected layers. Then, the original surface EMG was directly used as the input of the CNN. Wu et al. (2018) proposed a model based on long short-term memory (LSTM) and CNN, where LSTM reserves time information and CNN extract features. Its performance was better than the model proposed in the study by Santello et al. (2016). Chen L. et al. (2020) designed a compact CNN with a small number of parameters to improve the classification accuracy of EMG signals. However, all these approaches mainly focus on developing a CNN-based recognition model while ignoring to acquire the high-precision surface EMG. Hence, they still cannot fully satisfy the required recognition accuracy for real applications of artificial limb control and human-computer interaction.

To address this issue, this study proposes a novel FGR model that consists of two parts, namely, sensing and classification of surface EMG signal (SC-FGR). First, wireless sensors with high-precision surface EMG are developed for acquiring multichannel surface EMG signals from the forearm. Second, a CNN-based classification algorithm is proposed to classify the acquired surface EMG signals for FGR, where we named it CNN-FGR. A general chart of FGR with the proposed SC-FGR model is shown in Figure 1. The surface EMG signals of each channel are segmented by a moving window. A spectrum map is generated by continuous wavelet transform (CWT) from the segmented signals of each channel. Then, the spectrum maps of multiple channels are put into the CNN-FGR for classifying.

The main research contents and contributions of this study are as follows:

In the experiments, we evaluated the proposed SC-FGR model on the collected surface EMG dataset. The results demonstrate that the proposed SC-FGR model achieves 97.5% recognition accuracy, which is much higher than that of comparable models.

The rest of this article is organized as follows: A wireless surface EMG acquisition system is designed in section “A Wireless Surface EMG Acquisition System”; The data processing and CNN-FGR algorithm are described in detail in section “Data Processing and Network Architecture”; The proposed SC-FGR model is compared with several related models in Section “Experiment and Results”; and finally, section “Conclusion” concludes this study.

The EMG is a weak electrophysiological signal of a muscle fiber group. It can be detected by sensors placed on the surface of skin or needle sensors implanted in muscle tissue (De Luca et al., 2006). The EMG signal is closely related to neuron muscular activity information so that the surface EMG signals of the forearm can be used to analyze and recognize the finger gestures.

De Luca (1997) showed that the amplitude of the EMG signal was random and could be expressed by the arithmetic mean value of zero Gaussian distribution function. The surface EMG signal is a weak signal whose amplitude ranges from 0 to 10 mV (Peak-to-Peak) or 0 to 1.5 mV [root mean square (RMS)]. The frequency range of the available energy signal is limited from 0 to 1,000 Hz, and the dominant energy is distributed in the range from 50 to 150 Hz. In the same state of muscle motion, the amplitude-frequency characteristic curve of the EMG signal is similar, and the EMG signal has a certain regularity in the muscle motion state of different detection points. According to the characteristics of surface EMG, the frame of the acquisition module is designed as shown in Figure 2.

Inspired by the surface EMG sensor on the market, the surface EMG sensor consists of the surface EMG electrode and the signal conditioning circuit. This surface EMG sensor uses three parallel silver electrodes with a spacing of 10 mm, including two measuring electrodes and one reference electrode, which prevent saturation caused by the common-mode signals. The silver electrode is put close to the skin for complete polarization, forming a capacitor by surface skin and electrode. To improve the accuracy, the front analog amplifier circuit is designed as close as possible to the silver electrode. This measure is beneficial to weaken the disturbance of white noise for the acquisition of surface EMG signals. Then, the potential difference between the two measuring electrodes is detected by the differential amplifier circuit and converted into a digital signal for signal preprocessing. Finally, the digital signal is transformed into a computer by the Bluetooth data acquisition module.

The signal conditioning circuit plays a key role in amplifying the weak signal to improve the performance of the whole acquisition system. The expected conditioning circuit is with high input impedance, high gain, wide frequency band, low noise, and high CMRR. It should amplify surface EMG signals while suppressing other noise signals (Khokhar et al., 2010). The signal conditioning circuit uses instrument amplifier AD8220 with the JFET as the input of the preamplifier. The rail-to-rail amplifier OPA364 constitutes the band-pass amplifier. The instrument amplifier AD8220 plays the role of first-order high-pass filtering, while the amplifier OPA364 plays the role of second-order band-pass filtering. All in all, the function of the analog conditioning circuit is to amplify the original EMG signal 1,000 times and then signal processing by the second-order band-pass filtering with the range of 5–1,000 Hz. The schematic diagram of the signal conditioning circuit (Fu et al., 2013) is shown in Figure 3. The theoretical gain of the signal conditioning circuit is shown as follows:

where G represents amplifier gain; R_c1,R_c2, and R_c3 represent the impedance of the capacitance C₁,C₂, and C₃, respectively;V_i1,V_i2, and V_i3represent the input of the detection points; and V_o is the output of the signal conditioning circuit. The core design principles of the surface EMG acquisition system are anti-noise treatment, such as co-ground and anti-electromagnetic interference. This EMG acquisition system uses a Bluetooth module for physical isolation and anti-interference, avoiding 50-Hz interference from a wired connection with the computer. This data acquisition system contains a 16-bit AD conversion, an ARM processor, and a Bluetooth communication module, as shown in Figure 2. The output of the surface EMG sensor is connected to the input port of the AD converter by shielding line. It adopts the common ground technology between the analog signal and the digital signal. There is photoelectric isolation between the AD converter and the ARM microprocessor to reduce the crosstalk from digital signals to analog signals. On the one hand, the ARM controller stores the eigenvalues of the collected signal and stresses it in the local SD card. On the other hand, it transfers the collected signal to the HC-05 Bluetooth module through the USRT serial communication protocol. Bluetooth communication realizes the information interaction function between sensors and the computer. The Bluetooth communication module uses low-energy radio communication technology to realize data transmission, with the maximum rate of 1 Mb/s (Song et al., 2020) and the effective communication of 15 m. The multichannel wireless surface EMG module is designed with a highly extending function and could be extended to 4–8 channels. The surface EMG device is shown in Figure 4.

The parameter comparison between the high-precision wireless surface EMG acquisition system and the other surface EMG acquisition systems on the market is shown in Table 1.

This study proposes a novel SC-FGR model that consists of two parts, namely, sensing and classification of the surface EMG signal. First, wireless sensors are developed for acquiring multichannel surface EMG signals from the forearm according to the characteristics of the surface EMG signal. These sensors can provide a high-precision signal source of surface EMG for FGR. In addition, a CNN-based classification algorithm, i.e., CNN-FGR, is proposed for FGR based on the acquired surface EMG by the developed wireless sensors. The CNN-FGR is trained on a spectrum map transformed from the time-domain of surface EMG by CWT. The experimental results demonstrate that the proposed SC-FGR model achieves 97.5% recognition accuracy on eight kinds of finger gestures with five subjects, which is much higher than that of comparable models. In the future, we plan to adopt the techniques of latent factor analysis (Wu et al., 2019a, 2020, 2021a,b), cognitive computing (Wu et al., 2021c), and attention mechanism (Zheng and Chen, 2021) to simultaneously recognize the gesture and strength of the fingers based on the surface EMG of the forearm.

The datasets presented in this study can be found in online repositories. The name of the repository and accession number can be found below: Baidu Netdisk, https://pan.baidu.com/s/1wXT_i2kPMRALvfI17bP1YA (access code: f6wu).

JF contributed to writing—original draft, conceptualization, and methodology. SC contributed to the experiment design. LC contributed to the data curation. LY contributed to writing—review and editing. All authors contributed to the article and approved the submitted version.

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.