The proposed EEG classification method is based on the WPD and the proposed deep residual convolutional networks. The original EEG signals were shared by the BCI Competition database (Blankertz et al., 2006). Firstly, EEG waveforms are segmented into sub-signals. Then the EEG signal features are obtained through the WPD algorithm, and some selected wavelet coefficients are retained and reconstructed into EEG signals in their respective frequency bands. Subsequently, the reconstructed EEG signals were utilized as input of the proposed deep residual convolutional networks to classify EEG signals. Finally, EEG types of motor imagination are classified by the DRes-CNN classifier intelligently.

The international organizations have held several BCI competitions since 2001. The International BCI Competition provides a reliable data source and a unified test standard for researchers in the field of motor imaging EEG signal analysis. The experimental data in this article comes from the databases in BCI Competition 2005 (dataset IVa) and BCI Competition 2003 (dataset III).

These two databases contain data sets recorded by five subjects (aa, al, av, aw, and ay). All five subjects performed the BCI experiment, which included three exercises of motor imagination for the left hand, right hand, and right foot. In this experiment, only the right hand (R) and right foot (F) two types of motor imagination are used for data analysis, and they are named Class-1 and Class-2. Each EEG signal has 118 channels. These motor imaging tasks are classified by using the EEG signals recorded on the C3, Cz, and C4 channels. At the beginning of the experiment, a prompt appeared in the center of the screen to inform the subject of the motion imaging task to be performed. Each test takes 7 s. During the first 2 s, the subject remained sitting still. At t = 2 s, an auditory stimulus will appear, prompting the start of the experiment. At t = 3 s, an arrow will appear on the screen to indicate which imaginary exercise the subject is performing. At the same time, the subject began to perform an imaginary movement in the same direction as the arrow prompts. The subject’s imagination time is 3.5 s. After the motion imaging, the subjects had a short rest period, which ranged from 1.75 to 2.25 s. At t = 7 s, the arrow disappears and the subject ends the imaginary action. The sampling frequency of EEG is 250 Hz. The EEG waveform is divided into time samples of 3.5 s. There are 140 experimental samples for each type of EEG signal.

Electroencephalogram signals have time-varying and non-stationary characteristics. Time domain analysis mainly considers the geometric characteristics of signal variance and mean value, and frequency domain analysis mainly considers the characteristics of signal coherence and frequency band power. EEG signals are constantly changing with time. Neither time domain analysis nor frequency domain analysis alone can accurately reflect its characteristics. Time-domain joint analysis is more suitable for reflecting the transient characteristics of non-stationary signals.

Wavelet packet decomposition (Walczak and Massart, 1997; Manthalkar et al., 2003) is an improved method based on wavelet decomposition. This method makes up for the low resolution of the high-frequency part of WPD. It can analyze the signal more accurately. For different signals, WPD can automatically select the appropriate frequency band to match the frequency spectrum of the signal, thereby improving the time-frequency resolution. WPD has a good performance in signal local analysis. It can effectively remove the redundant information and retain the feature information that is beneficial to classification to best express the EEG signal feature information.

In multi-resolution analysis, WPD is regarded as a process of stepwise orthogonal decomposition of a function space. Multi-resolution analysis decomposes the L²(R) of the Hilbert space into the orthogonal sum of all wavelet subspaces W_l according to different scale factors l. A new subspace Ulm is defined to represent the wavelet subspace W_l and the scale space V_l.

The orthogonal decomposition of Hilbert space V_l⊕W_l can be expressed as:

Define the subspace Ulm as a closure space of the function u_m(t), so that u_m(t) satisfies:

where g(k) = (−−¹)kh(1−k), g(k), and h(k) are the coefficients of the high-pass filter and the low-pass filter, which are orthogonal to each other.

When m = 0, from Equation 3, we can get:

In the process of multi-resolution analysis, the wavelet basis function φ(t) and scale function ψ(t) satisfy:

From the Equations 4, and 5, we can know that φ(t) = u₁(t) and φ(t) = u₀(t). Therefore {u_m(t)}_m∈Z is an orthogonal wavelet packet. The calculation formula of the WPD coefficient is shown in the Equation 6:

The WPD has good time-frequency resolution in both high-frequency and low-frequency parts. This method saves all the energy of the signal, so it is very suitable for the analysis and processing of EEG signals. Since the information of the EEG signal reflected by the wavelet packet coefficients on each decomposition scale is different, it can be considered to extract features from part of the wavelet packet coefficients. In the process of WPD, the decomposition scale and basis function have a great influence on the decomposition effect. The higher the scale of WPD, the better its local characteristics. But the dimensionality of the feature is also larger, which will prolong the training time of the model. Studies have confirmed that the ERD/ERS phenomenon that characterizes motor imagination actions is mainly reflected in the 8–30 Hz of the EEG signal (Pfurtscheller et al., 1997). Therefore, the decomposition scale should ensure that the frequency band corresponding to the wavelet packet coefficient is within the frequency range. Therefore, the decomposition level of WPD is determined to be four. The four-layer WPD of the EEG signal is shown in Figure 1. The signal is divided into 16 frequency bands, and the frequency range corresponding to each node in each layer is shown in Figure 1.

According to the characteristics of the original EEG signal and the wavelet basis functions, the selected wavelet basis is Db4 (Guan et al., 2015). The original EEG is decomposed by wavelet packet to obtain wavelet coefficients on various scales. The wavelet coefficients of the 0th, 1st, 2nd, and 3rd nodes in the fourth-level decomposition are retained and reconstructed into EEG signals in their respective frequency bands. The reconstructed EEG signals filter out high-frequency noise in the original signal and signals in other frequency bands that are not related to motor imagination. The reconstructed EEG signals of S40, S41, S42, and S43 are used as the input of the EEG classifier. The examples of short EEG recordings and their reconstructed sub recordings are shown in Figure 2.

Convolutional neural network (CNN) is a deep feedforward neural network (Liu, 2018) inspired by the concept of “receptive field.” With the increase of the number of layers and neurons in the deep neural network, the non-linear fitting ability will increase. CNN is widely used in engineering fault diagnosis (Cao et al., 2019), medical signal recognition (Chandra et al., 2019; Huang et al., 2019), image recognition (Qu et al., 2016) and other fields. However, simply stacking the number of network layers will cause the problem of vanishing gradients. The network can be converged by normalizing initialization and introducing a median normalization layer. But for a deeper network, the accuracy of the model will decrease as the depth increases when the network model accuracy reaches saturation. This is the degradation problem of neural networks (He et al., 2016; Yu et al., 2016). The neural network learns an implicit abstract mapping relationship by adjusting its parameters. However, this implicit mapping relationship is difficult to optimize in a deeper network. The purpose of the deep residual convolutional neural network (DRes-CNN) method is to solve the degradation problem of traditional neural networks. The learning process of the DRes-CNN is using multiple consecutively stacked non-linear computing layers to fit the residual F(x) between the input data and the mapped output data. The residual F(x) is calculated as follows:

where H(x) is the optimal solution, and x is the input congruent mapping.

The closer the residual F(x) is to 0, the closer the features extracted by this network are to the original input. The DRes-CNN composed of the residual block local units can solve the difficulty in convergence and adjustment. It overcomes the degradation problem of CNN as the number of network layers increases.

In this section, we propose the deep residual convolutional neural networks (DRes-CNN). As shown in Figure 3, the DRes-CNN is mainly composed of four residual convolution modules and a classification module. In the proposed deep residual convolutional networks, a convolutional layer with a stride of 3, a random dropout layer and a batch-normalization layer are applied firstly to compress the input EEG data and enhance the generalization of the DRes-CNN model. The down-sampling module can effectively simplifies the calculation of deep network models, reduces data redundancy, and promotes model learning (Huang et al., 2020). In the four residual convolution module, convolutional layers in series are followed by residual short circuit. Then a random dropout layer and the max-pooling layer are added after the convolutional layers. Finally, in the classification module, a flatten layer follows the convolution layer and a random dropout layer is applied after the flatten layer to prevent overfitting. In the proposed DRes-CNN model, the learning rate is set as 0.001 and the batch size parameter is set as 250.

The performance of the classification model is mainly measured by the accuracy. The accuracy was calculated through Equation 8.

where TP stands for true positive, meaning the correct classification as Class-1 of EEG; TN stands for true negative, meaning correct classification as Class-2 of EEG; FP stands for false positive, meaning incorrect classification as Class-1 of EEG; FN represents false negative, meaning incorrect classification as Class-2 of EEG (Yin et al., 2016).

In order to verify the effectiveness of the proposed EEG classification model, we classify the EEG signals of the right hand (Class-1) and right foot (Class-2). One hundred forty groups of eight feature inputs can be obtained for each type of EEG signal after data preprocessing based on WPD. All EEG training sample data is randomly scrambled, and the last 200 samples are selected as the test set. The classification of EEG signals is based on the classification algorithm described in section “Method.”

The original EEG waveform is divided into sub-signals. Then, the characteristics of the EEG signal are obtained through the WPD algorithm. The specific wavelet coefficients are retained and reconstructed into the EEG signals of respective frequency bands. Subsequently, the reconstructed EEG signal is used as the input of the proposed deep residual convolutional network to complete the classification of EEG signals. The experiment was run on a PC with 32 GB of memory and 16 GB of GPU memory.

Electroencephalogram signals of different frequency bands contain different characteristics and information. The wavelet packet can decompose the information of each frequency band. It makes the characteristics of the EEG signal easier to identify in each frequency band. In this section, the performance of training the DRes-CNN with reconstructed sub-signals as model input is studied experimentally, and the experiment results are shown in Figure 4.

From Figure 4, we can find that the accuracy of the classification test using the original EEG signal as input reaches 91.56%. The reconstructed EEG data set of S40 (0, 7.8125 Hz) reached an average test accuracy of 89.56%. The reconstructed EEG data set of S41 (7.8125, 15.625 Hz) reached the best average test accuracy of 98.76%. The reconstructed EEG data of S42 (15.625, 23.4375 Hz) set reached an average test accuracy of 93.54%. The reconstructed EEG data set of S43 (23.4375, 31.25 Hz) reached an average test accuracy of 95.95%. From the experimental comparison demonstrated above, we can conclude that the proposed DRes-CNN model shows the best classification performance when the reconstructed EEG dataset of S41 (7.8125, 15.625 Hz) is used as model input.