Considering that convolutional neural networks (CNNs) are widely utilized to capture the EEG features of time series data (Hu et al., 2023), we designed a base feature extractor (BaseFE) to extract the key EEG features. The BaseFE network is designed in a shallow network structure. As shown in Figure 1, the S-CNN module contains five layers: a 1-D convolution layer, a batch normalization layer, a Softshrink activation layer, and a dropout layer.

The Conv1D (32, 64, 2) in Figure 1 refers to 1-D convolution layer with 32 filters, a kernel size of 64, and a stride of 2. Setting the kernel size to 64, half of the 128 Hz data sampling rate, enables the model to extract the EEG frequency features in the range above 2 Hz. Then, a batch normalization layer and a Softshrink activation layer are followed. The batch normalization is utilized to normalize small batches across each feature dimension, effectively mitigating internal covariate shifts (Ioffe and Szegedy, 2015). Softshrink function is a non-linear activation function that is mainly used for sparse representation of EEG data and noise suppression. To mitigate model overfitting, a dropout layer is introduced in the following layer.

Inspired that deep residual shrinkage networks achieved high fault diagnosing performance in vibration signals (Zhao et al., 2020), we introduce the residual shrinkage network to improve the feature learning ability from single-channel EEG signals. Both the attention mechanism and automatic soft thresholding are integrated into the residual network to adaptively eliminate redundant information and selected the most discriminative useful features during feature learning. This residual structure is to prevent vanishing gradients and exploding gradients in deep network.

Based on the fact that the importance between each channel of the features learned from the EEG signal is different, we use a residual shrinkage building unit with channel-wise thresholds (RSBU-CW) in this work. In particular, the squeeze and excitation (SE) network is adopted to obtain a set of thresholds related to the individual channels by modeling the inter-dependencies between the features (Hu et al., 2020). Soft thresholding incorporated in the RSBU structure can adaptively eliminate redundant information and select highly discriminative features.

As is shown in Figure 1, two convolutions Conv1D (32, 1, 1) with a kernel size of 1 and a stride of 1 are implemented in this block. Assuming that the BaseFE module generates a feature map I ∈ ℝ^L × T, we apply two convolutions operations (Conv1 and Conv2) to I to obtain U (U∈ ℝ^N × T) in Equation 1:

where T represents the length of U, and N represents the total number of features.

Then, the feature map X is squeezed to a 1-D vector using an absolute operation and an adaptive average pooling layer. The excitation operation is used to capture the correlation between the individual channels, which can retain the channels with the most useful feature information and suppress the channels with less feature information (Eldele et al., 2021). In this study, two fully connected (FC) layers are added for information aggregation. The first FC layer, followed by ReLU, aims to reduce dimensionality, while the subsequent layer, followed by a sigmoid function, aims to raise dimensionality. The output of the FC layers is scaled to between (0, 1) with the scaling parameters. And the scaling parameter is described in Equation 2:

where z_r represents the feature at the rth neuron, and σ_r represents the rth scaling parameter. To make the soft threshold positive and not too large, the scaling parameter σ_r is multiplied with the average absolute value of U_r to obtain the threshold. The threshold used in RSBU-CW is expressed in Equation 3:

where τ_r represents the threshold for the rth channel of the feature map, and i, r represent the indexes of length and channel of the feature map U, respectively. Rather than the artificial design of filters by experts, the threshold is automatically determined through the SE attention mechanism. The mechanism is that the CNN automatically conducts filter learning and transforms the original data to a new space for soft thresholding. The soft thresholding function formula can be expressed in Equation 4:

In this study, τ is a parameter that can be learned by means of automatic learning. The thresholds can be kept in a reasonable range, thereby preventing the output of soft thresholding being all zeros. Finally, the input I is combined with the enhanced features O learned from the residual unit by adding an identity shortcut connection. The final output of this residual unit is expressed in Equation 5:

To avoid overfitting, the GAP is used to replace the FC layer, and the number of model parameters is further reduced significantly. The GAP reduces the N dimensional filtered signal to N feature points through an average pooling operation. It not only helps to improve generalization capability of the network but also allows an EEG-based Class Activation Map (ECAM) interpretable method to reveal learned patterns in terms of the driver’s mental state.

In the fully connected layer, the weight freezing method is introduced to suppress the update of some learnable parameters by freezing some weights during the backpropagation process. Assuming the features inputted into the FC layer are denoted as F = {f₁,…, f _n }, where F∈ℝ^L × N. L denotes the size of the mini-batch, and N is the feature dimension. W _n denotes the weights and the vector of inputs of the FC layer, respectively. Weight freezing is proposed as a regularization method to improve classification accuracy, which can be implemented in Equation 6:

where K is a mask matrix with identical dimensions to W _n , and the elements of K are uniformly distributed within the range [0,1]. ⊙ is the element-wise multiplication and η denotes the learning rate of the optimizer.

When an element is masked, it cannot be updated during backpropagation. Here, we set the threshold value t (=0.2) of K, which determines the number of frozen parameters in W _n . In the FC layer, we utilize the weight freezing method not only to make sparse connections but also to improve classification accuracy. Finally, the cross-entropy loss function with label smoothing regularization of parameter α (=0.1) is utilized to optimize the classification model, which can be described in Equation 7:

where y_c is the true label, p_c is the predicted probability of the model for the class c, and C is the number of classes.

In this section, we compared the classification accuracy of five deep learning models tested on the balanced dataset for a standard evaluation. Each model was trained from 1 to 20 epochs, with network parameters randomized for each iteration. We conducted each evaluation by repeating each model on every subject 10 times, resulting in 110-fold for each epoch (10 times × 11 subjects).

Figure 3 shows that the proposed ID3RANet outperforms other benchmark DL models. After 5 epochs of training, it achieves a peak mean accuracy of 77.16% with the fastest convergence speed. In the rest of the first 20 epochs, its mean accuracy stabilizes at above 74.42%, outperforming the other four models. In contrast, the CNN-LSTM model exhibits slower convergence, reaching an average accuracy of 73.78% after 16 epochs of training. The CompactCNN and CNN-LSTM models achieve the overall higher mean accuracies than both models of the EEGNet and ShallowCovnet, and their highest mean accuracies in the first 20 training epochs are 73.80 and 73.78%, respectively. Even though the CompactCNN model reaches a good performance rapidly in the first 10 epochs and stabilizes at approximately 72.20%, it is lower than that of the proposed model. The results indicate that the proposed method can better capture the class-discriminative EEG features for drowsiness detection from single-channel EEG signals.

In this section, we show the accuracies and F1 score comparison results of different methods tested on the unbalanced dataset, which is closer to the real-life scenarios. As shown in Tables 2, 3, comparing to all baseline models (including conventional methods and six deep learning methods), our proposed method achieves the better performance, due to its enhanced features learning ability. Moreover, we can also draw the following conclusions. First, it can be found that conventional methods (KNN, RF, GNB, and SVM) generally achieve relatively inferior classification performance compared to deep learning-based approaches except EEGNet. This demonstrates that the end-to-end deep learning methods can learn more task-relevant discriminative features for classification. Second, our method improves the average classification accuracy of approximately 4.85% on average and 7.8% on maximum in comparison with the deep learning methods such as CompactCNN and TSANet. This demonstrates that our method can achieve effective mining of important information implicit within single-channel EEG signals by effectively exploiting attention and soft thresholding in the residual shrinkage network. Furthermore, our method achieves an average F1-score almost 1.38% higher than the best TSANet method, all of which demonstrates its effectiveness and potential practical value. Finally, multiple optimal or suboptimal classification results (optimal results bolded in black) are achieved in individual test for each subject, which also proves its strong generalization ability for classifying cross-subject drowsy states from single-channel EEG signals.

To verify the effectiveness of each module of our model, we conduct ablation experiments tested on the unbalanced dataset. Note that our proposed is made up of base feature extractor (BaseFE), residual shrinkage building unit (RSBU) with soft thresholding (ST), GAP, and a fully connected layer (FC) with weight freezing (WF). Specifically, we derive six model variants as follows.

From the results of ablation study shown in Figures 4A,B, we can probably summarize as follows. First, the model variable w/o RSBU, which completely removes the residual shrinkage building unit module, yields almost the worst results. This suggests that the channel-wise residual shrinkage building unit contributes significantly to enhance the features learning ability and model classification performance. Furthermore, in comparison with the model variable w/o ST and w/o RSBU, we know that the residual SE block contributes more to boosting feature representation. Second, the model variable w/o GAP achieves obviously inferior results, indicating that the GAP can effectively reduce overfitting and improve generalization capability of the proposed model. Finally, the proposed ID3RSNet achieves superior classification performance with each module integrated, which shows the necessity and effectiveness of each module.

For sensitivity tests, we randomly selected 50 samples of each class from each subject in the unbalanced dataset and thus have in total 11 (subjects) × 2 (classes) × 50(samples) = 1,100 samples for evaluation (Cui et al., 2023). Inspired by the sensitivity test proposed by Cui et al. (2023), we adopted the non-linear perturbation (sine wave combined with noise) and adjusted the perturbation strength dynamically with a ranged perturbation scale of the input sample. It is assumed that the perturbations will not cause the sample to deviate significantly from its original distributed. The sensitivity test is performed on the original contribution heatmap to reflect the best correlation obtained between the perturbed batches and the model output. Here, we set the scaling factor n of perturbation to 1–5 and calculate the Pearson correlation coefficient (PCC) between the original heatmap and perturbed heatmap as a quality metric of the contribution map. From Figure 5A, it can be observed that our method achieves higher average correlation coefficients in the sensitivity tests compared to the CompactCNN baseline (Cui et al., 2022b). It indicates that the contribution maps generated by our model have higher stability and consistency under different input perturbations.

According to the deletion test proposed in the reference (Petsiuk et al., 2018), we also use the metric of deletion test in this study. In this test, we ranked the sampling points of the input sample according to their descending order based on their values in the contribution map (Cui et al., 2023). By setting the sample threshold for deleting values in the heatmap, we calculated the predicted probabilities when the corresponding points were removed from the sample by setting their values to zeros. The indicative of a high-quality contribution map is a sharp drop of the predicted probabilities for the corresponding class. From Figure 5B, it can be seen that our method achieves lower average values of predicted probabilities with the different sample thresholds compared to the CompactCNN baseline. It indicates that the interpretation of our method is more effective than the CompactCNN due to the removal of important features.

In this section, we explore what EEG patterns our proposed ID3RSNet has learned using the proposed ECAM interpretation method, which is described in Section 2.3. From the generated heatmap of EEG amplitude fluctuation and the bar graph of relative power, we can explain the most discriminative features learned as evidence of the model classification. To verify the reliable interpretation of the model decision, we compare our method with the leading interpretable deep learning method CompactCNN (Cui et al., 2022b). Figures 6, 7 show the visualization of some representative EEG samples labeled as drowsy and alert, respectively. The label and prediction probability of each O_Z channel signal sample are titled.

Extensive experiments have found that most drowsy samples typically contain a high ratio of theta and delta waves, e.g., as shown in Figure 6A, or alpha waves, e.g., as shown in Figure 6B. From the visualization of amplitude fluctuation and relative power displayed in Figure 6A, we have discovered that both methods have captured several episodes, which contain rhythmic bursts of slow waves in the theta-delta band, which is identified as strong evidence for drowsy classification. Actually, these bursts located in the theta-delta band are closely associated with drowsiness (Britton et al., 2016). From the second sample displayed in Figure 6B, we found that the EEG signal in the alpha band has a stronger amplitude and higher relative power, characterized by the narrow frequency peaks. Compared to CompactCNN, our method has captured more regions of spindle-like structures in the alpha band from the central part of the input signal, recognized as strong evidence of drowsiness. It is well known that the captured alpha spindles in EEG signals have been demonstrated to be a strong indicator for recognizing fatigue driving (Simon et al., 2011). Compared with CompactCNN in Figure 6, we can discover that our method have learned more discriminative effective features such as the slow theta-delta waves and alpha waves, to achieve higher classification accuracy.

It can be seen that most alert samples typically contain a high ratio of beta waves, e.g., as shown in Figure 7A, or a high ratio of delta waves, e.g., as shown in Figure 7B. From Figure 7A, we can discover that both methods have captured some short EEG episodes containing lots of high-frequency beta waves as strong evidence for alertness. Notably, these identified beta waves were typically linked to active, busy, attention, or even electromyography (EMG) activities, which is known as the signals during wakeful state (Baumeister et al., 2008; Goncharova et al., 2003). Compared to CompactCNN, our method has not identified the pattern of one high amplitude peak wave that may be caused by eye blinking or eye movement, as the evidence of alertness. This suggests that our method is not only more resistant to the interference caused by the artifacts in the EEG signals, but it also achieves the superior classification result.

From Figure 7B, we can discover that both methods have captured these large-voltage and low-frequency waves (delta band) as strong evidence for alertness. With these discriminative invariant features found, both two methods have achieved high likelihoods. Due to the fact that delta waves are dominant during the deep sleep phase, they are more likely caused by sensor drifts or subject movements during wakeful state. In fact, these typical patterns in EEG signals including EMG and movements are the strongest indicators for wakefulness (Britton et al., 2016). The visualization results prove that our method achieves higher classification accuracy with neurophysiologically reliable patterns found in single-channel EEG signals.