The current work was carried out with the approval by the Ethics Committee of Dongyang People’s Hospital and in accordance with the Declaration of Helsinki. This study included 23 patients and 10 healthy subjects providing informed consent, Table 1 records demographic information for all patients. The subjects were divided into four groups as follows.

Right-sided brain injury group (PR group): patients with the regions of brain injury located on the right side of the brain.

Left-sided brain injury group (PL group): patients with the regions of brain injury located on the left side of the brain.

Bilateral brain injury group (PB group): patients with the regions of brain injury in both the left and right side of the brain.

Healthy control group (HC group): Ten healthy subjects, all right-handed (age: 24–27 years; 3 females, 7 males). None of these healthy subjects had neurological or musculoskeletal disorders and were not taking prohibited drugs.

EEG data were collected continuously for 3 min with the subjects in a relaxed closed-eye (1.5 min) and open-eye state (1.5 min). A 64-channel wireless EEG system (NeuSen.W64, Neuracle, China) was used to collect EEG data with a sampling frequency of 1,000 Hz. Referring to the international 10–20 system, as shown in Figure 1A, we selected 39 channels from the 64 EEG channels for measurement (the reference electrode is located in the middle of Cp1 and Cp2, the ground electrode is located in the middle of AF3 and AF4, and GREENTECH’s GT5 medical conductive gal is used). We eliminated the four electrodes located in the median line: Fz, FCz, Cz, and Pz, according to the arrangement and position of the 39 channels on the scalp, and grouped the remaining channels into seven scalp regions to obtain a regional analysis (Figure 1B). These regions include: left frontal (LF); right frontal (RF); left central-parietal (LCP); right central-parietal (RCP); occipital (O); left temporal (LT) and right temporal (RT). Features were extracted from each of the channels used for analysis. Each feature was averaged for each region and used to analyze differences between groups.

The raw EEG signal was downsampled to 250 Hz using the EEGLAB toolbox and further performed bandpass filtering from 0 to 32 Hz. In addition, Cleanline (an EEGLAB plug-in) is used to remove sinusoidal artifacts in the scalp channel that are not effectively removed by trap filtering. The combination of AC power line fluctuations, equipment power and fluorescent lamps may produce sinusoidal artifacts. Other artifacts, such as ECG, EMG and EOG, are removed by technical independent component analysis (ICA) using the Infomax algorithm in EEGLAB (the probability of an artifact is first given by the program, and then further eliminated by an experienced expert). The artifact-free signal is then denoised using the wavelet transform. This is an acceptable combination of ICA and wavelet denoising for removing noise from EEG signals (Mahajan and Morshed, 2014). For the activity segment (1.5 min) captured by each subject, this paper uses 30s as the window width and 2 s as the interval, and utilizes the sliding window method to segment the activity segment into 31 subsamples, which ultimately constitute the sample set for that subject. The EEG data were processed and analyzed in the MATLAB (version 8.2.0.701.R2017a).

From the introduction of subsection A and B, 14 sets of WPT features and 2 sets of FC features can be obtained. The resulting features are fused to obtain a total of 28 sets of multi-modal (WPT + FC) features:

14(# WPT Features) × 2(# FC features) = 28 features.

For the closed-eye data of a certain frequency band, 1 set of ER matrices and 2 sets of FC feature matrices can be obtained, and the ER matrices and the two FC feature matrices are subjected to matrix multiplication operations to obtain as shown in Equations (16, 17):

Similarly, multiplying the wavelet packet energy matrix of δ, θ, α₁, α₂, β₁, and β₂ frequency band with PLV and PDC matrix respectively, we can get θPLV , θPDC , α 1 P L V , α 1 P D C , α 2 P L V , α 2 P D C , β 1 P L V , β 1 P D C , β 2 P L V , β 2 P D C . The resulting matrices are all 39 × 39 two-dimensional square matrices.

The WEE matrix and the WSE matrix for each frequency band are matrix multiplied with the FC feature matrix as Equations (18, 19), respectively, we get:

Similarly, the WEE and the WSE are multiplied with the PDC to obtain the EEPDC , SEPDC . The resulting matrix is a 39 × 39 two-dimensional square matrix.

Resnet50 deep network can reprogram the network layers according to the residual learning function and can overcome the generalization error, overfitting, vanishing and explosion problems (ElGhany et al., 2021). Therefore, in this paper instead, the resnet50 deep network is used to construct the classifier. The input of the classifier is the features extracted in subsections A, B, C; the output is 0, 1, 2, and 3, where 0 means the subject belongs to the PR group, 1 means the subject belongs to the PL group, 2 means the subject belongs to the PB group, and 3 means the subject belongs to the HC group.

In the classifier, each convolutional layer and fully connected layer is followed by a linear unit with leakage correction (Leaky ReLU) as the activation function. The loss function of the model is chosen as the cross-entropy loss function as shown in Equation (20):

where y i refers to the real disease condition of subject i (the values of y i 0, 1, 2, and 3 refer to Group PR, Group PL, Group PB, and Group HC, respectively), and p i refers to the probability of subject i being predicted as category i by the neural network.

In this paper, we evaluate the classifier using 10-fold cross-validation. Since there are 31 samples per subject, there are a total of 1,023 samples. Based on these samples, the classification network was trained-evaluated using 10-fold cross validation using 102 samples for testing and 921 samples for training.

The model is trained using the Pytorch (version 1.11.0) framework based on CPU, and the optimizer is chosen as the Adam optimizer with an initial learning rate of 0.00001, and CosineAnnealing is used to adjust the learning rate. A total of 600 epochs are trained by reading all samples at a time.

A four-class classifier was used to classify the PR group, the PL group, the PB group, and the HC group, and a confusion matrix was determined and accuracy was calculated as Equation (21):

Where TP, TN, FP, FN represent the true positive, true negative, false positive, and false negative, respectively (Powers, 2020).

The statistical analysis software we chose was IBM SPSS statistics 23 (Datanine Software, China). One-way analysis of variance (ANOVA) tests was used to assess significant changes in the time-frequency underlying characteristics of the four groups. p < 0.05 was considered a statistically significant level. Bonferroni correction was applied to avoid spurious rejections when the variance was chi-squared (Cabin and Mitchell, 2000); Tamhane’s T2 method was used for comparison between groups when the variance was not chi-squared.

Classification accuracy was used to assess the ability of functional connectivity features and multi-modal (WPT + FC) features to distinguish regions of brain injury in stroke patients. Additionally, Figure 2 shows a block diagram with the different steps followed in this study.

A time-frequency analysis of the whole brain and each brain region was first performed. Figure 3 shows the average energy ratio of δ, θ, α₁, α₂, β₁, and β₂ frequency bands for 39 EEG channels in the PR group, PL group, PB group, and HC group. The energy ratios of all four groups tends to decrease with increasing frequency under both open-eye and closed-eye. The energy ratio values of δ band fluctuated between [90,95], while those of β₂ band fluctuated between [0.25,2]. It indicates that the subjects concentrated most of their energy in the lower frequency band in the quiet state. When comparing between groups, we found that the difference between groups for each frequency band was smaller in the open-eye state than in the closed-eye state, especially between stroke patients (p > 0.05). Meanwhile, we can conclude from Figure 3 that in the closed-eye state, there are significant differences (p < 0.01) between the four groups of PR, PL, PB and HC in the α₁, α₂, β₁ and β₂ frequency bands; while in the δ and θ frequency bands, the differences between the groups are smaller. In the δ and θ frequency bands, the energy ratio of the HC group was smaller than that of the other three groups in both the open-eye and closed-eye states, while in the α₁, α₂, β₁, β₂ frequency bands, the energy ratio of the HC group was larger than that of the other three groups. Reflecting the typical slowness of brain recordings in stroke patients due to high energy in the lower frequency bands and low energy in the higher frequency bands due to brain injury. In addition, the PR group had the lowest energy ratio values in the α₁, α₂, β₁, β₂ frequency bands in the closed-eye state. Figure 4 shows the mean energy occupancy of the seven brain regions in the δ, θ, α₁, α₂, β₁, and β₂ frequency bands in the PR group, the PL group, the PB group, and the HC group with closed-eye. In the δ band, there were no significant group differences between the four groups (p > 0.05); in the θ band, there were no significant group differences between the PB and PL groups (p > 0.05). This is the same conclusion as that obtained in Figure 3. In the θ frequency band, stroke patients had a much larger average energy ratio across brain regions than healthy controls. In the higher frequency bands, the mean energy ratio of stroke patients was much smaller than that of HC group: the mean energy ratio of the right frontal, occipital, and right temporal in the α₁, α₂, β₁, and β₂ bands was much smaller. In the α₁, α₂, β₁, and β₂ frequency bands, the PR group had the smallest average energy ratio in each brain region. Significant differences between all four groups in RF, RCP, RT, and O brain regions in the α₁ band (p < 0.01); there were significant differences between the four groups in RT and O brain regions in the α₂ band (p < 0.01); Significant differences (p < 0.01) were found between all four groups in seven brain regions in the β₁ and β₂ frequency bands.

The whole-brain mean WEE and whole-brain mean WSE for the 39 EEG channels in the PR, PL, PB, HC groups in the open-eye and closed-eye states are given in Figure 5. From the figure we can obtain that WEE and WSE are significantly different between stroke patients and healthy controls in the two resting states, but are less able to distinguish between regions of brain injury in stroke patients. We combine the wavelet packet energy analysis for the six frequency bands mentioned above, and considering that the apparent changes occurred in the α₁, α₂, β₁, and β₂ frequency bands in the closed-eye state, we calculated the WEE values for 39 EEG channels in the α₁, α₂, β₁, and β₂ frequency bands for the four groups and applied it to the whole brain analysis, as shown in Figure 6. As we know, the wavelet energy entropy depends mainly on information about the complexity of the nonlinear dynamical process of the EEG signal and the energy distribution of each subspace signal. It was found that the WEE values of the healthy control group were greater than the other three groups in all four frequency bands, while the WEE value of the PR group was the smallest, indicating that the EEG signals acquired in the HC group were more complex than the other three groups in the closed-eye state, and the EEG signals acquired in the PR group were more single and ordered than the other three groups (Figure 6). This indicates that the irregularity of the EEG signal is significantly reduced in stroke patients. We found that WEE values in all four frequency bands were significantly different between stroke patients (PR, PL and PB groups) and HC group (p < 0.001). Notably, in the α₂ band, the WEE of the FC3 channel was significantly different between the four groups (p < 0.01); in the β₁ band, the WEE of the F6 and FC4 channels was significantly different between the four groups (p < 0.01).

To show the differences between groups more clearly, we further calculated the wavelet singular entropy values for the four frequency bands in the PR, PL, PB, and HC groups in the closed-eye state, as shown in Figure 7. Similarly, the wavelet singular entropy values are larger for the HC group and smallest for the PR group in the four frequency bands. Statistical analysis allowed us to conclude that WSE values in all four frequency bands in the closed-eye state were significantly different between stroke patients (PR, PL, PB groups) and HC group (p < 0.001), and between the PR and PL groups (p < 0.01); while for the PB and PR groups, and between the PB and PL groups, there were no significant differences (p > 0.05). Meanwhile, we found that in the α₁ band, the WSE of the C6 channel had significant group differences among the four groups (p < 0.01); in the α₂ band, the WSE of the F1, FC3, CP1 channels had significant group differences among the four groups (p < 0.01); in the β₁ band, the WSE of the FC3 and CP5 channels had significant differences among the four groups (p < 0.01). However, it is not clear which is more effective in classifying regions of brain injury, wavelet energy entropy or wavelet singular entropy based on frequency features. This will be followed by a categorical analysis to compare the two types of wavelet entropy.

Functional connectivity is a measure of the relationship between channels, formally a square matrix with each element of the matrix taking values for the rows and columns corresponding to the metrics between channels. To further explore the phase coupling information of the EEG signal, we investigated the PLV values between the group average channels of the four groups in both closed-eye and open-eye state, which are presented in Figure 8. The closer the PLV value is to 1, the more synchronous the two channels are, i.e., there is a connection between the two channels. As can be seen in Figure 8, there are differences between all four groups in both the closed-eye and open-eye states. The difference between the four groups is more significant in the closed-eye state. And the differences between the PR group and the other three groups were highly significant regardless of the open-eye or closed-eye state. We further found that the synchronization between channels was stronger in the closed-eye state than in the open-eye state in the four groups. For the PR group, the synchronization between channels in the right side of the brain was stronger in the closed-eye state, whereas the synchronization between channels in the whole brain was stronger in the open-eye state. For the PL group, synchrony was stronger between channels in the left and right temporal regions in both the closed-eye and closed-eye states. For the PB group, only the synchronization between the CP5 channel and the other channels was stronger in the closed-eye state; in the open-eye state, the synchrony was stronger between channels in both frontal and central parietal regions. For the HC group, the synchronization between channels in the central parietal region and the right temporal region was stronger in the closed-eye and open-eye states. Therefore, calculating PLV values may be a sensitive analysis method to distinguish regions of brain injury in stroke patients.

PLV shows whether there is a connection between the two channels, but which affects which cannot be obtained from the simultaneity analysis alone. So, we analyze the relationship between the two channels in terms of causality. Figure 9 shows the PDC values between the group average channels of the four groups in the closed-eye and open-eye state. A value of PDC close to 0 indicates that there is no connection between the two electrodes, while a value greater than 0.1 is considered a connection between the two channels. All channel combinations with PDC values greater than 0.156 are plotted in Figure 9. From the figure, it can be seen that in the closed-eye condition, there is a difference between the PB group and the other three groups, and there is also a difference and more significant between the PR group and the other three groups. However, there was no significant difference between the PL and HC groups. In the open-eye condition, there was no significant difference between the four groups. We further found that the PDC values between the AF4 channel and other channels were larger in both the PL and HC groups in the open-eye and closed-eye states. The PDC values between the C5 channel and other channels were larger in the PR group in the closed-eye state; and the PDC values between the AF3 channel and other channels were larger in the open-eye state. In the PB group, the PDC values between the CP6 channel and the other channels were larger in the closed-eye state; when the eyes were open, the PDC values between the CP5 and the other channels were larger. In other words, the calculation of PDC values may only be used to distinguish stroke patients with right-sided brain injury from healthy individuals, and may not be able to distinguish between other regions of brain injury.

From the two parts A, B, it can be obtained that the features extracted from the EEG signals collected in the closed-eye state in all four groups of subjects have more significant differences than those in the open-eye state. In this paper, we propose to fuse the WPT features with FC features, and the form of the fused features is exactly the same as the previous FC features, represented as multi-modal (WPT + FC) features. The form of the fused multi-modal features is identical to the previous FC features, all of which are a square array measuring 39 × 39. The difference is that for each row of the square array of multi-modal features, the value of each element is the product of the value of the element at the same position of the FC feature and the value of the WPT feature of the electrode corresponding to that row, i.e., the value of the element in row i and column j of the square array of multi-modal features is the product of the value of the FC between electrodes i and j and the value of the WPT feature of the i-th electrode. For each row of the multi-modal features matrix, if the value of the element in that row is larger than the other rows, it means that the value of the WPT feature of the channel corresponding to that row is larger; if the value of some elements in that row is larger than the other elements in that row, it means that the channel corresponding to the column in which those elements are located has a larger FC index than the other channels. Thus, information on both FC features and WPT features can be obtained from the square matrix of multi-modal features.

From subsection A, we obtained that in the closed-eye condition, there were significant differences between the PR, PL, PB and HC groups in the α₁, α₂, β₁, and β₂ frequency bands (p < 0.01); while in the δ and θ frequency bands, the differences between groups were smaller. From subsection B, we can get that the PLV index is more effective for distinguishing between PR, PL, PB and HC groups than the PDC index. Thus, in Figure 10, we show the circle plots of the four multi-modal features obtained by multiplying the wavelet energy ratio of the α₁, α₂, β₁, and β₂ frequency bands with PLV for subjects in the PR, PL, PB, and HC groups in the closed-eye state. For each feature, we set the feature values between channels displayed in the circle plot according to the values in the matrix. As can be seen from the figure, the value of the multi-modal features in the HC group are maximum in all four features, which is the same conclusion obtained in subsection A. The energy ratio in the higher frequency bands is significantly higher in healthy subjects than in stroke patients, which also shows that the features obtained after we performed feature fusion also retain information on the energy ratio in the frequency bands. In Figures 11, 12, we show the circle plots of the four multi-modal features obtained by multiplying the WEE and WSE of the α₁, α₂, β₁, and β₂ frequency bands with PLV for subjects in the four groups in the closed-eye state. As can be seen from the figure, stroke patients differ significantly from healthy subjects in the values of multi-modal features; the effect of differentiating regions of brain injury in patients we will further compare by categorical analysis. Meanwhile, from the diagram we can see more clearly which two poles have more significant related to each other.

To investigate whether multi-modal features can be further used to differentiate regions of stroke injury, we designed two controlled trials with subjects in a closed-eye state. Based on 14 WPT features and 2 FC features, we were able to obtain 28 different sets of multimodal features. The results of the first set of controlled experiments are shown in Table 2, where the input was either these 28 sets of multi-modal features, or the sum of these 28 features. Table 2 shows which part of the data these 28 features were obtained from and the accuracy of distinguishing between the PR, PL, PB, HC groups. A comparison of the accuracy rates in Table 2 shows that the accuracy rates obtained for the multi-modal features in groups 13, 15, 17, 19, 21, 23, 25, and 27 are much higher than those obtained for the other groups, and even higher than the accuracy rates obtained for the multi-modal features in all groups. We refer to these eight groups of features collectively as multi-modal frequency features. This suggests that for the available data, the use of multi-modal frequency features in subjects’ resting-state data during closed-eye can be effective in detecting regions of stroke injury and distinguishing them from normal subjects.

To further investigate the difference in accuracy between feature fusion and individual features, we further designed a second set of control trials. In this group of control trials, the inputs are the individual frequencies of the multi-mode frequency features with high accuracy in the first set of control experiments, respectively. The results of the second set of control experiments are shown in Table 3. Compared to the classification results in Table 2, we obtain that the accuracy of the classification using a fusion of two features is higher than that of the classification using a single feature. This demonstrates that the optimal combination of features can improve the classification accuracy.