Affective computing is a science involving multiple disciplines, such as psychology, biology, and philosophy, among which research on affective models and emotion recognition are two important branches in the field of affective computing. Generally, affective models can be mainly categorized as discrete models and dimensional models. Discrete emotion theorists think that people's emotions consist of several basic emotions, such as Ekman's (1992) six-basic emotions model and Izard's (2007) 10-basic emotions model. However, dimensional emotion theorists believe that human's emotions are continuous and have some special characteristics, such as the valence-arousal model (Russell, 1980) and the Plutchik (2001)'s Wheel of Emotions.

With the development of information technology and artificial intelligence technology, emotion recognition plays an increasingly important role in the field of human–computer interaction, because machines can become more intelligent through emotional interaction with humans (Picard, 2003). For this reason, emotion recognition has received widespread attention from researchers in various fields, such as emotion recognition based on facial expressions, emotion recognition based on speech, and emotion recognition based on EEG signals (Koolagudi and Rao, 2012; Ko, 2018; Abramson et al., 2020; Houssein et al., 2022; Yi et al., 2024). However, among the many directions of emotion recognition, EEG signals can establish a closer and more realistic mapping relationship with emotions due to their “unconcealability,” that is to say, people can easily disguise their true emotions by adjusting their expressions and voices, but it is difficult to change the corresponding EEG signals at the same time.

Feature extraction is one of the core modules in the EEG emotion recognition process for highly discriminative features that can help improve the performance of emotion recognition models. In recent years, various features have been developed to solve the EEG-based emotion recognition task, where these features can been roughly divided into two classes, i.e., the single-channel features and the multi-channel features. The calculation of single-channel features does not depend on other channels of EEG, therefore, the features of each channel can be considered independent of each other. Duan et al. (2013) first developed differential entropy (DE) to decode the EEG signals and classify different emotional states, then Zheng and Lu (2015) verified that the DE features have higher discrimination and robustness compared with other features by experiments in two public emotional datasets, i.e., the SEED and the DEAP (Koelstra et al., 2011). Moreover, the sample entropy (SE) and the approximate entropy (ApEn) are also two common entropy-based features to measure the uncertainty of emotional EEG signals (Zeng et al., 2019; Wang et al., 2022). In addition, the wavelet energy and the power spectral density (PSD) are also two common features to measure the frequency domain information for EEG signals (Zheng and Lu, 2015; Mohammadi et al., 2017; Wang et al., 2022). Compared with single-channel features, multi-channel features, or functional connection features, are more concerned with measuring the interactive information between channels. The differential asymmetry (DASM) and the rational asymmetry (RASM) are widely employed to measure the difference of EEG signals between the left and right hemispheres (Zheng and Lu, 2015; Zhang et al., 2021). Both Dasdemir et al. (2017) and Nguyen and Artemiadis (2018) used the phase-locking value (PLV) to build a functional brain neural network and perform the emotion recognition task. Khosrowabadi et al. (2010) utilized the mutual information (MI) to establish a dynamic emotional system. In addition, the Pearson correlation coefficient (PCC), the spectral coherence coefficient (COH), the phase lag index (PLI), and the covariance matrix (COV) are also usually adopted to represent the emotional interaction information between different channels of EEG (Jadhav et al., 2017; Keelawat et al., 2021; Wu et al., 2022; Lin et al., 2023).

Furthermore, a large number of techniques have been developed to construct the mapping relationship between EEG features and emotions. Zheng and Lu (2015) introduced the deep belief network (DBN) to decoding the DE features and achieved 86.08% accuracy for positive, negative, and neutral states. Du et al. (2020) proposed an attention-based LSTM with Domain Discriminator (ATDD-LSTM) with DE as input features to solve the subject-dependent and subject-independent emotion recognition tasks in three public emotional datasets. Moon et al. (2020) employed the convolutional neural networks (CNNs) to extract the spatial domain information from PLV and PCC and transform entropy (TE). Song et al. (2018) developed a novel dynamical graph convolutional neural networks (DGCNNs) to model the multichannel EEG features and perform the emotion recognition task and achieved average accuracies of 90.4% and 79.95% for subject dependent and subject independent cross-validation on the SEED, respectively. Liu et al. (2023) combined the attention mechanism and pre-trianed convolutional capsule network to extract the spatial information from the original emotional EEG signals. Zali-Vargahan et al. (2023) introduced CNN to extract the deep time-frequency features and employed several machine classifiers such as decision tree to classify different emotional states, where the average accuracy of 94.58% had been achieved in SEED. Ma et al. (2023) developed the transfer learning methods to reduce the distribution differences of emotional EEG signals between different subjects, thereby enabling more robust cross-subject emotion recognition.

In recent years, Riemannian manifolds (RMs) have received a lot of attention in the field of brain–computer interfaces due to the simplicity, accuracy, robustness, and transfer learning capabilities (Congedo et al., 2017). Barachant et al. (2013) developed the Riemannian-based kernel support vector machine (RK-SVM) to solve the motor image task in a public brain–computer interface (BCI) competition dataset. However, since the conventional deep learning model mainly utilize the non-linear function to map the features located in the Euclidean space, the features located in RM usually cannot be fed into the deep model, for the non-linear function will change the SPD of the features and the mapped features will then locate in Euclidean space. Therefore, Huang and Van Gool (2017) designed a RM network architecture by the combination of bilinear and non-linear learning, and achieved the state-of-the-art accuracies in three datasets. Yair et al. (2019) utilized the parallel transport to achieve the domain adaptation for symmetric positive definite (SPD) matrices in RM, and achieve accuracy of 78% for four-class motor imagery task by the leave-one-session-out cross-validation. In addition, it can also map Riemannian features to Euclidean space by establishing a mapping between Riemannian manifold and Euclidean space, and then use deep learning methods (Wu et al., 2022). Particularly, in the domain of affective BCIs, Riemannian manifolds approaches have been instrumental in feature extraction and classification tasks related to emotion recognition. The utilization of covariance matrices and manifold-based representations allows for a more nuanced understanding of the underlying neural patterns associated with different emotional states (Abdel-Ghaffar and Daoudi, 2020; Wang et al., 2021). However, the limitation of the Riemannian manifold is that its features must be symmetric and positive definite, which greatly limits the application of the Riemannian methods.

In the present study, we adopted four functional connection features, i.e., PLV, PCC, COH, and MI, to perform the emotion recognition task in the SEED database. The main contributions of this study can be summarized as follows: (1) We introduced the the Laplace matrix with a max operation to transform the four functional connection features into SPD; (2) Four functional connection features almost achieved similar performances in recognizing emotions, especially PLV, PCC, and COH, which may indicate the stability of brain functional connection when subjects are evoked emotions; (3) The decision fusion strategy is employed to fuse the four features to achieve higher accuracy and robustness, and a detailed comparison about the combination for different features in emotion recognition has been made.

The layout of this study is organized into following sections: In Section 2, the detailed descriptions about the four functional connection features, the SPDnet, the Laplace matrix, and the overview about the proposed model are presented. The detailed experimental results, analysis, and discussion for the SEED dataset are presented in Section 3. Finally, Section 4 presents the conclusion about this study, as well as the discussion about future works.

In this study, we proposed a novel method which consists of the functional connection features, the Laplace matrix, and the SPDnet to perform the EEG-based emotion recognition, where the Laplace matrix was utilized to transform the functional connection features and the SPDnet was utilized to extract the deep spatial information from the transformed features. The proposed method achieved desirable performance on the SEED dataset for subject dependent cross-validation with the highest average accuracy of 91.05%/5.54% subject to the fusion of PLV, PCC, and COH. In addition, the experimental results showed that PCC has higher discriminability in identifying different emotions, while MI had the lowest discriminability. Although there are differences among the four functional connection features, the recognition performance were almost similar, especially for PLV, PCC, and COH, which may indicate that, when emotions are induced in subjects, the brain functional connections measured by different functions show a certain degree of stability. However, experimental results using different thresholds applied for the four functional connection features can also draw a similar conclusion. Furthermore, the experiment for different thresholds also indicated that for each subject, the optimal thresholds for the four functional connection features always kept relatively stable within a fixed interval.

However, the current study has certain limitations. We only tested the proposed model in the SEED dataset, which results in an inability to fully demonstrate the generalization performance of the model. Therefore, in future, we will test the model in more public datasets. In addition, this study briefly discusses the effectiveness of the proposed method. Of course, there is more information that can be mined, such as building a brain network and combining the proposed method with complex networks. Finally, we only considered the subject dependent cross-validation strategy, since there are differences between subjects that lead to inconsistent distribution of EEG data. Therefore, we will further combine the transfer learning to test the performance of the proposed model with the subject independent cross-validation strategy.

The original contributions presented in the study are included in the article/supplementary material, further inquiries can be directed to the corresponding author.

MW: Conceptualization, Data curation, Formal analysis, Funding acquisition, Investigation, Methodology, Software, Writing – original draft, Writing – review & editing. RO: Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Writing – original draft. CZ: Investigation, Software, Validation, Visualization, Writing – original draft. ZS: Investigation, Software, Validation, Visualization, Writing – original draft. PL: Conceptualization, Funding acquisition, Project administration, Supervision, Writing – review & editing. FL: Conceptualization, Supervision, Writing – review & editing.