A psychological crisis manifests across various dimensions, including cognitive, physical, emotional, behavioral, and interpersonal realms. Cognitively, an individual in the throes of such a crisis may be enveloped by sorrow, leading to alterations in memory and perception. This can result in challenges distinguishing between the nuances of experiences and a blurring of the lines between similar and dissimilar events. Decision-making and problem-solving skills are compromised, although clarity often returns swiftly once the crisis subsides. Physically, a crisis can trigger symptoms such as insomnia, headaches, exhaustion, heart palpitations, chest constriction, overall malaise, loss of appetite, and indigestion, thereby temporarily impairing bodily functions.

Emotionally, a psychological crisis can dramatically alter one’s affective state, marked by intense feelings of tension, anxiety, emptiness, and a sense of loss, often accompanied by secondary emotions such as fear, anger, guilt, distress, and shame. Anxiety is the most prevalent response, with depression being a graver outcome, and anger potentially escalating to aggressive actions. Behaviorally, the external conduct of body may shift as part of its stress response, serving as a coping mechanism to the crisis at hand. Interpersonally, during a psychological crisis, individuals may become increasingly withdrawn into themselves, becoming averse to communication, resistant to assistance or advice, unable to form trusting relationships, and consequently, experiencing isolation.

For university educators specifically, the manifestations of a psychological crisis are particularly pronounced. The emotional fluctuations experienced by teachers under stress are intricately tied to their anticipation and appraisal of stress outcomes. Successfully managing stress can lead to positive emotional experiences; however, the spectrum of emotional responses can range from anxiety and fear to depression and anger. Anxiety is the most typical reaction, with its impact on an individual’s varying environmental coping behaviors. Low levels of anxiety can slow response times and diminish productivity, while moderate levels can heighten alertness and enhance a teacher’s adaptability and stress management. In contrast, excessive or misplaced anxiety can diminish one’s capacity to adapt to environmental shifts, potentially leading to generalized anxiety that hinders effective coping mechanisms in response to environmental changes.

In emotion recognition research, which features of physiological signals are extracted has a significant impact on the classification effect. Different features will contain different information about the corresponding physiological signals.

In this study, we have established criteria for classifying emotional states based on extensive research in psychology and behavioral science as well as previous studies in the field of human–computer interaction. Specifically, our classification of emotional states is based on the following core dimensions: physiological activation level, electrodermal activity, and changes in respiratory patterns. For example, states of stress or anxiety are often accompanied by increased respiratory rate and changes in depth. Based on the characteristics of these physiological signals, we categorize emotional states into the following four categories: (1) Baseline State: In this state, individuals are relaxed and not experiencing significant emotional changes due to external stimuli. (2) Stress State: Individuals experience stress or anxiety, with physiological signals showing a higher level of activation. (3) Amusement State: Individuals exhibit higher positive emotions and physiological activation when engaged in enjoyable or pleasant activities. (4) Meditation State: During meditation or deep relaxation, physiological signals show lower activation levels and regular respiratory patterns.

In order to comprehensively study which features can contain more physiological signal features in different emotional states, as shown in Figure 1, this study uses four methods for physiological signals. Feature extraction includes time and Hilbert-Huang transform (HHT)-based features. Although different types of features can more comprehensively reflect the physiological signal changes in different emotional states, too many features are prone to feature redundancy, optimal feature set before emotion classification and recognition.

Such an algorithm is to decompose the components of different frequencies in the signal, and the separated frequency components are different. These components are called features:

Step 1: Let u₁(t) be the signal composed of the maximum points and u₂(t) be the signal composed of the minimum values. The mean m t of these two signals is given by (as shown in Equation 8):

Where u 1 t and u 2 t represent the signals composed of the maximum and minimum values of the original signal, respectively.

Step 2: Subtract the original signal sequence from Equation 9 to obtain a new sequence h t :

Where x t is the original signal sequence.

Step 3: Judge whether conditions of the IMF. If does not satisfy the conditions of IMF, it is regarded as a new x t , and it satisfies the conditions of the IMF. In the actual calculation, too much repetition of the above process will lose the practical significance of the signal to a certain extent, so it is necessary to formulate a stopping criterion. The emotional changes of individual teachers under stress are also closely related to their prediction and evaluation of stress outcomes. Successfully coping with stressors often brings pleasant and happy emotional experiences to teachers. According to the severity of emotional changes, it can be expressed as anxiety, fear, depression, anger, and so on. Anxiety is the most common emotional response in the stress response. When the anxiety level is low, it affects the individual’s behavior in coping with the environment. The response is often slow and the efficiency of homework is not high. Moderate levels of anxiety can enhance an individual’s alertness and bolster teachers’ capacity to adapt to their environment and manage stressors effectively. Conversely, excessive or misplaced anxiety can diminish the ability to address environmental changes. Moreover, there is a risk of anxiety becoming generalized, potentially impairing teachers’ efficacy in responding to environmental shifts.

The commonly used stopping criterion is as follows (as shown in Equation 10):

In the equation, S D represents the stopping criterion, with a threshold value typically ranging from 0.2 to 0.3. When it is less than this threshold, the iteration process will be stopped.

Step 4: The IMF component is obtained as C 1 = h 1 k t , and the remainder after separation is as follows (as shown in Equation 11):

Where r 1 t is the residuals after the separation of the first IMF components from the original signal.

Step 5: Repeat the above steps to obtain subsequent IMF components C 2 , C 3 , … C n .

Through the above steps, the original signal composed of multiple original signals can be reconstructed by adding each IMF component and the residual. The equation is (as follows as shown in Equation 12):

After the above decomposition process, we can see that the well so that we can extract the internal features of the signal, but the EMD algorithm has a problem that cannot be ignored; that is, it has modal aliasing, which will lead to the IMF obtained after decomposition loses its physical meaning. A new decomposition method for the aliasing problem is the overall empirical mode decomposition algorithm.

However, the instantaneous frequency of any time series is not always meaningful, it must meet certain conditions, which is why the signal should be EMD decomposed to obtain IMF before HHT is performed on the signal. After the signal decomposition process, the equation to reconstruct the original signal is as follows:

Where s t is the reconstructed signal after the Hilbert–Huang transform (HHT), which is the sum of the real parts of the complex exponentials representing each IMF component. a i t represents the amplitude of the ith IMF component. ϕ i t represents the phase of the ith IMF component. ω i represents the instantaneous frequency of the ith IMF component.

In Equation 13, Re means taking the real part. The Hilbert time spectrum H ω t is as follows:

Following Equation 14 and the process described above, we find that the Hilbert-Huang Transform (HHT) is more adaptable to the instantaneous frequency of the signal and can represent varying frequencies. This makes the HHT a more suitable tool for analyzing non-stationary signals. As shown in Equation 15:

Where Z t is the complex form of the signal after applying the Hilbert transform, where X t is the original signal and Y t is the Hilbert transform of the original signal. a t represents the amplitude of the signal Z t . θ t represents the phase of the signal Z t .

The emotion recognition model described in the study employs a multitude of physiological signals, including ECG, EMG, electrodermal activity, and respiratory signals, to create a comprehensive representation of emotional states. By fusing these diverse biological indicators, the model provides a more detailed and accurate depiction of the emotional landscape than analyses based on a single signal type.

In the feature extraction phase, the model utilizes both time-domain and time-frequency domain techniques to capture the intricacies within each physiological signal. The time-domain analysis focuses on statistical properties such as the mean heart rate and respiratory rate, while the time-frequency domain analysis uses wavelet transforms to reveal the spectral content of the signals. This dual approach offers insights into the dynamic changes associated with varying emotional states.

Following feature extraction, the model subjects the features to a feature selection process, where redundant or less informative features are eliminated. This step is essential for preventing overfitting, ensuring that the model maintains high predictive accuracy and the ability to generalize across different datasets.

The decision-making process of the model is primarily driven by a neural network structure, specifically a convolutional neural network (CNN). This structure is adept at capturing non-linear relationships between physiological features and emotional states. The layered architecture of CNN enables it to automatically learn hierarchical feature representations, where complex features are constructed from simpler ones, leading to a sophisticated understanding of the data.

To enhance the interpretability of model, attention mechanisms are integrated into the neural network. These mechanisms allow the model to focus on the most informative features during the classification process. By highlighting the most relevant aspects of the input signals, such as significant ECG peaks or variations in respiratory rates, the model not only improves its predictive capabilities but also gains the ability to explain its predictions.

This level of transparency in the decision-making process is crucial, as it allows for a deeper understanding of the predictions of the model and the physiological indicators that underpin them. It clarifies which specific changes in physiological signals are most indicative of certain emotional states, effectively demystifying the often opaque nature of neural networks.

The emotion recognition model is distinguished by its integration of multiple physiological signals, sophisticated feature extraction and selection processes, and a neural network-driven decision-making process enhanced by attention mechanisms. These elements collectively strengthen the predictive accuracy of the model and provide a level of interpretability that is vital for its practical application in real-world scenarios.

In the experiment of this paper, the public data set WESAD and the electrocardiographic signal ECG of the self-collected data set are used to build a model for the four-category recognition of the human mental state. The following is an introduction to the two data sets.

This study is based on the disclosed data set, Wearable Stress and Affect Detection Dataset (WESAD), which uses the ECG signal to identify the psychological state of the human body. The WESAD dataset is a publicly available dataset specifically designed for research on the application of wearable devices in stress and emotion detection. We chose this dataset because it provides a wealth of physiological signal data (such as electrocardiography ECG) along with corresponding psychological state labels, providing an ideal data foundation for our study on emotion recognition systems. Furthermore, the WESAD dataset includes physiological signals under various psychological states, which aids in training and validating our emotion recognition model across different emotional states. The following is an introduction to the acquisition signals and experimental procedures of the WESAD dataset. The public data set contains the physiological signal data and mental state labels of 15 subjects, 12 men and 3 women. The average age of the subjects is 27.5 years old, and their personalized physiological characteristics such as height, age, gender, and weight record it. Psychological crisis can be manifested in cognitive, physical, emotional, behavioral, interpersonal, and other aspects. In terms of cognition, when an individual is in a psychological crisis, his body and mind are immersed in grief, which will lead to changes in his memory and perception, manifested as difficulty in distinguishing the similarities and differences between things, and ambiguous relationships between things experienced.

Initially, we performed denoising on the raw physiological signal data to eliminate outliers caused by sensor malfunctions or improper user operation. Subsequently, we applied band-pass filtering to the signals to remove high-frequency noise and low-frequency drift, ensuring the quality of the signals. To facilitate analysis, we segmented the continuous physiological signal data into fixed-length time windows. For the WESAD dataset, we chose a 10-s time window, allowing us to capture short-term changes in emotional states. Some signals underwent baseline correction to eliminate long-term trends and baseline drift in the physiological signals. During the preprocessing, we removed data with poor signal quality or unclear labels. We believe that such data could affect the training effectiveness of the model and the accuracy of classification. Additionally, we excluded cases with too few samples in specific psychological states to ensure that each category has sufficient data to support model learning.

Figure 3 shows the ECG signal data waveform of an experimental subject in WESAD under different experimental conditions. It can be seen intuitively that the ECG signal waveform characteristics of the four different psychological states of baseline, amusement, stress, and meditation are quite different. Among which is the waveform of the ECG meditation state. Compared with the baseline, the ECG signal in the entertainment state has a steeper slope. This also proves the rationality of human mental state recognition through ECG signals to a certain extent.

The readme.txt of the WESAD data set records the age, gender, height, weight, and other personal information of the subjects; the quest questionnaire file provides the subjects’ subjective scores on their psychological state under the experimental process at each stage at that time, using PANAS, SSSQ and other personal assessment questionnaires; the pkl file provides all physiological signal data and corresponding labels of the two devices of the experimental subjects. This paper chooses to collect and preprocess ECG signal data based on the pkl file corresponding to each experimental object because the tags and data in the pkl file are stored in an array, which is convenient to establish one-to-one corresponding data and tag pairs.

Through the settings of the above different experimental procedures and conditions, four different psychological states of the experimental subjects were stimulated, which provided the data basis for the construction of the human psychological state recognition model in this paper. The corresponding psychological state labels are 1, 2, 3, and 4, respectively. The data volume and data distribution are shown in Figure 4.

Many previous experiments utilizing the WESAD dataset have focused on recognizing two-category (baseline and nervous) or three-category (baseline, nervous, and entertainment) mental states, often omitting the meditation state. This oversight is primarily due to the similarities in ECG signal waveforms between the meditation, entertainment, and baseline states, which can lead to significant confusion during identification. Consequently, expanding the classification to include four categories, based on the original three, is likely to result in some decrease in accuracy due to the inherent similarities in signal characteristics. It is important to note that signals carry distinct meanings. While the academic community has not yet reached a consensus on the definition of a psychological crisis, the perspectives mentioned above convey essentially the same concept. We interpret a psychological crisis as a state of psychological disequilibrium that arises in response to crisis events.

Beyond merely denoising the data, physiological signals from the experimental subjects must be segmented into time windows before mental state feature extraction can occur. The chest device ECG signals of WESAD dataset, used in this paper, have a sampling frequency of 700 Hz. For ease of processing, these signals are divided into fixed-size time windows, with this study considering 10-s intervals of continuous data as a single sample, equating to 7,000 data points. The self-collected dataset, MSSFT, has a sampling frequency of 125 Hz and is processed using the same time window approach, yielding 1,250 data points per sample.

In addition to the WESAD dataset, our model was also tested on the self-collected MSSFT dataset. The MSSFT dataset contains ECG signals from participants with diverse backgrounds, providing us with an opportunity to assess the model’s generalization capabilities. Similar to the WESAD dataset, we applied the same preprocessing steps to the ECG signals in the MSSFT dataset, including denoising, band-pass filtering, and segmentation into fixed-length time windows. The MSSFT dataset has a sampling frequency of 125 Hz, and we also used a 10-s time window, with each sample containing 1,250 data points. The experimental results, as shown in Table 1, indicate that our model also demonstrated the ability to recognize different psychological states on the MSSFT dataset, although the accuracy was slightly lower than on the WESAD dataset. This may be attributed to differences in data distribution and noise levels between the two datasets. These results not only suggest that our model has some applicability on new datasets but also highlight the need to further improve the generalization capabilities of model.

To comprehensively evaluate the performance of our model, we employed a variety of evaluation metrics, including accuracy, precision, recall, and F1 score. These metrics are calculated by comparing the predicted outcomes of the model with the actual labels. Accuracy represents the proportion of samples that the model predicted correctly out of the total samples; precision represents the proportion of samples predicted as positive that are actually positive; recall represents the proportion of actual positive samples that are correctly predicted as positive by the model; the F1 score is the harmonic mean of precision and recall, used to measure the overall performance of the model. Furthermore, to assess the stability and generalizability of our model, we used 5-fold cross-validation. This method involves dividing the dataset into five equal parts, using four parts for training and the remaining part for testing in each iteration, repeating this process five times, and calculating the average performance metrics.

Four classifications (baseline, nervousness, entertainment, and meditation) are implemented on the WESAD public data set, and the epoch is set to 60. The loss line graph and the accuracy, precision, recall, and F1 score line graph are shown in Figure 5. The accuracy is 87.90%, with an F1 score of 87.71%.

Given that the source publication of WESAD dataset already presents a three-category mental state recognition model—differentiating among baseline, entertainment, and nervous states using ECG signals—this paper refrains from replicating that experiment. Instead, we directly reference the outcomes of the aforementioned model to benchmark against the novel methods introduced in this study. The methodologies outlined in the original text encompass a multi-level dual-channel fusion human mental state recognition model (parallel) and a multi-level dual-channel fusion model (composite), both of which are developed in this chapter. A comparative analysis of results of these methods is depicted in Figure 6, showcasing the initial five model outcomes constructed in this chapter.

According to the above figure, it can be concluded that compared with the original results, the multi-level dual-channel fusion human mental state recognition model proposed in this chapter can better identify the three mental states of baseline, entertainment, and nervousness on the WESAD data set. The effectiveness of the method can be used to further refine the classification of mental states. The following experiments will pay more attention to the study of the four classifications.

To compare the performance differences between different model configurations, we conducted a statistical significance analysis using paired t-tests. The significance level was set at 0.05. By calculating the correlation coefficients between features and performing t-tests, we assessed the changes in physiological signal features across different emotional states. These analyses indicate that the differences in emotional states identified by our model are statistically significant, further validating the effectiveness and reliability of our model.

In the 1,303 samples, the results of the internal and external propensity classification model in the GA-RF are shown in Figure 7.

Figure 8 illustrates the comparative analysis between the original model group, which includes Decision-Tree, SVM, and RF, and the control group that incorporates the genetic algorithm for feature extraction, including GA-Decision-Tree, GA-SVM, and GA-RF. It is observed that the models augmented with the genetic algorithm demonstrate improved performance, suggesting that the feature extraction process of genetic algorithm enhances the representativeness of the data across each feature dimension. Specifically, the ability of genetic algorithm to refine the internal and external tendency labels results in eight parameters that are more indicative of the depressive psychological state. This enhancement is particularly evident when examining the outcomes from the 1,433 samples, where the accuracy of the two-class depression model is significantly elevated in the control group models that leverage the genetic algorithm.

Figure 8 presents the results, demonstrating that the F1 scores of the various model algorithms exceed 0.6, with the optimal GA-RF model achieving an F1 score of 0.81. This indicates that it is indeed possible to predict students’ depressive psychological states with a significant correlation to their online behavioral characteristics. By leveraging these characteristics, we can achieve real-time monitoring of depressive psychological states of students. Furthermore, when comparing the performance of models trained with features extracted by a genetic algorithm versus those trained solely on original data features, it becomes evident that the genetic algorithm-enhanced models exhibit a higher degree of reliance on specific online behaviors, such as WeChat dependency, the regularity of map website usage, and the degree of game dependence.