This study aims to investigate the impact of different emotional states on sustained attention, with a side focus on flow. The necessary instruments are detailed below.

Immersive VR videos were utilized to induce different emotional states, presented in a HTC Vive Pro HMD. These were selected from the Stanford Immersive Virtual Reality Video Database (Li et al., 2017), which contains 73 VR clips categorized according to emotion valence and arousal (Russell, 1980). In the end, four videos with scores closest to the quadrant extremes of the valence and arousal dimensions were chosen to achieve optimal emotion induction, as shown in Figure 1.

A subjective emotion self-report scale, the Self-Assessment Manikin (SAM; Bradley and Lang, 1994), was used to evaluate participants’ emotion arousal and valence. The AX-CPT (continuous performance test) paradigm was used to assess sustained attention. In AX-CPT, participants are instructed to respond to letter sequences, with “X” as the target stimulus, but only if preceded by the letter “A”. The sequences include four types: AX (target), AY, BX, and BY, where “B” can be any letter other than “A,” and “Y” can be any letter other than “X.” The classic AX-CPT paradigm comprised 70% AX sequences and 10% each of AY, BX, and BY sequences (Braver et al., 2007). Participants’ accuracy and reaction times were recorded during the task as performance criteria. The Flow Short Scale (Engeser and Rheinberg, 2008), a 10-item Likert scale, was used to measure flow experience during the AX-CPT task.

During the AX-CPT task, a Shimmer3 wearable device was used for ECG data collection, and an ANT Neuro system was used for EEG data collection, as shown in Figure 2.

A total of twelve participants were recruited for this study, 6 male and 6 female, aged 21 to 45 years (mean age = 31.2), right-handed and with normal or corrected-to-normal vision. Fields of expertise of the participants include computer science, education, psychology, foreign languages, marketing, and applied chemistry. All participants had no prior experience with similar experiments.

The experiment was conducted one participant at a time without time limit, guided by a trained researcher in a quiet room and under constant screen brightness. Before the experiment, participants were introduced to the study, asked to sign a consent form, prepared for EEG and PPG collection, and instructed to fill out a demographic questionnaire covering gender, age, occupation, handedness, as well as previous VR experience. Then, participants were asked to stay stationary for 5 min, where a baseline SAM score was acquired. The experiment consists of four sessions, each with five steps, as shown in Figure 3. First, participants were randomly assigned to one of four conditions, each representing one of the four emotional dimensions: negative-valence high-arousal, negative-valence low-arousal, positive-valence high-arousal, or positive-valence low-arousal, to receive corresponding emotion induction through a VR video approximately 3 min and 30 s long. Afterwards, participants filled out the SAM scale, completed the AX-CPT attention task, and filled out the Flow Short Scale. Finally, participants take a 5-min rest before the next session.

Before EEG data analysis, preprocessing was performed with EEGLAB (Delorme and Makeig, 2004), an open-source toolbox based on MATLAB, during which a band-pass FIR filter from 3 Hz to 47 Hz was applied.

This study analyzed Heart Rate Variability (HRV), as calculated from PPG, to reflect autonomic responses that result from the sustained attention task. HRV analysis generally consists of two aspects, time-domain features and frequency-domain features.

Time-domain features reflect heart rate variability and autonomic regulation, including:

Frequency-domain features, which primarily reflects parasympathetic activity (Kumar et al., 2023), were analyzed using resampling, interpolation, and Fast Fourier Transform (FFT), to obtain Power Spectral Density (PSD). The final features includes:

Absolute Power VLF (VLF[abs]), LF (LF[abs]), and HF (HF[abs]): Reflect the energy distribution in their respective frequency ranges, related to specific physiological mechanisms.

Due to the small sample size, we choose a non-parametric test, the Wilcoxon signed-rank test, to analyze differences in accuracy and reaction times in the AX-CPT task under different emotional states. Mean SAM scores among 12 participants are calculated for the four virtual reality videos. Firstly, we check to ensure that the emotion induction took effect. Results are presented in Table 1. Wilcoxon signed-rank tests showed significant differences in valence (Z = 3.07, p = 0.002) and arousal scores (Z = 3.01, p = 0.003) between corresponding videos. The virtual reality videos effectively elicited corresponding emotional responses.

Using the Wilcoxon signed-rank test, we analyze accuracy rates and reaction times during the AX-CPT task under the four emotional dimensions, and consider the influence of gender on the results.

As reported in Table 2, accuracy rates for positive emotions (M = 98.33, SD = 1.13, Mdn = 98.25) and negative emotions (M = 98.50, SD = 1.19, Mdn = 98.75) show no significant difference (Z = 0.49, p = 0.620). Accuracy rates for high-arousal emotions (M = 98.29, SD = 1.01, Mdn = 98.50) and low-arousal emotions (M = 98.54, SD = 1.16, Mdn = 98.75) also show no significant difference (Z = 1.08, p = 10.280). Also, reaction times for positive emotions (M = 30098.58, SD = 5758.64, Mdn = 29501.00) and negative emotions (M = 28465.17, SD = 5166.93, Mdn = 28273.00) show no significant difference (Z = 1.73, p = 0.084). Similarly, reaction times for high-arousal (M = 29061.67, SD = 4897.09, Mdn = 29341.00) and low-arousal emotions (M = 29502.08, SD = 5712.97, Mdn = 28618.00) show no significant difference (Z = 0.784, p = 0.433).

Further, as shown in Table 3, analyzing the influence of emotional valence, arousal, and gender on accuracy using the Wilcoxon signed-rank test revealed no significant differences (Z = 0.68, p = 0.500) in accuracy rates between males under high-arousal negative (M = 97.67, SD = 1.63, Mdn = 97.50) and high-arousal positive emotions (M = 98.33, SD = 1.37, Mdn = 98.00). Also, as shown in Table 4, for females, accuracy rates under high-arousal positive emotions (M = 98.00, SD = 1.55, Mdn = 98.00) and low-arousal positive emotions (M = 99.17, SD = 0.98, Mdn = 99.50) show no significant difference (Z = 1.63, p = 0.100). In comparison, for reaction time, males exhibit significantly shorter reaction times (Z = 2.20, p = 0.028, Cohen′sd = 1.16) under high-arousal negative emotions (M = 26449.17, SD = 4046.69, Mdn = 26046.00) compared to high-arousal positive emotions (M = 27511.17, SD = 3594.44, Mdn = 30373.00). For females, reaction times are significantly shorter (Z = 1.99, p = 0.046, Cohen′sd = 0.79) under high-arousal positive emotions (M = 30362.50, SD = 5326.55) compared to low-arousal positive emotions (M = 33004.17, SD = 8145.73).

There results suggest that emotional valence and arousal do not significantly affect task accuracy performance across different genders, but significantly impact reaction time performance. Males display shorter reaction times, i.e., better sustained attention, under high-arousal negative emotions than high-arousal positive emotions, while females display shorter reaction times under high-arousal positive emotions than low-arousal positive emotions. These results correspond with previous studies such as Bradley et al. (2001), that found males to show greater physiological reactivity toward negative emotions, and females to show greater reactivity toward positive emotions.

Flow experience scores are displayed in Table 5. The Wilcoxon signed-rank test is used to assess differences in flow experience across the four sessions of emotional dimensions. Results indicate that emotional valence (positive: M = 52.46, SD = 9.17, Mdn = 50.75 and negative: M = 55.71, SD = 9.41, Mdn = 57.50) and arousal (high: M = 54.58, SD = 8.68, Mdn = 55.25 and low: M = 53.58, SD = 9.34, Mdn = 52.25) do not pose significant impact on flow experience during the AX-CPT (valence: Z = 1.49, p = 0.136 and arousal: Z = 0.45, p = 0.656). When accounting for gender, no significant difference (Z = 0.32, p = 0.750) in flow experience is found for males between high-arousal negative emotions (M = 50.67, SD = 10.29, Mdn = 50.50) and high-arousal positive emotions (M = 51.50, SD = 3.67, Mdn = 50.00). Neither is significant differences (Z = 0.68, p = 0.500) found for females between high-arousal positive emotion (M = 54.67, SD = 12.45, Mdn = 54.50) and low-arousal positive emotion (M = 56.00, SD = 11.35, Mdn = 56.00) conditions.

Differential brain activity due to variations in attention are commonly reflected in the frequency bands (Wang et al., 2011). Here, we extract and compare frequency domain characteristics, specifically Power Spectral Density (PSD), to reflect the changes of sustained attention in relation to emotion. PSD describes the distribution of signal power across frequencies. We calculate the PSD of α, β, and γ bands, as well as the sustained attention formula βα+θ, using the Welch method from the Python-MNE toolkit (Gramfort et al., 2013). In detail, the Welch method divides the signal into n segments that allow overlap, which improves the signal’s variance properties (Solomon, 1991), windows the data, and computes the average PSD of multiple segments. In particular, the Hanning window is chosen for windowing to mitigate spectral distortion caused by rectangular windows (Harris, 1978). Additionally, baseline correction is performed by subtracting PSD values from the baseline phase, i.e. the first 5 min of each session, from the PSD values of each session. Finally, these PSD values are averaged over α (7–13 Hz), β (14–29 Hz), and γ (30–47 Hz) bands, to be used for analysis.

For the sake of analysis, high and low sustained attention emotion conditions are divided based on AX-CPT task performance, where only the groups with significant differences are retained, resulting in 12 samples with high sustained attention and 12 samples with low sustained attention. Meanwhile, normality tests are conducted for EEG power in α, β, γ bands, and βα+θ ratio, for all subjects in each channel. The results suggest that the data does not have normality. Therefore, we use the Mann-Whitney U test to compare power differences between different sustained attention levels. We found the following patterns in EEG data that could represent sustained attention during the task, as shown in Figure 4, where the red circle represents the channels with significant differences between groups.

As shown in Table 6 and Figure 4a, the analysis of α band indicates that power in the C3 channel is significantly lower in high sustained attention state compared to low sustained attention state. This concurs with prior research, that found α wave activity in the C3 to be generally implicated in relaxation (Klimesch, 1999), meaning that decrease in activity can indicate heightened attention.

As shown in Table 7 and Figure 4b, for β band, analysis indicate that the power in the F7 and POZ channels is significantly higher in high sustained attention state compared to low sustained attention state. Correspondingly, β wave activity has been associated with attention and alertness (Rouhinen et al., 2013).

As shown in Table 8 and Figure 4c, for γ band, the analysis show that powers in the AF3, C4, CP2, CZ, F3, F7, FC1, FC2, FC5, P4, and POZ channels are significantly higher in high sustained attention state compared to low sustained attention state. In congruence with prior studies, γ wave activity is closely related to higher cognitive functions and information processing (Masuda, 2009; Rouhinen et al., 2013), which may be more active during high sustained attention.

For βα+θ ratio, the analysis indicates that this ratio is significantly higher in high sustained attention state compared to low sustained attention state, as shown in Table 9 and Figure 4d. This is consistent with research, that suggested the βα+θ ratio reflects changes in attention and alertness (Wang et al., 2011).

EEG results from this study indicate significant differences between high and low sustained attention conditions in the power of α, β, γ bands, as well as βα+θ ratio. In high sustained attention state, EEG power in α band decreases in the C3 channel, β band power increases in the F7 and POZ channels, and both γ band power and βα+θ increase in the parietal and frontal regions. These findings corroborate previous studies that found α wave power decrease, as well as β and γ wave power increase during high sustained attention states. This might be due to the increase in cognitive resources required for high sustained attention tasks (Başar et al., 2001; Klimesch, 1999).

We also examine heart rate variability (HRV) characteristics in participants under high and low sustained attention states, calculated from PPG data. Considering the sample size in general PPG data analysis, we divided each task into two halves, and HRV features are extracted from each half, resulting in 48 samples. After normality testing, HRV indices VLF[%], LF[%], HF[%], LF/HF, LF[n.u.], and HF[n.u.] follow a normal distribution, while MEANRR, MEDIANRR, MEANHR, SDNN, RMSSD, NN50, pNN50, VLF[abs], LF[abs], and HF[abs] do not.

Indices that adhere to normality are analyzed with t-test. Among these, HF[%] shows a significant decrease in high sustained attention state (M = 35.94, SD = 10.03) compared to low sustained attention state (M = 42.24, SD = 10.80), t(46)=2.09, p = 0.042, Cohen′sd = 0.60. Indices that do not adhere to normality are analyzed with the Mann-Whitney U test. Among them, VLF[abs] displays a significant increase in high sustained attention state (Md = 868.66) compared to low sustained attention state (Md = 500.26), U = 176.50, p = 0.021, Cohen′sd = 0.82. These findings align with previous research that found high sustained attention tasks to require more cognitive resources, which lead to changes in autonomic nervous system regulation: reduced parasympathetic activity, associated with decrease in HF[%], and increased sympathetic activity, associated with increase in VLF[abs] (Krygier et al., 2013; Thayer et al., 2012).