Students at Purdue University were invited to participate in this study via email announcements. The study was approved by the Purdue University institutional review board (IRB), and participants provided written consent before participation. We collected data from 29 participants in total. Five participants were removed from the analysis due to poor data quality from either an inadequate EEG headset fit or head movement during the experiment, leaving 24 participants (14 male, 10 female; age: M = 24.38, SD = 3.50; age range: 19 − 33) for our statistical analysis. All participants were right-handed with the exception of two (one: left-handed, one: ambidextrous). We examined FAA values recorded during four image sequence events (see Section 3). An a priori power analysis was conducted using G*Power3 (Faul et al., 2007) to test the difference between four image sequence measurements (two levels of valence and two levels of arousal) using a repeated measures Analysis of Variance (ANOVA), a medium effect size 0.25 (partial η ² = 0.06) (Cohen, 1988), and an alpha of 0.05. This analysis showed that 23 participants were necessary to achieve a power of 0.80. Our group sample size is greater than that of Schöne et al. (2016), who displayed images to participants while investigating FAA through EEG, and similar to Kisker et al. (2021), who used EEG to investigate FAA during an immersive VR experience. Participants received a 10 United States dollar Amazon gift card as compensation.

We presented our virtual environment (VE) to participants using the Oculus Quest VR headset. The Oculus Quest VR headset has a visual resolution of 1,440 × 1,600 per eye, a refresh rate of 72 Hz, and user adjustable interpupillary distance (IPD). While slight viewing distortion may exist in HMDs, all participants used the IPD settings on the headset and determined a comfortable viewing setting. No participant stated that the images were unclear.

We used a movie theater environment downloaded from the Unity Asset store and placed image sequences on the screen, mimicking a movie playing at a theater, as seen in Figure 2. Our decision to place participants in a virtual theater environment was based on work by He et al. (2018), who stated that making participants feel like they were in a theater could create a strong immersive experience for them, in turn inducing a strong emotional effect (Ding et al., 2018). Additionally, Kim et al. (2018) showed that such an immersive experience could more significantly impact emotional responses of participants compared to a non-immersive experience.

Although a direct, one-to-one relationship between color and emotion does not exist, the color red is thought to contain the most energy (Lindström et al., 2006). Therefore, it is possible the red seats and carpet of the virtual environment provided an energizing atmosphere for participants, which may have influenced feelings towards image sequences. Because all participants experienced the same virtual theater environment, we assume any effects from the color of the environment are comparable between participants. Participants were positioned a few rows back, in the middle of the theater environment. Images were positioned in the virtual environment with no degree of tilt; the z-axis rotation parameter was set at zero. Our image sequence viewing application was developed using the Unity game engine.

To collect brain activity data from participants, we used the consumer-grade Emotiv Epoc X EEG headset ¹ (Emotiv Systems Inc., San Francisco, CA, United States), which has demonstrated EEG data consistent with conventional EEG recordings (Bobrov et al., 2011; Duvinage et al., 2013) as mentioned in Le et al. (2020), and relatively good research performance (Sawangjai et al., 2019). Additionally, this EEG headset is quick to set up and comfortable for participants, making it potentially more ecologically valid than wet electrode set-ups. The Emotiv Epoc X is a 14 channel headset based on the international 10–20 system which includes electrode sites AF4, AF3, F3, F4, F7, F8, FC5, FC6, O1, O2, P7, P8, T7 and T8. Additional electrodes, M1 and M2, served as the reference and ground electrodes, respectively. Felt pads used to make contact between the electrode and the participant’s scalp were fully soaked in saline solution prior to inserting into each electrode compartment of the headset. The Emotiv Epoc X records data at a 256 Hz sampling rate which ensured that alpha power data was free from aliasing. Emotiv acquisition filters consist of built-in 5th order Sinc filters, with bandwidths set to 0.16–43 Hz; the maximum bandwidth available. In line with best practices in human EEG (Sinha et al., 2016), broadband EEG signals were acquired in situ with alpha bands extracted in data analysis. Line noise at 50 and 60 Hz were filtered using built-in 5th order Sinc notch filters. We used F3 and F4 electrodes to calculate FAA, as this electrode pair is both common in FAA research (Smith et al., 2017; Kuper et al., 2019; David et al., 2021), and unobstructed by the VR headset during recording.

Raw EEG data was recorded in Emotiv’s software, EmotivPro, and exported to EEGLAB (Delorme and Makeig, 2004) in MATLAB (MathWorks) for preprocessing. Data was filtered between 2 Hz (high pass) and 30 Hz (low pass) using eegfiltnew. Next, bad channels were rejected and then interpolated. An average reference was applied to the data, and Independent Component Analysis (ICA) was run on the data with the goal of removing artifacts from the data without removing those portions of data which are affected by artifacts (Delorme and Makeig, 2004). Eye components were removed from the data with 70% probability. Continuous EEG data was segmented into 2 s epochs for each event. Epochs that were ±100 μV were rejected. spectopo from EEGLAB was used on the cleaned EEG data to determine the power spectra in the 8–13 Hz alpha range. To determine FAA, the natural log-transformed alpha power of the left electrode (F3) was subtracted from the natural log-transformed alpha power of the right electrode (F4) using the following formula: (ln [Right] − ln [Left]) (Smith et al., 2017).

As an exploratory analysis, temporal dynamics of FAA bursts were investigated using time series wavelet decomposition analyses adapted from studies by Allen and Cohen (2010). FAA burst analysis codifies fluctuations of FAA activity over shorter time scales than standard FAA measures, and is used to assess micro-scale FAA changes underlying conventional FAA measures. For example, FAA burst analysis has been utilized in understanding resting state neural dynamics in major depressive disorder (MDD). Participants exhibiting lifetime presentation of MDD have shown a high number of total negative FAA bursts as well as increases in temporal precision of positive FAA bursts relative to participants without MDD (Allen and Cohen, 2010). For our FAA burst analysis, EEG signals were mapped to analytic signals by way of the Hilbert transform (Equation 1): F t = 1 π ∫ − ∞ ∞ f x t − x d x . (1) The analytic signal F(t) is a complex-valued function analogous to Fourier representations with negative frequencies discarded. Instantaneous power reflective of the squared magnitude of the total EEG signal at each time sample was calculated from the resulting analytic signal as (Equation 2): P I t = l n | F t | 2 . (2) FAA through time between two given electrodes was then calculated as (Equation 3): A t = P I 1 t − P I 2 t . (3) Density estimation of FAA values was performed to estimate the underlying FAA distribution. Positive FAA bursts corresponding to greater relative right EEG power across all frequency bands were taken as values greater than the 90th percentile of the FAA distribution. Likewise, negative FAA bursts corresponding to greater relative left EEG power were taken as values less than the 10th percentile of the FAA distribution.

After identification of positive and negative FAA bursts, wavelet decomposition was performed to extract instantaneous alpha power for assessment of time-frequency dynamics of FAA bursts. The time-frequency decomposed signal Φ(t, f) was calculated by convolving the 2s epoch FAA time series with a complex Morlet mother wavelet of the form (Equation 4): Ψ t = e j 2 π f t ∗ e − t 2 2 σ 2 , (4) with t being t, f being frequency, and j being the imaginary number − 1 and represents a complex sinusoid windowed by a Gaussian kernel. For this study, sigma was set as 4.5 2 π f in line with previous studies Allen and Cohen (2010). Frequency for the complex Morlet wavelet was increased from 7.9 to 13.04 Hz corresponding to the alpha power band in 50 logarithmic spaced steps. Instantaneous power per frequency was calculated as (Equation 5): P I f = r e a l Φ t , f 2 + i m a g Φ t , f 2 . (5) Average alpha power was calculated as the mean power per frequency in the alpha band and converted to a decibel scale.

In this study, we investigated four image sequences: V2A2: low valence and low arousal; V2A8: low valence and high arousal; V8A2: high valence and low arousal; and V8A8: high valence and high arousal, shown in Figure 3. The order of image sequence presentation for each participant was randomized within our application. Each image sequence included 10 images and was displayed for 4 seconds, following image presentation recommendations by Chen et al. (2006), for a total of 40 s per image sequence. Please see Mousas et al. (2021) for more information concerning the development of the image sequences.

The images used in our image sequences were originally taken from the Open Affective Standardized Image Set (OASIS) (Kurdi et al., 2017), which provides an open-access database of 900 color images comprising a variety of themes. The OASIS dataset additionally contains normative ratings of both valence and arousal dimension of affect, collected from an online study with 822 participants (aged 18–74) in 2015, with an equal gender distribution. In our previous work (Mousas et al., 2021) we developed a system to generate image sequences with target valence and arousal values from the annotated image dataset, and validated our image sequences through a user study.

Participants first signed our consent form upon arriving at the lab. Next, participants were fitted with the EEG headset. The researcher showed participants their brain signal visualization in real-time within the Emotiv software, and asked participants to try frowning, talking, and moving. In this way, participants were able to see their brain signal and understand how motion artifacts could decrease data quality. Participants were fitted with the Oculus Quest VR headset and could adjust the headset for comfort. A participant wearing both devices can be seen in Figure 4. Before starting the application, the researcher explained that participants should think about how they feel while viewing each image sequence.

Once the application was started, participants were invited to look around and become familiar with the VE, so that they might become immersed in the environment before beginning the image viewing task. Once the participant was ready to start the experiment, they faced the screen in the VE and the researcher started the experiment in Unity. After starting the experiment, the researcher left the room to ensure a private viewing experience (Davidson et al., 1990). First, participants viewed a fixation cross (white crosshair on black background) on the screen in the VE for 40 s, which served as an FAA baseline. Participants were instructed to minimize mental wandering during this time and to remain as still as possible. After the baseline period, image sequences were presented in a randomized order. Each image sequence was separated by a 5 s fixation cross. Participants were instructed to look at the fixation cross during these intervals. Participants spent about 35 minutes in the lab to complete the experiment. Please see Figure 5 for a flowchart of the study and Figure 6 for a visualization of the VR application components.

We investigated the following four image sequences: V2A2: low valence, low arousal; V8A2, high valence, low arousal; V2A8, low valence, high arousal; and V8A8, and high valence, high arousal. We used Greenhouse-Geisser corrections when Mauchley’s test of sphericity was violated. Bonferroni corrections were used for post hoc comparisons. Missing data and outliers were replaced with mean values prior to statistical analysis. Our statistical analyses are described below.

First, we used a two-way repeated measures Analysis of Variance (ANOVA) to explore differences in FAA between two levels of Arousal and Valence image sequences. The Shapiro-Wilk test indicated these values were normally distributed. These results can be found in Section 3.1.

As FAA scores alone do not indicate the contribution of left and right prefrontal cortices to FAA (Smith et al., 2017), we conducted an additional analysis to determine if greater FAA scores were indicative of increased left cortical activity, or decreased right cortical activity. We investigated alpha power from each side separately in order to learn about each hemisphere’s contribution to the FAA score. FAA is calculated by subtracting the natural log of the alpha power from the left electrode from the natural log of the alpha power from the right electrode. Therefore, examining alpha power of each electrode separately is informative in understanding which side has higher cortical activity. Because alpha power is inversely related to cortical activity, a higher alpha power from the right electrode would indicate lower cortical activity from the right electrode. According to Harmon-Jones and Gable (2018), greater left than right activity is associated with happiness and greater approach motivation; the tendency of an organism to expend energy to go towards a stimuli, while greater right than left activity is associated with fear and withdrawal motivation; the tendency to expend energy to move away from a stimuli.

To understand the contribution of left and right alpha power to FAA, we conducted three-way repeated measures ANOVAs with Side (left vs. right), Valence (high vs. low) and Arousal (high vs. low) as factors. Alpha power data was not normally distributed. Therefore, we transformed data following recommendations by Templeton (2011) prior to conducting our parametric analyses on the normally transformed data. These results can be found in Section 3.2. Additionally, we examined differences between FAA recorded during baseline and FAA recorded during each image sequence through one-tailed paired samples t-tests. These results can be found in Section 3.3.

In our FAA burst analyses, we investigated total FAA bursts, total positive FAA bursts, total negative FAA bursts, positive FAA burst power, and negative FAA burst power; all of which were normally distributed according to the Kolmogorov-Smirnov test of normality. A two-way analysis of variance (ANOVA) was was performed with Arousal and Valence of image sequences as factors. These results can be found in Section 3.4.

While our results do not demonstrate a significant interaction effect between valence and arousal ( F [ 1,23 ] = 2.276 , p = . 145 , η p 2 = . 090 ) , we determined a significant main effect of valence ( F [ 1,23 ] = 4.631 , p = . 042 , η p 2 = . 168 ) on FAA. Post hoc comparisons showed that FAA was significantly greater for high valence (M = 0.28, SD = 0.27) than for low valence (M = 0.22, SD = 0.33) image sequences. We determined a significant main effect of arousal on FAA ( F [ 1,23 ] = 7.143 , p = . 014 , η p 2 = . 237 ) . Post hoc comparisons indicated that FAA was significantly higher for low arousal (M = 0.30, SD = 0.37) than for high arousal (M = 0.19, SD = 0.29) image sequences.

We investigated the contribution of Side (left vs. right), Valence (high vs. low), and Arousal (high vs. low) on Alpha Power values with a three-way repeated measures ANOVA. From this analysis, we determined a main effect of Side on ( Λ = . 773 , F [ 1,23 ] = 6.751 , p = . 016 , η p 2 = . 227 ) on Alpha Power. Pairwise comparisons showed that Alpha Power on the Left (M = 1.29, SD = 1.14) was significantly lower than Alpha Power on the Right (M = 1.78, SD = 1.69). Because alpha power is inversely related to cortical activity, our results suggest greater left prefrontal cortical activity than right prefrontal cortical activity during the experiment. Although it did not reach statistical significance, our results suggest a trending main effect of Valence on Alpha Power ( Λ = . 847 , F [ 1,23 ] = 4.170 , p = . 053 , η p 2 = . 153 ) , with Alpha Power greater during low valence (M = 1.63, SD = 1.61) than high valence (M = 1.46, SD = 1.22) image sequences, suggesting greater cortical activity during high valence image sequences.

To understand if FAA values during image sequence viewing were different from baseline FAA, we conducted one-tailed paired samples t-tests with FAA measured during V2A2, V2A8, V8A2, and V8A8 image sequence viewing. We determined a significant increase in FAA from baseline to V8A2 (t [23] = −3.166, p = 0.002), which showed that FAA during V8A2 (M = 0.36, SD = 0.36) was significantly higher than FAA during baseline (M = 0.15, SD = 0.37). Neither V2A8 (M = 0.19, SD = 0.29) nor V8V8 (M = 0.19, SD = 0.29) differed significantly from baseline; (t [23] = −0.670, p = 0.255) and (t [23] = −0.671, p = 0.254), respectively. Lastly, FAA during V2A2 (M = 0.26, SD = 0.37) did not significantly differ from than FAA during baseline (M = 0.15, SD = 0.37), (t [23] = −1.636, p = 0.058), as seen in Figure 7.

In examining the total number of positive and negative FAA bursts, no significant differences in burst count between arousal and valence levels were found, as seen in Figure 8. Furthermore, there were no significant differences in number of positive bursts, number of negative bursts, and burst power between arousal and valence levels, as seen in Table 1.