EEG-based BCIs have demonstrated significant potential across various downstream tasks, with auditory attention decoding (AAD) and emotion recognition representing two prominent application domains. In AAD research, the challenge stems from the cocktail party effect—the neurocognitive ability to selectively focus on target speakers in multi-talker environments (Cherry, 1953), which contrasts with difficulties experienced by hearing-impaired populations (Cai et al., 2024). Neurophysiological signal analyses through ECoG (Mesgarani and Chang, 2012), MEG (Akram et al., 2016), and EEG (O'sullivan et al., 2015) have enabled AAD implementations, catalyzing developments in neuro-steered hearing aids (Ceolini et al., 2020). For emotion recognition, the field seeks to model higher-order cognitive functions encoded in neurophysiological signals (Tan et al., 2021). While emotional states manifest through various modalities, the susceptibility of physical expressions to masking effects positions non-invasive EEG as a robust solution for emotion decoding (Xu et al., 2024; Li et al., 2019).

SNNs have emerged as a promising computational framework for both applications, leveraging their inherent low-latency processing and energy-efficient characteristics. In AAD research, Faghihi et al. (2022) developed efficient left/right attention pattern decoding, while Cai et al. (2023) proposed BSAnet, integrating biologically plausible mechanisms with attention modeling for temporal dynamics capture. Recent advances include spiking GCNs for spatial feature extraction (Cai et al., 2024), demonstrating promising results in low-density electrode scenarios. In emotion recognition, pioneering SNN applications have shown methodological viability. Tan et al. (2021) implemented NeuroSense achieving 78.97%/67.76% (arousal/valence) accuracy on DEAP, while Alzhrani et al. (2021) attained 94.83% accuracy using bidirectional spiking networks on DREAMER. Recent developments include fractal SNN architectures (Li et al., 2023), SGLNet for spatiotemporal extraction (Gong et al., 2023), and EESCN achieving 94.81% accuracy on DEAP and SEED-IV (Xu et al., 2024). However, previous research has predominantly relied on manually extracted EEG features such as power spectral density (PSD) and differential entropy (DE) (Jiao et al., 2018; Song et al., 2018), and automatic EEG feature extraction in this domain remains largely unexplored.

Traditional SNNs, despite their inherent advantages in energy efficiency and biological plausibility, still exhibit a performance gap compared to their DNN counterparts. Therefore, many recent works have integrated attention mechanisms into SNNs to enhance their performance and capabilities (Yao et al., 2021; Zhu et al., 2024; Zhou et al., 2024; Lu et al., 2025). Yao et al. (2023) addressed this through Spike-Driven Self-Attention (SDSA), reformulating matrix multiplications as masking operations to ensure purely binary spike signal transmission. Building on this foundation, Yao et al. (2024) introduced the Meta-Spikeformer architecture that extended the SDSA operator. Those advancement inspired subsequent research exploring SNN-specific attention mechanisms. Wang et al. (2023) proposed Spatiotemporal Self-Attention (STSA) for SNNs, maintaining asynchronous transmission while capturing spatiotemporal feature dependencies. More recently, Wang et al. (2025) developed Saccade Spike Self-Attention (SSSA), enabling comprehensive spatiotemporal feature processing for holistic visual scene understanding in SNN paradigms. Overall, these novel spiking self-attention mechanisms have significantly advanced SNN performance. However, there remains a lack of effective spiking self-attention designs specifically tailored for EEG signal processing.

SNNs rely on spiking neurons (Maass, 1997) as their basic unit of information transfer, and common spiking neurons include the Hodgkin-Huxley (Abbott and Kepler, 2005), Izhikevich (Izhikevich, 2003), and Leaky Integrate-and-Fire (LIF) (Izhikevich, 2003) model. In this work, we use the LIF model as the spiking neuron in the proposed method. The LIF model is a simple and effective spiking neuron model. When the membrane potential reaches a certain threshold, the neuron emits a spike, followed by a reset of the membrane potential to the resting potential V_reset. The dynamic model of LIF is described as:

where τ is the membrane time constant, and X[t] is the input current at time step t. When the membrane potential H[t] exceeds the firing threshold V_th, the spiking neuron triggers a spike S[t]. Θ(·) is the Heaviside step function which equals 1 for v≥0 and 0 otherwise. V[t] represents the membrane potential after the trigger event which equals H[t] if no spike is generated, and otherwise equals to V_reset.

Wavelet transforms (WTs) are powerful signal-processing tools that enable the localization of signals in both time and frequency domains, which is particularly useful for analyzing non-stationary signals like EEG. The discrete wavelet transform (DWT), in particular, provides an efficient method for multi-resolution analysis by decomposing signals into sub-bands corresponding to different frequency scales. This decomposition enables the extraction of local features at various scales, making it well-suited for EEG signal processing. EEG signals are nonlinear and non-stationary, posing challenges for traditional analysis methods in capturing their time-varying and multiscale nature. Wavelet transforms, and specifically DWT, offer a significant advantage in feature extraction and time-frequency characterization of EEG signals. The DWT decomposes EEG data into frequency bands such as delta (0.5–4 Hz), theta (4–8 Hz), alpha (8–13 Hz), beta (13–30 Hz), and gamma (greater than 30 Hz). This decomposition allows us to extract meaningful features from the EEG data that correspond to various cognitive and emotional states.

For our application, we employ the Haar wavelet due to its simplicity and computational efficiency. Haar wavelets are among the earliest and simplest wavelet functions, characterized by a two-tap filter with minimal support, which results in fast computations. Compared to other common wavelets like Daubechies or Morlet, Haar wavelets are computationally less expensive, requiring only additions and binary shifts, which makes them well-suited for real-time, low-power applications such as SNN-based systems. Haar wavelets are particularly efficient in extracting local, low-frequency components (such as delta and theta waves) as well as high-frequency components (like beta and gamma waves), which are essential for distinguishing different cognitive states in EEG analysis. The efficiency and simplicity of Haar wavelets also make them ideal for handling the sparse, event-driven nature of SNNs.

The Transformer architecture, originally devised for natural language processing tasks (Vaswani et al., 2017), has subsequently permeated multiple subfields of artificial intelligence. At its core lies the self-attention mechanism, which facilitates selective information processing by focusing on relevant contextual elements. Spikformer (Zhou et al., 2022) pioneered the integration of self-attention into SNNs through their Spiking Self-Attention (SSA) framework and spikformer architecture. This approach innovatively employs sparse spiking representations for the query (Q), key (K), and value (V) matrices:

here, Q, K, and V form tensors of dimension ℝ^T×C×H×W, with BN(·) representing batch normalization and SN(·) denoting the spiking neuron layer that maintains the attention mechanism's spiking nature. The similarity computation between spiking Q and K matrices proceeds via dot-product:

The attention output is subsequently calculated as a scaled weighted sum of V, transformed through spiking neuron activation, and further processed through linear transformation and batch normalization before final spiking neuron conversion to produce the output Z:

Given an EEG dataset Deeg, it can be represented as:

where xieeg∈Xeeg denotes the raw EEG input signal for the i-th sample, and yi∈Y represents its corresponding label (emotion category or auditory attention state). Our objective is to learn a spiking neural network model F_θ with parameters θ to predict the class label from the EEG input. The model is optimized by minimizing the expected risk based on the cross-entropy loss L_CE:

In this study, we present a novel spiking transformer model, denoted as F_θ, to learn discriminative spatio-temporal representations directly from raw EEG signals for the joint tasks of emotion recognition and auditory attention decoding. To achieve this, we introduce a novel Spiking Wavelet Self-Attention (SWSA) mechanism within a spiking transformer framework. While conventional Spiking Self-Attention (SSA) enables efficient event-driven computation, it is limited in its ability to capture the multi-scale frequency dynamics intrinsic to non-stationary EEG signals. The proposed SWSA overcomes this limitation by integrating Haar wavelet transforms for joint time—frequency analysis, which offer a minimal filter length and computational simplicity, making them highly efficient for real-time processing. Compared to other wavelet bases, such as Daubechies and Morlet, Haar's shorter filters and multiply-free operations align well with the event-driven, low-power nature of spiking neurons. This integration allows the model to focus on neurophysiologically relevant rhythms (e.g., alpha and beta bands) critical for emotional and attentional processes, while maintaining energy-efficient computation. Finally, a cross-entropy loss function is employed to enable effective gradient-based optimization for learning highly discriminative features across both tasks.

The overall workflow of the proposed method is depicted in Figure 2. Raw EEG signals are first preprocessed and segmented into overlapping windows via a sliding window strategy to preserve temporal continuity. To more effectively capture the spatial characteristics of EEG activity, α-band cortical signals are extracted and projected onto 2D topographic maps, thereby maintaining brain-region dependencies. These maps are subsequently divided into patches and tokenized into fixed-length sequences, which serve as the input to a stack of N spiking encoder blocks. Finally, the resulting features are fed into an MLP classification head to predict the corresponding emotional or attentional state. In summary, this high-level pipeline constitutes the basis of the proposed model architecture, which is elaborated in the following section.

Building on the workflow described above, we design SpikeWavformer—an end-to-end spiking transformer architecture that combines wavelet-based multiscale analysis with spiking attention to enhance EEG feature representation. As shown in Figure 3, The SpikeWavformer can be written as follows:

Given the EEG input X, SpikeWavformer first visualizes the spatial focus position via the topographic distribution of oscillatory cortical activities in the α band and converts it into a 2D image. Subsequently, the SPS module partitions the input into patches and progressively extracts features, optionally incorporating wavelet transformation to enhance multiscale feature representation. Then, L× spiking wavelet encoder blocks with spiking wave attention mechanism are employed to encode the features. Finally, the features obtained from extraction and encoding are compressed into a fixed-dimension vector via global average pooling (GAP) and fed into a fully connected layer classification head (CH) to produce classification results.

As an essential neurophysiological signal, EEG plays a pivotal role in research areas such as affective computing and auditory attention decoding. Nevertheless, its multi-channel structure, low signal-to-noise ratio (SNR), pronounced temporal non-stationarity, and intricate time–frequency characteristics present substantial challenges for existing analysis techniques. Conventional CNNs are limited in capturing long-range temporal dependencies inherent in EEG data. In contrast, vanilla Transformers possess strong long-range modeling capability but incur prohibitive computational costs when processing long-sequence EEG signals. Furthermore, many existing approaches employ irreversible downsampling during multi-scale feature extraction, resulting in the loss of critical frequency-domain information. This drawback is particularly detrimental to neural decoding tasks that rely on specific frequency bands.

To address these issues, we propose a Spiking Wavelet Self-Attention (SWSA) mechanism for EEG signal processing. It combines the biological plausibility of SNNs with the flexible time-frequency analysis of wavelet transforms, offering an efficient, biologically inspired solution for EEG-based emotion recognition and auditory attention decoding. Specifically, given multi-channel EEG inputs X∈ℝ^{T×B×C×H×W}, where T denotes time steps, B batch size, C EEG channels, and H×W spatial-topological 2D arrangement. The frequency-domain features of EEG signals are crucial for neuro-decoding. Different frequency bands correspond to different cognitive states: δ with deep sleep, θ with memory encoding, α with relaxation, β with attention and cognitive activities, and γ with perception and higher-order functions. We adopt the Haar wavelet for its minimal filter length and computational simplicity, which enable fast, low-power multiscale decomposition and align well with the event-driven, resource-constrained nature of SNN-based BCI systems. Specifically, the Haar wavelet is used for multiscale decomposition and perform DWT on EEG features at each time step t:

here, XLL(t) captures low-frequency components (like δ, θ), while high-frequency sub-bands XLH(t), XHL(t), XHH(t) retain high-frequency information (β, γ). Then, spatial local convolution enhances frequency-band interactions:

here, BN is batch normalization, LIF a spiking neuron layer. IDWT reconstructs spatial-domain features:

Our encoder, inspired by vanilla encoder (Vaswani et al., 2017), first calculates block-input spikes for self-attention. Three matrices Wq∈ℝd×dq, Wk∈ℝd×dk, Wv∈ℝd×dv map tokens to vectors. Spiking neurons convert vectors to spiking sequences Q, K, V:

Next, we compute Q-K similarity. Following Zhou et al. (2022), a scaling factor s controls matrix-multiplication magnitude without affecting attention properties:

To integrate wavelet and attention features effectively, we use channel-wise concatenation:

By integrating wavelet decomposition with spiking mechanisms, SpikeWavformer enables efficient processing of long-sequence EEG data while facilitating the analysis of cross-frequency neural dynamics, thereby providing richer feature representations for complex neuro decoding tasks. Specifically, we analyze the advantages of integrating wavelet transform into SNNs from the perspectives of convergence and convergence speed. First, We define the EEG signal space as X = {x∈ℝ^T×C×H×W}, where T represents time steps. C denotes channels, and H×W represents spatial dimensions. The discrete wavelet transform operator is defined as:

where Y = {(X_LL, X_LH, X_HL, X_HH)} represents the wavelet coefficient space.

The SWSA mechanism can be formalized as a composite operator:

where W is the DWT operator, Attn is the spiking attention operator, F is the fusion operator, ⊕ denotes concatenation and W⁻¹ is the inverse DWT (IDWT).

Theorem 1 (Lipschtiz Continuity of SWSA): The SWSA mechanism satisfies Lipschitz continuity (Hager, 1979; Gouk et al., 2021; Goldstein, 1977) with constant L_SWSA, ensuring stable convergence during training.

Proof: First, we establish the Lipschitz properties of individual components: Haar Wavelet Transform Lipschitz Property: For the Haar wavelet transform W, we have: ||W(x₁)−W(x₂)||₂ ≤ L_W||x₁−x₂||₂. Since Haar wavelets are orthonormal, L_W = 1. Spiking Attention Lipschitz Property: For the spiking attention mechanism with LIF neuron, let ϕ(μ) = Θ(μ−V_th) be the spike generation function. The membrane potential dynamics: V[t] = τV[t−1]+X[t]−v_resetS[t−1]. For bounded inputs, the LIF neuron satisfies: |ϕ(μ1)-ϕ(μ2)|≤1Vth|μ1-μ2|. Therefore, LA=1Vth. Combined Operator: The SWSA operator combines these components: ||SWSA(x₁)−SWSA(x₂)||₂ ≤ L_SWSA||x₁−x₂||₂, where LSWSA=LW·LA·LF=LFVth with L_F being the Lipschitz constant of the fusion operation.

Corollary 1: Under the assumption that L_SWSA < 1, the SWSA operator is a contraction mapping, guaranteeing convergence to a unique fixed point.

Theorem 2 (Accelerated convergence): The SWSA mechanism achieves faster convergence compared to vanilla spiking self attention.

Proof: Consider the optimization landscape with loss function L(θ). The gradient update for SWSA parameters follows: θ_t+1 = θ_t−α∇_θL(θ_t). The wavelet decomposition provides a natural regularization through frequency localization:

This L₁ regularization on wavelet coefficients promotes sparsity. The convergence rate is bounded by:

where the wavelet regularization reduces the effective variance σ², leading to faster convergence.

We conduct experiments on the DEAP and KUL datasets using proposed SpikeWavformer and compare the results with existing methods for emotion recognition and auditory attention decoding. As shown in Tables 1, 2, our method achieves state-of-the-art performance on all datasets. On the DEAP dataset for emotion recognition, the SpikeWavformer method reaches an Arousal accuracy of 76.51% (std: 5.48%) and a Valence accuracy of 77.10% (std: 5.68%). Existing methods like EEGNet (Lawhern et al., 2018) achieve 58.29% (std: 8.60%) for Arousal and 54.56% (std: 8.14%) for Valence. SCN (Schirrmeister et al., 2017) attains 61.19% (std: 10.28%) for Arousal and 59.42% (std: 8.30%) for Valence. DCN (Schirrmeister et al., 2017) gets 61.03% (std: 8.58%) for Arousal and 59.92% (std: 7.82%) for Valence. Tsception (Ding et al., 2022) achieves 61.57% (std: 11.04%) for Arousal and 59.14% (std: 7.60%) for Valence.

We further compared the performance of the SpikeWavformer for different detection window sizes, ranging from 0.1 to 10 seconds, with the results presented in Table 2. On the KUL dataset, the SpikeWavformer achieved an average decoding accuracy of 96.5% across all subjects for a 1-second decision window, 97.1% for a 2-second decision window, 97.3% for a 5-second decision window, and 98.6% for a 10-second decision window. Generally, larger decision windows yielded better results, corroborating findings from previous studies (De Taillez et al., 2020; Ciccarelli et al., 2019; Vandecappelle et al., 2021). Notably, our proposed method is capable of decoding auditory spatial attention with a very short decision window of less than 1 second. For decision windows of 0.5 seconds and 0.2 seconds, the SpikeWavformer attained high accuracy rates of 94.2% and 86.7%, respectively. Although the accuracy for the 0.1-second decision window was lower than that of the 1-second decision window, SpikeWavformer maintained a high accuracy rate of 80.5%. In all comparisons with related work (De Cheveigné et al., 2018; Cai et al., 2021; Su et al., 2022), the SpikeWavformer demonstrated competitive performance.

In this section, we validate the energy efficiency of our proposed model over its ANN counterpart. Based on the energy calculation standard in neuromorphic computing (Sengupta et al., 2019), we use the method proposed by Wang et al. (2024) to compute the energy consumption ratio between our model and the equivalent ANN model:

In the equation, ACMAC denotes the energy consumption ratio of an accumulate (AC) operation in SNNs to a multiplication (MAC) in ANNs. Extensive studies confirm the theoretical value of ACMAC is 17 (Horowitz, 2014). Here, SpikingRate is the average spiking rate, and TimeSteps the simulation time window. In our model, SpikingRate is 12.3%, and TimeSteps is set to 4. Based on Equation 24, our model achieves over 7× energy efficiency compared to its ANN counterpart.

In this section, saliency maps (Simonyan et al., 2013) are employed to visualize the areas of the data that contain the most information and contribute to classification performance. The saliency map is one of the most widely used tools for illustrating which regions of the input data hold classification-relevant information. To enhance the visualization of the saliency maps, the original maps were averaged along the time dimension to capture the topology of the EEG channels. Additionally, the normalized saliency maps were averaged across different samples for each subject to produce generalized average saliency maps. The average saliency maps for the DEAP dataset and the KUL dataset are presented in Figures 4, 5, respectively.

DEAP. For arousal, as illustrated in Figure 4a, the temporal and frontal regions of the brain contain a wealth of information. This indicates that these regions are more involved in processing emotions, aligning with findings from previous studies (Gao et al., 2021; Huang et al., 2012; Mickley Steinmetz and Kensinger, 2009). Emotional arousal is predominantly represented in the temporal and frontal lobes. The asymmetry between the frontal and temporal lobes is closely associated with emotion recognition within the arousal dimension. In terms of valence, Figure 4b shows that the parietal and temporal lobes are also rich in information. This observation is consistent with earlier research (Huang et al., 2012), suggesting that the network effectively learns from these relevant regions.

KUL. It is expected that the areas of neural activity contributing to speech processing will exhibit greater significance. As illustrated in Figure 5, the average saliency map of the KUL dataset reveals that the frontal and temporal regions contain more substantial information. These findings align with previous research indicating that activation is prominently observed in the frontal and temporal cortices (Ciccarelli et al., 2019; Geirnaert et al., 2020; Vandecappelle et al., 2021).