Brain-computer interfaces (BCIs) establish a communication link that directly connects the human brain to external devices (Yadav et al., 2020; Kawala-Sterniuk et al., 2021). BCIs primarily enable individuals with neuromuscular disorders, such as amyotrophic lateral sclerosis (ALS) and spinal cord injury, to communicate with the outside world (Abiri et al., 2019; Bonci et al., 2021). There are various types of BCI-based brain signals, including magnetoencephalography (Mridha et al., 2021; Peksa and Mamchur, 2023), electrocorticography (Miller et al., 2020), functional magnetic resonance imaging (Guo, 2020), and electroencephalogram (EEG)-based systems (Rashid et al., 2020; Ydaav and Maini, 2023). EEG-based BCIs have gained significant attention owing to their non-invasiveness, affordability, high temporal resolution, and widespread availability (Rashid et al., 2020; Ydaav and Maini, 2023).

The human brain produces different types of event-related potentials (ERPs) in response to particular stimuli (McWeeny and Norton, 2020). The present study focuses on the P300-based ERP, which is a positive peak observed in the EEG waveform that is associated with cognitive information processing in the brain (Valakos et al., 2020). The amount of stimulus information is reflected by the amplitude of the P300 wave, with larger P300 waves being generated by greater deviation. P300 responses are frequently triggered using the “oddball” paradigm or rapid serial visual presentation (RSVP), which involves presenting a series of different stimuli, one of which is infrequent relative to the others (Lees et al., 2018).

The P300 typically occurs about 300 ms after a targeted stimulus and can vary depending on an individual's ability to differentiate events (Farwell and Donchin, 1988; Zhang et al., 2021). The P300 paradigm is regarded as being user-friendly and appropriate for use in BCI applications, particularly those that involve spelling, as it is less likely to cause eye strain (Mendoza-Montoya et al., 2021). Farwell and Donchin introduced the Matrix Speller, which was the first device built on the P300 component (Farwell and Donchin, 1988). Following their work, other studies have suggested diverse paradigms and strong algorithms improve performance, resulting in significant enhancements in both theoretical and experimental approaches (Lotte et al., 2007; Kaufmann et al., 2011; Abiri et al., 2019; Lu et al., 2019, 2020a,b). One of these paradigms is RSVP, which involves presenting a series of stimuli in quick succession, usually at the center of the screen. RSVP can elicit P300 responses when the user sees a target stimulus among distractors (Lin et al., 2018). RSVP has some advantages over the oddball paradigm, such as reducing the number of flashes required to spell a word and avoiding the need for gaze control (Won et al., 2019).

Although the P300 paradigms exhibit promising potential in the BCI field, a significant obstacle is the fact that ERP signals are specific to each subject (Wu et al., 2022). Individuals have varying patterns of brain activity when responding to the same stimuli. As a result, it is necessary to calibrate the system for each person. In the calibration process, the individual is requested to execute a set of tasks that enable the system to acquire knowledge and adjust to their distinctive brain patterns. Although calibration is critical for achieving accurate and reliable results, its time-consuming nature presents a notable challenge for the widespread implementation of BCI.

By utilizing transfer learning (TL) techniques, it is possible to decrease the training time and data needed for a new model, especially when training from scratch is not possible. TL allows machine learning models to be trained on existing datasets from other subjects (known as the source dataset) to apply the acquired knowledge to a new subject (referred to as the target dataset).

One common approach in transfer learning is to adapt a pre-trained model from a source domain to a target domain. Fine-tuning is a variant of this method that utilizes the weights of a pre-trained deep neural network (DNN) based on the source dataset as the initial synaptic configuration for training the target network. Fine-tuning of DNN-based convolutional neural networks (CNNs) is widely used for P300-based BCI applications (Kundu and Ari, 2020a; Kilani et al., 2022), allowing the model to benefit from the general feature extraction capabilities learned from the source dataset. This provides a strong foundation for learning even with limited labeled data from individual subjects and offers faster training times due to the pre-trained weights.

In Kundu and Ari (2020a), a CNN-based TL is introduced to extract high-level features from a fully connected layer of CNN for P300-based character recognition. Then a Fisher-based feature selection technique was employed to achieve the most optimal feature set. Their results demonstrated that a selected set of CNN-extracted deep features outperformed the manually designed features.

Another method in TL is subspace alignment (SA) which aims to increase the distributional similarity between the source and target datasets by aligning the feature distributions between the two datasets (Kundu and Ari, 2020a). Since the source and target datasets have different characteristics in P300 signals, such as variations in the recorded signals, domain adaptation can reduce the need for extensive calibration and training, ultimately resulting in a more efficient and accurate BCI system (Zanini et al., 2018; He and Wu, 2020). Zanini et al. (2018) suggested a method called Riemannian alignment (RA) for aligning EEG data recorded during motor imagery (MI) task. RA calculates the covariance matrices of resting trials and computes their Riemannian mean, and uses it as the reference matrix to align all covariance matrices. To overcome the limitations of RA in terms of flexibility, lower computational cost, and unsupervised nature, He and Wu (2020) proposed a technique called Euclidean alignment (EA) that aligns EEG trials from different subjects in Euclidean space. EA works by identifying a projection matrix that can align the dataset in Euclidean space. The objective is to mitigate the distributional shift between the two domains and enhance the resemblance between their data distributions. In He and Wu (2020) the efficacy of the EA algorithm was assessed on two distinct datasets: one centered on P300 and the other on MI.

In transfer learning, the quality and relevance of source samples utilized for model training can considerably affect the model's performance. To achieve better transfer learning results and avoid negative transfer, it is important to select the most informative and applicable source samples (Wang et al., 2019; Zhuang et al., 2021; Kilani et al., 2023). Selection data from source domains that are more like the target data has also been used in other works. In Wei et al. (2020), the authors adopted a performance-based approach to select the source datasets. A classifier was trained for each source subject, then target data was given to it as evaluation data. Finally, source subjects that exhibited the highest classification performance were selected. While this approach led to improved performance compared to using all source datasets, the challenge of training different classifiers made it difficult to implement in real-time. In Qi et al. (2018), a novel sample selection method using Riemannian geometry measurement has been introduced for P300-based character recognition. A reference epoch was constructed by using a limited number of epochs from target data. The source samples were then selected based on their Riemannian distance to this reference epoch. Their results showed higher character recognition accuracy for sample selection based on Riemannian distance compared to other selection methods. However, Riemannian measurements require more computation, and the choices for classification models are limited compared to Euclidean measurements (He and Wu, 2020; Kilani et al., 2023).

In response to the challenges presented by P300-based BCI, we suggest a new approach to transfer learning that combines the advantages of fine-tuning and subspace alignment. Our approach utilizes finetuning for extracting the high-level features of deep neural networks. Then a domain adaptation technique is used to improve the similarity between the source and target feature distributions. Moreover, to reduce negative transfer, a source sample selection approach is used to choose the samples that are more similar to the target domain data samples. Our transfer learning approach is subsequently implemented on the discriminative restricted Boltzmann machine (DRBM) as the classifier.

Section 2 provides a detailed description of the methods and materials used in our study. We begin by presenting the datasets and explaining the preprocessing steps that were taken to ensure the quality of the data. Next, we describe our proposed approach, which combines the fine-tuning method and subspace alignment using the EA approach. Finally, we outline the process of applying the extracted features using our transfer learning approach to the DRBM classifier. In Section 3, we present the simulation results, including a comparison with other methods and an analysis of the effectiveness of our approach. Finally, in Section 4, we conclude with a summary of our contributions and discuss research in this area.

We utilized two datasets (dataset II from BCI competition III and RSVP dataset) to evaluate the proposed method, as introduced in Section 2.1. In the initial phase, we showcase our effectiveness using Dataset 1, illustrating our commendable performance in both offline and online scenarios (more details about the scenarios in Section 2.3.2). Subsequently, in the second phase, we present the findings obtained from Dataset 2, providing more comprehensive insights through ablation experiments. Given the larger pool of subjects in this dataset, we can effectively showcase our robustness in this context.

This study introduced a new transfer learning approach for the ERP-based brain-computer interface that incorporates source sample selection to improve the performance of the system further. Source sample selection in transfer learning has several advantages. It can help to reduce the negative effects of domain shift, where there are differences between the distribution of data in the source and target domains. By selecting appropriate sources, it is possible to mitigate these negative effects and improve performance on the target task. The proposed selection approach chooses the most relevant source samples based on their similarity to the target samples to enhance the transferability of the model further.

In addition, the proposed approach utilized fine-tuning and data alignment techniques to improve the performance of the P300 BCI, especially when dealing with limited labeled data. Specifically, the approach fine-tunes a pre-trained model on a similar task using a small amount of labeled data and aligns the data distributions between the source and target domains to minimize the domain shift. The results of our experiments showed that the proposed approach outperforms the baseline models and achieves state-of-the-art performance on the datasets used in this study.

Furthermore, the proposed approach is shown to be generalizable to other datasets, demonstrating its potential for wider applicability. One of the most important aspects of the proposed method is extracting high-level features in a new domain (EA) while minimizing the difference between both feature groups of source and target. Importantly, while Euclidean Alignment has been explored in the context of transfer learning for BCIs, our work demonstrates its novel application to high-level features extracted from convolutional neural networks. This approach directly addresses the inherent variability of EEG signals between subjects, facilitating more robust knowledge transfer and improving performance compared to methods that focus solely on raw time samples. The findings of this study suggest that fine-tuning, data alignment, and source sample selection could be promising techniques for enhancing the performance of ERP-based BCIs while reducing calibration time and could pave the way for further research in this area.

The original contributions presented in the study are included in the article/supplementary material, further inquiries can be directed to the corresponding author.

Ethical approval was not required for the study involving humans in accordance with the local legislation and institutional requirements. Written informed consent to participate in this study was not required from the participants or the participants' legal guardians/next of kin in accordance with the national legislation and the institutional requirements.

SK: Conceptualization, Formal analysis, Investigation, Methodology, Software, Validation, Visualization, Writing – original draft, Writing – review & editing. SA: Conceptualization, Formal analysis, Investigation, Methodology, Software, Supervision, Writing – original draft, Writing – review & editing. YF: Supervision, Writing – original draft, Writing – review & editing. AS: Supervision, Writing – original draft, Writing – review & editing.