Patients diagnosed with drug-refractory temporal lobe epilepsy (TLE) who underwent stereoelectroencephalography (SEEG) for localization of seizure focus were included in the study. The indication for SEEG was clinically necessary and determined in a multidisciplinary patient management conference. Within this cohort, consenting adults who had thalamic implantation for research were included in the present analysis. The multi-step consenting and evaluation process has been described in detail in our previous studies (13, 14, 17). The electrophysiological sampling of the thalamic subnuclei was performed under the supervision of the IRB, and all patients provided written informed consent. To mitigate the risk associated with implanting an additional depth electrode for research sampling of the thalamus, we modified the trajectory of a clinically indicated depth electrode sampling the operculum-insula to track medially for recording from the thalamus. Clinician-identified seizures were documented for all patients, and the SEEG data was clipped and parsed for analysis. Ictal (N = 84 from 13 patients) and baseline interictal data (Length: 550 s prior to seizure) were obtained. The demographic details of the study participants are detailed in Table 1.

All SEEG implantation procedures were performed using robotic assistance (ROSA device, MedTech, Syracuse, NY) (12–16 contacts per depth electrode, 2 mm contact length, 0.8 mm contact diameter, 1.5 mm intercontact distance, PMT® Corporation, Chanhassen, MN). Once implanted, the patients were monitored over 4–12 days in the epilepsy monitoring unit (EMU). SEED data as recorded using Natus Quantum (Natus Medical Incorporated, Pleasanton, CA, sampling rate 2,048 Hz). Signals were referenced to a common extracranial electrode placed posteriorly in the occiput near the hairline.

The details of the accuracy of the implantation strategy have been reported in our previous study (17). Here we highlight the main steps to localize the SEEG electrodes to the various cortical regions and the thalamic subnculei. The post-implant CT-scan was coregistered to preimplant MRI using Advanced Normalization Tools (ANTs) and refined registration of deep structures was performed using brain shift correction to improve the registration of subcortical structures using Lead-DBS v2 software (18). Both the images were normalized to ICBM 2009b NLIN asymmetric space using the symmetric diffeomorphic image registration. Following this, the localization of the thalamic contacts was performed in Lead-DBS, while the cortical channels were performed in iElecetrodes (19). Thalamic contacts were registered to Morel's thalamus atlas, while cortical contacts were localized using AAL2 atlas (20).

The time of the seizure onsets and offsets was annotated by a board-certified epileptologist (SP). Seizure onset in the cortex was marked as “unequivocal EEG onset” (UEO) at the earliest occurrence of rhythmic or repetitive spikes that was distinct from the background activity. SEEG segments were clipped to include 10 min before this UEO and 10 min after seizure termination. Four different clinical seizure types were included for analysis: focal aware seizures (FAS), focal impaired awareness seizures (FIAS), and focal to bilateral tonic clonic seizures (FBTCS) (21). Epochs of “interictal state” (28 epochs with each epoch lasting 9 min) were visually screened and identified from the non-seizure segment of the SEEG that was at least 1 h preceding seizure. Our previous study showed that the interictal spikes in baseline data need not be actively removed for classifying ictal states from baseline states (13). Secondly, interictal spikes will be present while training real-time data, e.g., line length detection in responsive neuromodulation systems (RNS) (22). Hence for translational purposes, no effort was made to exclude epileptiform spikes in the baseline epochs. Supplementary Material shows the details of the ictal and baseline data.

The input to the BiLSTM classifier was the interictal baseline and ictal thalamic EEG data. The baseline and the ictal data from the 84 seizures (13 subjects) were initially grouped by the subject identification. Since the length of ictal data was variable, we chose the length of the shortest seizure for any given subject (i.e., 14 s) as the length of the analyzable data. To avoid a discrepancy in the length of data between ictal and baseline segments, we chose a similar 14 s length of SEEG data from the initial segment of the baseline data. The input to the BiLSTM classifier is a 2D array of data. Hence, the 14 s of the data were then clipped into multiple 1 s epochs and rearranged into a two-dimensional array of 14 × 2,048 samples (Figures 1A–C).

Principally, the LSTM network only obtains information from the previous input observations but cannot use that information for future input observations. However, the BiLSTM model, composed of two independent LSTM networks, can transmit information bi-directionally and increase the learning ability of the system output (Figure 2) (23). Sixty three seizures from 11 subjects (were used to train the classifier differentiating ictal from baseline. Subsequently, 21 seizures collected from 2 patients were used for testing the model. Each BiLSTM classification model consisted of 64 units of LSTMs in the encoding layer and a kernel regularizer of 12 followed by a drop-out layer with a drop-out ratio of 0.25 and a batch normalization layer. This was followed by a dense layer of 64 units with rectified layer unit (ReLU) activation function (24). For the final output, a dense layer with SoftMax activation function of two units was used for the binary classification of baseline interictal and ictal states.

GANs learn and generate synthetic data by preserving the data distributions. For a generation of sequential data, the temporal dynamics need to be preserved. Yoon et al. (25) proposed the concept of time-series GAN (TGAN) that was able to capture not only the distribution of data at each instant but also the presence of various features across time (Figures 3A,B). TGAN differs from other GAN architectures in two ways. (a) By introducing an embedding network, it reduces the dimension of the adversarial learning space, and (b) uses supervised adversarial loss, unlike GAN, where unsupervised methods are used. In our analysis, TGAN was used to generate synthetic data that was 10 times the original data. The data (ictal and baseline) were fed into the TGAN model to produce the augmented data (Figure 1D).

The second level BiLSTM analysis classifies ictal and baseline states based on the synthetic TGAN data (Figure 1E). Hence it was necessary to validate the similarity of the synthetic data with the original data. The validation was quantified using the Train-Synthetic-Test-Real method (TSTR), where a logistic regression classifier model with a single layer gated recurring units (GRU) with 12 units was used for training both the original and the synthetic data. Each model was trained separately with 75% of the original and the synthetic data, respectively. The testing of both the models is done with the remaining 25% of only the original data set (25). This allowed us to estimate individual seizure level and subject level coefficient of determination (R²) values and the percentage difference between original and synthetic data (Table 2). Finally, a t-distributed stochastic neighbor embedding (t-SNE) analysis was performed to visualize if the ictal and baseline data could be better segregated using the original or the synthetic data. T-SNEs were generated in MATLAB using the “exact” algorithm, with Mahalanobis distance, the perplexity of 50, and PCA dimensions of 3.

To estimate the performance of the BiLSTM following metrics were computed: sensitivity (Sn or recall), specificity, accuracy (for training, validation, and testing data), positive predictive value (PPV or precision), F1-score, and area under the curve (AUC). The difference in the performance of the BiLSTM classifiers for original and synthetic data was visualized using ROC (receiver operating characteristic) curve by testing the relationship between sensitivity and 1-specificity.

The BiLSTM and TGAN models were tested in Python, and t-SNE analysis was performed in MATLAB. We utilized Keras (26), scikit-learn, an open-source Python API that takes into account the neural organization structures based on top of TensorFlow, to construct all learning models.

Thirteen subjects were included in the study, with 10 had electrodes localized to the anterior nucleus of the thalamus (ANT) and 3 in the centrolateral thalamic nuclei (Table 1, Figure 4). CT brain (post-implant and post-explant) did not show any thalamic hemorrhage. Eight subjects were implanted on the right side and three on the left side.

Table 1 summarizes the clinic-demographic details of the subjects included in this study. A total of 84 seizures from 13 subjects were analyzed. The seizure onset zone was determined based on the clinical consensus among the epileptologists during the epilepsy surgical conference. The identified seizure focus was: medial temporal (4 subjects), mesial + temporal pole onset (3 subjects), temporal plus (5 subjects), with the plus representing additional seizure foci (orbitofrontal or insula or suprasylvian operculum) (27). The seizure types were: ES (19), FAS (24), FIAS (28), and FBTCS (6).

The TGAN generated synthetic data was similar to the original SEEG data. At baseline, there was no difference between the mean coefficient of determination (R²) of the original and synthetic data (original: 0.484 ± 0.251, synthetic: 0.487 ± 0.256, t = −0.6, p = 0.27). Similarly, there was no difference in the mean absolute error (MAE) of original and synthetic data (original: 10.34 ± 8.25, synthetic: 10.32 ± 8.21, t = 0.008, p = 0.49). Similarly, the TGAN augmented data synthesized during the ictal period did not differ from the original data in R² (original: 0.491 ± 0.221, synthetic: 0.492 ± 0.224, t = −0.0097, p = 0.49) and the MAE (original: 12.95 ± 15.42, synthetic: 12.94 ± 15.42, t = 0.001, p = 0.49).

We constructed ROC curves to determine the performance of the BiLSTM on original, and TGAN augmented synthetic data. The classification of the ictal from the baseline data was superior with the synthetic TGAN augmented data compared to the original data (original: AUC: 60% and synthetic: 78.5%, Figure 5A). This improvement in the performance of the BiLSTM models could be better visualized using three component t-SNE plots (Figures 5B–D). T-SNE of original data failed to parse the ictal and baseline clusters separately (Figure 5B), while T-SNE performed on the TGAN augmented synthetic data with the same parameters, demonstrated a clear separation into ictal and the baseline clusters (Figure 5C). We initially noted that the ictal clusters were further separated in space into multiple clusters. A t-SNE indexed by the subject ID showed that TGAN amplifies the ictal data specific to each patient that is distinctly different from their comparable baselines (Figure 5D). The result suggests that the patient-specific electrographic seizure onset patterns were retained in the TGAN augmented data (Figure 5D).

Overall, the performance of the BiLSTM in classifying ictal and baseline states from thalamic SEEG data was enhanced by the use of TGAN generated synthetic data over the original data. The accuracy of the training data improved by 31.75%, the validation data improved by 32.1%, and finally, the testing data improved by 18.5%. The sensitivity and PPV of the BiLSTM classifier on improved by 13 and 10% on the testing data (Figure 6).

Table 3 summarizes the performance of deep learning algorithms in detecting seizures from electrophysiological signals recorded from the scalp and intracortical regions (LFPs). To date, deep learning algorithms to detect seizures were applied to EEG obtained from the cortical areas that participate in seizure generation. Our study is distinct and the first of its kind to perform deep learning detection on EEG recordings from a brain region that is remote to the seizure focus.

As expected, the performance of these classifiers was higher with biosignals obtained directly from the cortex than from the scalp which is likely to be closer to the seizure focus. In those studies, the sample size for deep learning consisted of over 100 ictal EEG data. The main motivation of this study is to highlight how data augmentation techniques can improve the accuracy of the classifier, albeit a lower overall performance of our classifier in comparison to other studies. Even when data is smaller (84 seizures), we can use data-augmentation methods to enhance the performance of the classifier (accuracy improved by 18% in our current study) and improve the detection performed in subcortical neuromodulatory targets such as the thalamus, which are distant and outside the seizure cortex. Wei et al. (38) were among the pioneering teams in demonstrating improved seizure detection in scalp EEG with GAN models. They used the Wasserstein Generative Adversarial Nets (WGANs) combined with a convolutional neural network (CNN) to demonstrate a 3% improvement in accuracy (81.5–84.4%) and a near 2% improvement in the sensitivity (70.68–72.11%). In our study, the accuracy and sensitivity improved by 18.5 and 13%, respectively. Zhao et al. (39), in their model with a 1D-CNN with data-augmentation on data obtained from intracranial EEG data that was close to seizure focus, achieve an improvement of only 3% (accuracy of 89.28% compared to 86.89 with a support vector machine). They proposed a data augmentation method which leverages feature correlations in the transformed domain rather than in the original domain where time-domain data is converted to the frequency domain by discrete cosine transform (DCT), and new artificial data is generated by combining different frequency bands from different data, and converted back to time-domain data. Overall, these studies and ours, point to the promising future of using data augmentation techniques for better seizure detection to improve therapeutic stimulation.

GANs, are deep-learning algorithms where two competing networks, namely the generator and the discriminator, compete against each other until the generator generates artificial data of high quality. According to Goodfellow et al. (40), “the generative models are analogous to a team of counterfeiters trying to produce fake currency without being detected. The discriminative model is analogous to the police trying to detect fake currency. The competition between the generator and discriminator drives improvement until the counterfeiters are indistinguishable from the genuine currency.” Thus, GAN has been used to classify interictal spikes and in EEG-based brain-computer interfaces. Our result supports the use of GAN to produce synthetic data to augment the performance by 18% as compared to using only the original data. The ability to classify seizures from limited samples of unprocessed LFP signals may provide a clinical advantage in neuromodulation devices where efficient processing at a lower computational expense is desired.

There is a proof-of-concept study evaluating the use of synthetic data in augmenting sample size for deep learning. The study needs to be extended to a larger cohort with thalamic recordings of seizures and interictal baseline. One major challenge of the temporal GAN model is that it is computationally intense and consumes time to learn or converge to local minima and hence slows the training process. Another limitation of GAN is that the presence of discontinuous (e.g., ECoGs obtained from clinical neuromodulation devices) data may synthesize incorrect data. In our study, the duration of the data used for time-GAN analysis was 14 s based on the shortest duration of the ictal event from our cohort and in the future the results need to be optimized to individual patients, in whom the seizure durations are likely to vary significantly. This will also have a bearing on minimizing the detection latencies in the future models. Such sophisticated models will help build closed-loop neuromodulation strategies where early seizure detection can be used to pace the brain to abort seizures. Regarding the size of our data-set, we used 63 seizures from 11 patients' data for training and 21 seizures from 2 patients's data for testing the BiLSTM and Time-GAN models. While our study did show a reasonable improvement in the accuracy of the BiLSTM models and can be used as a proof of concept, in the future it is essential to validate this using random sampling of the patients' data and at individual subject level to emphasize and validate its clinical use. Also, cross validation across all patients' data would further strengthen the validity of the model. Since GAN is time consuming and resource demanding, our purpose was not to run it on all subjects but show how even in few subjects an improved accuracy can be obtained. Another limitation is that we did not determine the exact cause of improved accuracy and test the fidelity, diversity, and generalization of the data augmentation method, i.e., TGAN. These measures help determine the point at which the generative model surpasses and fools the discriminative network. Once the TGAN augmented data robustly mimics the real data, the TGAN-output is then used as the input in BiLSTM models improve accuracy. TGANs supersede BiLSTMs at finding a better low dimensional representation and hence may contribute to improved accuracy. In our study, we were interested in showing if the TGAN is able to produce synthetic data effectively from available limited samples and whether that use of the synthetic data shows elevated performance as compared to the using the original data only and a more detailed validation of TGAN was to voluminous for this study.