The experiment was approved by the ethical committee of Chongqing University of Posts and Telecommunications. Written informed consent was signed prior to participating in the study. Subjects received a monetary compensation after experiment. None of them had cognitive impairments, mental and neurological disorders.

Twenty healthy and right-handed male subjects (mean age 21 years old) participated in the experiment. Subjects were asked to keep relaxed and to restrain wide movements as much as possible during the whole experiments. Subjects were requested to perform a continuous delayed match task consisted of two sessions at three levels of task difficulty. The two sessions were exactly equal and the only distinction between the two sessions was that subjects carried out the first tasks without training but they performed the second tasks after WM training.

Figure 1 showed an example of the stimulus sequence. A fixation cross (0.5 × 0.5°; at the center of the monitor) was displayed throughout the entire block of trials. Each trial started with the fixation cross flashing for 50 ms. Following that, the memory array which was randomly consisted of two, four, or eight letters (0.5°× 0.5°) was presented for 200 ms with the same appearing frequency of two, four, and eight letters. After 3000 ms retention interval, the test array was presented on the screen for 100 ms as a probe item. Then subjects responded with a button press, indicating whether the probe item was in the memory array. Subjects pressed the key “F” with their left index fingers for the absence of the probe item from memory array, and pressed the key “J” with their right index fingers for the attendance of the probe item in the memory array. The number between the absence and the attendance of the probe items in the memory array was equal. Response accuracy (RACC) and speed were equally stressed in the instructions.

Subjects were required to maintain central fixation and minimize eye blinks and body motion throughout the recordings. The experiment consisted of two sessions per subject and each session was composed of 60 trials with three kinds of memory loads (two, four, or eight letters). Stimuli were presented and behavioral results were recorded using E-prime software¹.

EEG recording was accomplished by using a 64-channel NeuroScan system (Quik-Cap, band pass: 0.05–100 Hz, sampling rate: 1000 Hz, impedances < 5 kΩ) with a vertex reference. To monitor ocular movements and eye blinks, Electro-Oculogram (EOG) signals were simultaneously recorded from four surface electrodes, one pair placed over the higher and lower eyelid and the other pair placed 1 cm lateral to the outer corner of the left and right orbit.

The data was re-referenced to the infinity reference (IR) (Yao, 2001; Tian and Yao, 2013) using the software REST². In the study, EEG was segmented from 100 ms before the onset of the memory array to 100 ms after the subjects’ response onset. In other words, the retention and retrieval stages of WM were extracted for the following preprocessing. EOG and Electromyography (EMG) were excluded by Blind Source Separation (BSS) (Negro et al., 2016). Other noise was removed by automatic artifact rejection (±100 μν). The data was baseline corrected using 100 ms EEG signal before the memory array onset. Then the EEG recordings were filtered with a band-pass of 0.5–45 Hz. After the above preprocessing, 60 trials were obtained for each subject under three memory loads (two, four, and eight items). The retention stage (3000 ms) of each trial was extracted for subsequent analysis.

When performing time limit tasks incorporating both reaction time (RT) and RACC, subjects either sacrificed accuracy in exchange for response speed, or exchanged response speed for high accuracy (Mordkoff and Egeth, 1993; Breukelen, 2005). Here, we utilized the Signed Residual Time (SRT) scores (Maris and Han, 2012) for speed accuracy trade-off (Schmitt and Scheirer, 1977) to represent subjects’ behavioral performance. The SRT scoring rule was defined as follows (Schmitt and Scheirer, 1977; Maris and Han, 2012; van Rijn and Ali, 2017):

where the subscript i denoted the item index, RACC_i = {0,1} was the response accuracy (RACC: 0 for incorrect and 1 for correct) for a single trial, and MT denoted the maximum allowable response time for item i. t_i denoted response latency. The total scores were simply the sum of the scores in each item. In other words, for a correct response, subjects earned the residual time as score, and for an incorrect response, subjects lost the residual time as score.

For testing the left–right hand effect and memory-load effect on behavioral WM performance, 2 (hand: left vs. right) × 3 (memory load: 2 items vs. 4 items vs. 8 items) repeated measures analysis of variances (ANOVAs) were conducted. The dependent variables were the SRT scores in intra-sessions (session 1, session 2) and the change rates of subjects’ SRT scores in inter-sessions (from session 1 to session 2). Greenhouse–Geisser correction was used when the sphericity assumption was violated in repeated measures ANOVAs (Greenhouse and Geisser, 1959), and a factor had more than two levels in the current study to protect against Type I errors (Kisley et al., 2005). In addition, false discovery rate (FDR) was utilized to the correction of multiple comparisons.

Spectral entropy based on Shannon entropy in physics, quantifying the regularity/randomness of the power spectrum during a given period of time, was used to establish the biomarker for WM performance in the current study. The methodological details were similar to those adopted in a previous study on the prediction of BCI performance (Zhang et al., 2015).

The retention period for each trial was extracted to calculate PSD P_sd(f) via the Welch’s method (Akbar et al., 2016). The PSD of a time series was defined as the distribution of power as a function of frequency. The normalized PSD was defined as the P_sd(f) divided by the total power to obtain probability density function.

where _^Psd(f) was the normalized PSD of P_sd(f). We estimated spectral entropy based on the PSD within 0.5–45 Hz. The entropy of _^Psd(f) was generated by using the following equation:

where k = 1. The base of the logarithm was 10 and the unit of SEn was dit (i.e., Decimal Digit) in the current study (Schneider, 2007). In effect, spectral entropy reflected the uniformity of the power spectrum distribution. The greater the spectral entropy was, the more uniform the power spectral distribution was.

Spearman’s rank correlation coefficient was widely applied to measure the monotonic relationships. It was defined according to the following equation (Gautheir, 2001; Sedgwick, 2014):

where each variable was ordered respectively from lowest to highest. d_i was the difference between two ranks for paired variables x_i and y_i. m was the number of data pairs. In the current study, the relationships between SRT and spectral entropy on each channobserved. Then, we constructed the fingerprint figures and the scalp spatial distribution according to the spearman’s coefficients on each channel (the processing steps as shown in Figure 2). Based on the above method, the electrode site for highest correlation between SRT and spectral entropy was identified to carry out the next steps (see below).

Subjects were divided into two groups according to their standard z scores of SRT. Here, z-score was defined as:

where x_i (i = 1,2,3…,20) was the SRT scores of the ith subject, _μ was the mean of the 20 subjects’ SRT scores and _σ was the standard deviation of SRT scores between 20 subjects in the current study. Subjects whose z values of SRT were above zero were allocated to the group with high WM performance, and the rest were assigned to the group with low WM performance. The high and low memory performance groups were defined as positives and negatives, respectively.

The evaluation of generalization performance for SVM classifier was a required step after using the SVM to classify the high from low memory performance groups. SVM was developed by Vapnik based on statistics learning theory (SLT) (Netherlands, 2008). As a result of its excellent generalization performance, SVM has been applied in a wide variety of issues, such as text classification, images classification, hand writing recognition and gene classification. Furthermore, SVM had the feature of empirical risk minimization (ERM) and global optimum solution (Netherlands, 2008). If the SVM classifier could reflect the relationship between features and the class labels very well, then the classifier was considered that it could predict the classes of new samples with good performance. Therefore, classification accuracy (CA), sensitivity (SE), specificity (SP), and area under ROC curve (AUC) were utilized to evaluate the classification performance of SVM classifier (Galar et al., 2012). At the same time, leave-one-out cross-validation (LOOCV) was applied to evaluate the generalization performance of SVM for a relatively small sample size in the present study.

(1) The percentage of the number of samples predicted correctly in the test set over the total samples, CA, was calculated as follows:

where true positive (TP) was the number of high-performance samples correctly predicted and true negative (TN) was the number of low-performance samples correctly predicted. False positive (FP) denoted the number of high-performance samples incorrectly predicted and false negative (FN) denoted the number of low-performance samples incorrectly predicted.

(2) SE and SP were calculated by the following formulae, respectively:

SE referred to the ratio of correctly classified high-performance samples to the total population of high-performance samples, whereas SP was the ratio of correctly classified low-performance samples to the total population of low-performance samples.

(3) AUC was defined as the area under ROC curves, which was discovered and proved to be better than CA to evaluate the predictive performance of classification learning algorithms (Jin and Ling, 2005). Moreover, AUC was indeed a statistically consistent and more discriminating measure than CA (Ling et al., 2003). Originally, the ROC curves were introduced to evaluate machine learning algorithms (Provost et al., 1997). On ROC curves, TP was plotted on the Y axis and FP was plotted on the X axis. It described the classifiers’ performance across the entire range independent of class distributions (Provost et al., 1997; Jin and Ling, 2005). However, often there was no clear dominating relation between two ROC curves in the entire range. Therefore, AUC was introduced to provide a good “summary” for the performance of the learning algorithms based on ROC.

To examine the stability of the SVM classification performance, the classification was performed respectively under intra- and inter-session conditions. In the intra-session, LOOCV was utilized separately for the first and second sessions (Vehtari et al., 2016). One subject was picked out as a test sample, while the rest of the subjects were regarded as the train samples for obtaining the classification model. The above steps were repeated until every subject has been allocated as a test sample for one time. During the inter-session prediction, subjects in the first session were regarded as test samples and subjects in the second session were regarded as the training samples. Then subjects in the first session were considered as training samples and subjects in the second session were considered as test samples.

The subject’s spectral entropies derived from WM data at the retention period were selected to predict the corresponding SRT scores. According to the Section “Grouping Rules and Classification,” the spectral entropies of the channel achieving the highest correlations between SRT and spectral entropy were applied to SVR to develop predictive model for a relatively small sample size (Ju et al., 2014) respectively in session 1, session 2, and merged session (i.e., session 1 + session 2). Then, LOOCV was utilized to test the stability of the model and the performance of the predictions. Detailed information for the LOOCV procedure for prediction was described as follows:

We assumed that there were n samples in the dataset. One sample was picked out as a testing set, and the rest of the samples were regarded as training sets to develop the predictive model. The above steps were repeated until every subject had been assigned as a test sample for one time and eventually n SVR models were obtained. The correlation coefficients between predicted and actual SRT scores, together with root-mean-square error of prediction (RMSEP), were calculated to evaluate the prediction performance of the SVR model (Ju et al., 2014; Kichonge et al., 2015).

For a single subject, behavioral scores usually varied by training. Thus we attempted to study whether the spectral entropy of WM data at retention stage could predict changes in memory performance before and after training.

Since the subjects’ SRT scores and spectral entropy were in different unit scales, a new measure called Change Rate (CR) was defined as follow (Zhang et al., 2015):

where TB denoted the subjects’ SRT scores or spectral entropy of WM data at retention period before training (i.e., session 1). TA was the subjects’ SRT scores or spectral entropy predictors at retention period after training (i.e., session 2).

For SRT scores, the significant main effects of memory load were separately observed in intra-sessions (session 1: F = 218.99, P < 0.001, ηp2 = 0.920; session 2: F = 162.32, P < 0.001, ηp2 = 0.895), indicating that different memory-load tasks affected the behavioral performance on SRT scores regardless of before training and after training. Non-significant main effects of hand were found in intra-sessions (session 1: F = 0.435, P = 0.518, ηp2 = 0.01; session 2: F = 1.51, P = 0.234, ηp2 = 0.074), indicating that SRT scores were not affected by left–right hand effect in intra-sessions. There were non-significant interactions between hand and memory load (session 1: F = 4.24, P = 0.053, ηp2 = 0.183; session 2: F = 0.92, P = 0.405, ηp2 = 0.046).

For the change rates of SRT scores from session 1 to session 2, there were non-significant main effects of hand (F = 3.636, P = 0.072, ηp2 = 0.161) and memory loads (F = 0.028, P = 0.973, ηp2 = 0.002), indicating that the change rates reduced by training were not affected by both hand and memory-load effects. There were non-significant interactions between hand and memory load (F = 3.99, P = 0.060, ηp2 = 0.169).

The correlations between SRT scores and the spectral entropy on 64 channels were illustrated in the fingerprint map (Figure 3A) and the scalp topographic map (Figure 3B). Among them, the spectral entropy on channel FC4 showed the strongest correlation with subjects’ SRT scores (session 1: r = 0.814, P < 0.001; session 2: r = 0.761, P < 0.001; and merged session: r = 0.698, P < 0.001; FDR correction, and also shown in Figure 3C).

Therefore, the spectral entropies on channel FC4 from WM EEG recording at the retention stage were then used as features for SVM classifiers to distinguish the high-performance group from the low-performance group, and further used to predict subjects’ SRT scores with SVR prediction model.

In the first session (i.e., before training), 11 subjects were divided into high-performance group and nine subjects were divided into low-performance group; while for the second session (i.e., after training), 12 of 20 subjects were assigned to high-performance group and eight subjects were assigned to low-performance group (Table 1).

The CA was 95 and 85% for the first and second session with spectral entropy features, respectively. The sensitivity (SE) and specificity (SP) at the optimal operating point, as well as the resulting AUC were showed for the first and second session, respectively. The grouping results were showed in Table 2.

For intra-session prediction, the resulting CA, AUC, SE, and SP were respectively 0.950, 0.976, 1.000, and 0.900 for session 1 with spectral entropy as classification feature. CA, AUC, SE, and SP were respectively 0.850, 0.918, 0.800, and 0.900 for session 2 with spectral entropy as classification feature.

For inter-session classification, CA, AUC, SE, and SP were respectively 0.950, 0.973, 0.898, and 1.000 for session 1 with session 2 as training samples. The resulting CA, AUC, SE, and SP were respectively 0.900, 0.942, 0.900, and 0.900 for session 2 with session 1 as training samples.

The classification results were illustrated in Figure 4 and Table 2. The red line demonstrated the ROC curves of session 1, and the blue line showed the ROC of session 2.

Support vector regression prediction model and LOOCV revealed that the spectral entropy could be utilized to predict individual WM performance on SRT scores in the current study. The RMSEP after doing LOOCV were 4.635 (session 1), 3.339 (session 2), and 6.972 (merged session), respectively. As illustrated in Figure 5, the predicting SRT scores were significantly correlated with the original SRT scores (session 1: r = 0.749, P < 0.001; session 2: r = 0.864, P < 0.001; merged session: r = 0.732, P < 0.001; FDR correction).

Within single subject, we explored whether the increased (or decreased) changes in spectral entropy could be predictive of the increased (or decreased) changes in SRT scores. Figure 6A showed that 16 in 20 subjects’ SRT scores increased by short-time training. Figure 6B showed that 15 in 20 subjects’ SRT scores consistently varied with spectral entropy predictor before and after training. The results also could be seen in Table 1.

As shown in Figure 3B, the spatial distribution of the r-values (i.e., the relationship between spectral entropies and behavioral scores) focused on the right frontal area, which was consistent with the previous findings that the frontal cortex might play a prominent part in WM tasks (Gentili et al., 2015; Thürer et al., 2016). The previous study showed that right inferior frontal junction (rIFJ) as a prefrontal cortex (PFC) control region mediated the causal connection between top–down modulation in the service of attentional goals and WM performance (Gazzaley and Nobre, 2012). Among these channels (Figure 3A), the spectral entropy of channel FC4 from WM at retention period generated the strongest correlation (r = 0.698) in comparison to other channels in the merged session. Likewise, in the two separate sessions, the highest correlation coefficients also appeared on channel FC4 before training (r = 0.814) and after training (r = 0.794), respectively. The good classification effect of SVM showed that the spectral entropies on channel FC4 from WM EEG recording at the retention stage might be a dependable biomarker to classify two memory groups successfully.

In the current study, there was no significant effect on the change rates of subjects’ SRT scores before and after training when different WM items (i.e., two, four, and eight) were loaded. Thus, subjects’ SRT scores were utilized to reflect individual WM performance while carrying out the WM tasks regardless of memory load. As illustrated by Figure 3C, the spectral entropy indexes of the subjects were significantly related to individual SRT scores in all sessions including separate sessions and merged session, indicating that spectral entropy indexes might be applied to predict individual WM performance (i.e., SRT scores).

For the further analysis, SVR prediction model combined with LOOCV was established to estimate the predictive ability of spectral entropy indexes on WM performance separately in session 1, session 2, and the merged session (Figure 5). The resultant RMSEP after using LOOCV, as well as the high correlation between original SRT scores and predicted SRT scores, demonstrated that prediction models constructed by SVR were effective. The spectral entropy obtained from channel FC4 could be a biomarker to predict individual WM performance.

Converging evidence revealed that one’s memory ability was absolutely not innate and could be promoted through proper training (Klingberg et al., 2005; Holmes et al., 2010; Klingberg, 2010). Moreover, this improved performance was related to training-induced plasticity from the intracellular level to functional organization of the cortex for WM (Klingberg, 2010). Vogt et al. (2009) conducted a WM training study on multiple sclerosis. They found that the patient’s memory was improved effectively and the spread of bad mood was delayed after receiving the special training for memory (Vogt et al., 2009).

As illustrated by Figure 6A and Table 1, 16 subjects’ behavioral scores were obviously increased after training, indicating that individuals’ WM performance could be promoted effectively through training, consistent with the previous study (Klingberg et al., 2005; Holmes et al., 2010; Klingberg, 2010). The CR of 15 in 20 subjects’ SRT scores increased (or declined) with the increase (or decline) of the spectral entropy before and after training, which demonstrated that the variations in the spectral entropy predictor could be predictive of the WM performance variations. The findings revealed the consistent changes in SRT scores and spectral entropy by training (Figure 6B and Table 1).

Previously, P300 in motor imagery tasks and steady state evoked visual potentials (SSVEP) were frequently applied in BCIs (Parra et al., 2003; Dal et al., 2009; George and Lécuyer, 2010). Recently, BCIs also were designed to recognize human emotions (Garcia-Molina et al., 2013; Chanel and Mühl, 2015) and to monitor WM load (Sánchez et al., 2015), while there are still little research on the detection of WM performance or individual memory in BCIs. Consequently, it is of great significance to find a reliable feature for the monitor of individual WM performance in BCIs. The development of predictors on WM performance could recognize the potential memory impairment subjects, assisting them in preparation for the positive life and avoiding the frustration from disordered memory. On the other hand, the relevant study might in turn be heuristic for making effective training strategies for those subjects with low WM performance. Moreover, spectral entropy could be a potential instructive biomarker applied in the detection of schizophrenia, depression, ADHD and two-way affective disorder patients and meanwhile provide a new thought for BCIs on the feature extraction of WM.

As shown in Figure 6A, 16 in 20 subjects’ SRT scores obviously was improved after training, while there were still four subjects’ SRT scores representing a downward trend which might be induced by individuals’ state: either their mental state such as easy to feel tense when carrying out the unfamiliar WM tasks or staying unsuitable to experimental environment, or their physical state such as exhaustion without rest well before experiment or easy to be tired when doing the “dull” experiment. For small sample size in the current study, we just utilized LOOCV and four different evaluation indexes of the classifier’s generalization to avoid overfitting as much as possible. It is noted that extracting classification features (like channels) outside of the LOOCV method still could induce overestimation problems. Therefore, we verified the robustness of the selected channel. And the correlation between spectral entropies and WM-related SRT scores of all subjects was highest on channel FC4 regardless of session 1, session 2 and merged session, suggesting the relationship was robust. In addition, spectral entropy was used to predict the individual SRT scores thus indirectly reflecting the WM performance. In the future, a direct way represented the online WM-related BCI would be explored by optimizing classifiers, expanding sample size and improving experimental paradigm.