Thirteen able-bodied participants were recruited for this study. Participants had normal or corrected to normal vision, and no reported history of neurological, metabolic, respiratory, cardiovascular, or drug/alcohol-related conditions. One participant was excluded after the first session due to the inability to accurately describe the study protocol. A second participant was excluded upon disclosing post-study, a medical history that violated inclusion criteria. A third participant was excluded due to inadequate transtemporal windows, which precluded the location of the MCAs. The ten remaining participants included for study (aged 18–40 years; all female), were all right-handed. All participants provided written informed consent. This study was approved by the research ethics boards of both Holland Bloorview Kids Rehabilitation Hospital and the University of Toronto.

The Doppler spectra of blood flow velocities through the left and right MCAs were monitored using the MultiDop X-4 TCD (Compumedics Germany) and the accompanying bilateral headgear with two fixed 2 MHz ultrasonic transducers. The data were recorded at a sampling frequency of 100 Hz. The probes were positioned over the transtemporal insonation window according to an established insonation procedure (Alexandrov et al., 2007) as seen in Figure 1.

Ultrasound gel was applied between the probe and the user's skin to ensure proper signal transduction. Once the probe was placed over the transtemporal window, the TCD was turned on with an initial depth setting of 50 mm. The insonation angle and depth were then adjusted to find the bifurcation of the internal carotid artery into the middle cerebral artery (blood flowing toward the probe) and the anterior cerebral artery (blood flowing away from the probe). The insonation depth was then decreased until the maximum unidirectional flow toward the probe was detected. All participants were given 5 min breaks per every 15 min of TCD usage to provide sufficient time for probe cooling. Throughout the recording process, the thermal cranial index (TIC) of the probes did not exceed 1.5, thus avoiding discomfort or thermal injury to the participants, which is in accordance with the British Medical Ultrasound Society safety guidelines (Group, 2010). The TCD device (MultiDop X-4) was approved by Health Canada's Medical Devices Directorate for investigational testing.

Participants performed two mental tasks (i.e., activation and rest) throughout the study. Mental spelling accompanied by imagined writing of each letter with the right hand was used as the activation task, with the intent of eliciting left-lateralized brain activity. To restore CBFV to non-lateralized basal levels, visual tracking of a time-evolving strip chart of left and right mean CBFV (Figure 2) was used as the rest task. The participant performed mental spelling throughout each 15 s activation period and the visual tracking task throughout each 15 s rest period.

During the training session, participants were presented with either a single letter or multiple letters forming part of a word. Upon seeing this cue, participants were instructed to repetitively rehearse the spelling of the desired word while simultaneously imagining the writing of the word with their right hand. Likewise, participants were instructed to shift their gaze to the TCD feedback signal whenever an hourglass appeared on the screen. Both tasks were completed without any vocalization to avoid an increase in blood flow due to speech.

In the testing sessions, the participants used the TCD BCI to spell target words online. During these sessions, the participants were asked to perform the activation mental task when the desired letter appeared among the currently available letter choices and to perform the rest mental task when the desired letter was not displayed.

A custom on-screen keyboard was developed based on the concept of the dynamic keyboard developed by the University of Victoria. In our implementation, each level of the keyboard hierarchy contained multiple bins, although at any given time, only one bin was displayed to minimize mental workload and user confusion. Each bin contained multiple letters or words. Figure 2 depicts the user interface of the dynamic on-screen keyboard. The dynamic keyboard behaviors were governed by the following operating principles.

Figure 3 portrays an example of dynamic keyboard progression. For simplicity, only a subset of paths is shown. Here, the bin “t, a, s, i, o” is selected. Bypassing the “undo” bin confirms the selection. The first bin on the next level of the hierarchy contains the highest frequency words starting with one of “t, a, s, i, o.” Here, this word bin is bypassed, triggering the presentation of individual letters from the previous level of the hierarchy. The letter “t” is chosen and confirmed (bypassing undo), prompting the presentation of high frequency words starting with “t.” In this example, this word bin is selected and confirmed, resulting in the presentation of the individual words from this bin.

Each participant completed three sessions. At the beginning of the first session, each participant was given an information sheet, highlighting the nature of each task. In addition, prior to each session, participants also received verbal instruction about how to perform the activation and rest tasks. The first session involved two training blocks and one testing block while subsequent sessions contained one training block followed by two testing blocks. A one minute baseline recoding was obtained before each block for the purpose of normalizing data collected from the block. During baseline, participants performed the rest task. A five minute rest period was offered between blocks.

For each training block, the participants performed a total of forty task segments. Each segment was either an activation or rest task. The sequence of task presentation was randomized (Figure 4). A 10 s recovery period was included after each activation task to allow the participant's blood flow velocities to return to baseline levels. During the recovery period, the participants performed the rest task to restore basal blood flow velocities. In the first session, participants had a 10 min break while the two blocks of training data were used to train the appropriate classifier. For sessions two and three, the 10 min break occurred after the first training block. During this break, the classifier was trained with data from the current and initial sessions. After each session, the participants' level of fatigue was ascertained via a written survey.

For each testing block, the participants were asked to spell a given target phrase to the best of their abilities using the dynamic keyboard. The participants performed the activation task only when the bin containing the intended selection was presented. If a false positive occurred, the participants were instructed to select the undo button and correct the error before continuing the spelling process. If a false-negative occurred, the participants were instructed to simply wait until the keyboard looped back to the intended bin.

All data collected from the training blocks were used for classifier training. Therefore, for each participant, a total of forty activation data segments and forty rest data segments were used to train a user-specific classifier in session I. For each subsequent session, training data from session I (40 activation and 40 rest segments) and the session at hand (20 activation and 20 rest segments) were pooled for training (i.e., 60 activation and 60 rest data segments). Each segment was 15 s in duration. A total of forty-four features were extracted from each segment. Twenty four features were based on the left and right CBFV signal mean, slope, standard deviation, and entropy over the following intervals: 0–5, 5–10, and 10–15 s. Six features were extracted from the differences in mean and slope between the left and right signals over the same time intervals. Nine features were extracted based on the correlation, dot product, and mutual information between the left and right signals over the aforementioned time intervals. The last five features were extracted from left and right CBFV signal standard deviation and entropy from 0 to 15 s and the mutual information between the left and right CBFV signals over the 0 to 15 s interval. An example feature computation is presented in Figure 5. These features were chosen according to the findings of previous TCD brain lateralization studies (Myrden et al., 2011).

Weighted sequential feature selection (WSFS) was used to algorithmically select three to five features for each session, for each participant, to train a Naïve Bayes classifier. WSFS extended the sequential forward search (SFS) approach (Mamun et al., 2012) by explicitly considering feature contributions (i.e., the number of times a specific feature was chosen). For each fold of a 10-fold cross-validation for feature selection, all features were first ranked according to F-score (Duda et al., 2012) for interclass separability, using the training set. These ranked features were then organized into cumulative subsets such that the first subset contained the top ranked feature, the second subset contained the top two ranked features, and so on. The last subset contained all features. Within each fold, the subset with the highest validation accuracy was selected. Therefore, 10-fold cross-validation yielded 10 such subsets.

We enumerated the occurrence of each feature within these 10 subsets. Certain features appeared consistently across all subsets while others surfaced intermittently. The selected features were regrouped based on their frequency of occurrence, such that the mth group contained all the features that appeared at least m times, where m = {1, 2,…,10}. These new subsets were evaluated through a subsequent constrained 10-fold cross-validation (i.e., only the m pre-determined feature subsets were cross-validated) with newly randomized testing and training sets. The final set of features was then selected as that with the highest average validation accuracy. The chosen features were used to train a Gaussian Naïve Bayes classifier.

Figure 5 demonstrates a single trial of a rest task (left) and activation task (right). The three most common features selected across sessions and participants are highlighted. The least common of the three features (slope of the right MCA CBFV) was selected in six out of ten participants.

To capture the different nuances of online classification performance, several metrics were invoked as suggested by Thomas et al. (2013) and Schlögl et al. (2007). To gauge the correctness of classification for a biased classifier (i.e., unequal performance for each class), sensitivity, and specificity were estimated from the confusion matrix Schlögl et al. (2007). Specificity is the number of true negatives divided by the actual number of negatives in the test set while sensitivity is the number of true positives divided by the actual number positives in the test set.

To measure the agreement between the predicted and desired selections (Cohen, 1960; Thomas et al., 2013) in the presence of unbalanced data (i.e., unequal number of samples per class due to the nature of the experiment), Cohen's kappa (κ) coefficient was estimated. Kappa ranges from 1 (perfect match) to 0 (chance level). If all values of κ within the 95% confidence interval around the mean are above 0 (κ ± 1.96 × φ(κ) > 0, where φ(k) is the standard error), then the average kappa value is significantly above chance (Friedrich et al., 2012). The classification accuracy ACC (overall agreement) was derived from the 2×2 confusion matrix H, as (1)ACC=p0=∑iHiiN where H_ii are the main diagonal elements (i.e., number of correct classifications) of the confusion matrix H and N = ∑_i∑_jH_ij is the total number of trials. The chance expected agreement p_e, is the probability of observing the current confusion matrix and is given by, (2)pe=∑in+ini+N2 where n_+i and n_i+ are the marginal column and row sums, respectively. The estimate of the kappa coefficient κ is thus, (3)κ=p0−pe1−pe while its standard error φ(κ) is given by, (4)φ(κ)=p0+pe2−∑i[n+ini+(n+i+ni+)]/N3(1−pe)N

This method of evaluation is preferred for problems with unbalanced classes (Danker-Hopfe et al., 2004; Anderer et al., 2005), such as sleep classification.

To gauge performance of the system as a communication channel, we estimated the Nykopp information transfer rate (ITR), which is recommended for classification problems with unbalanced class sizes (Thomas et al., 2013). Letting x_i represent the actual input category (x₀ = rest, x₁ = activation) and y_j represent the predicted output (y₀ = rest, y₁ = activation), the ITR was given by (5)ITRNykopp=∑i = 01∑j=01p(xi)p(yj xi)log2[p(yj xi) ] where (6)p(yj)=∑i=01p(xi)p(yj xi) (7)p(yj xi)=Hijni+ while p(x₀) = 0.7 and p(x₁) = 0.3 are the prior probabilities of rest and activation tasks, respectively to be, estimated from the average frequency of occurrence of each task when spelling an intended message with no mistakes. To calculate the bit-rate, we multiplied the Nykopp ITR by the average number of trials per min (Thomas et al., 2013).

To assess system efficiency, the average throughput, defined as the number of characters output per min, was determined. Only correct characters were counted while the measured duration included the time required to make error corrections. Since participants were asked to correct mistakes during the spelling process, the estimated throughputs were generally conservative with the low char/min.

To measure the resemblance of the actual output to the intended output, the Levenshtein or edit distance was calculated. The edit distance compares the similarity between two strings of unequal length and is defined as the number of editorial operations required to convert the actual output into the intended output (Sankoff and Kruskal, 1993). Each deletion and insertion of a character was given a weight of 1 while a substitution was given a weight of 2, being equivalent to a deletion followed by an insertion (Sankoff and Kruskal, 1993). Since the intended outputs were of different lengths for the testing blocks of the three sessions, the edit distances were normalized based on the longest string length of the intended outputs (Equation 8). Other normalization methods more severely penalize a lack of input over an incorrect selection (Marzal and Vidal, 1993; Weigel and Fein, 1994; Li and Liu, 2007). However, due to the study design, an incorrect selection should have a higher edit distance than a lack of input since the effort required to correct an incorrect selection is far greater than that needed to produce an intended output with no corrections. The normalized edit distance, D_EN, is given by, (8)DEN=DE|X|×|X*| where |X| is the length of the intended output, D_E is the raw edit distance between intended and actual output, and |X^*| is the length of the longest intended output from all sessions. Given the longest string length in our experiment was 18, i.e., |X^*| = 18, a normalized edit distance of 18 indicated no output and 0 meant perfect match between the intended output and the actual output. Any score above 18 indicated that the actual output mismatched the intended output. The larger the normalized edit distance is, the further away the actual output was from the intended output.

Bilateral features were more frequently selected (Figure 6), which could be due to the left-lateralized nature of the language task. The higher selection frequency of bilateral features was consistent with that reported in a previous offline TCD-BCI study using verbal fluency (Myrden et al., 2011). Therefore, our modified verbal fluency task (i.e., rehearsing the spelling while imagining the writing of the target word) appeared to elicit machine-discernible left-hemispheric lateralization.

The online performance on the testing blocks in sessions II and III is reported in Table 1. The system achieved an average specificity and sensitivity of 81.44 ± 8.35 and 82.30 ± 7.39% respectively, resulting in an average kappa coefficient of 0.60 ± 0.03. All participants exhibited a kappa coefficient that exceeded chance. Seven out of eight participants achieved a kappa coefficient over 0.4, which is equivalent to an accuracy >70% had the classes been balanced (Friedrich et al., 2012).

Classification performance across all sessions is summarized in Table 1. Only four out of ten participants were able to achieve above chance level kappa coefficient [κ ± 1.96 × φ(κ) > 0]. Of the four participants, three were able to achieve a moderate agreement within the first session (> 0.4). For sessions II and III, all participants achieved accuracies above chance. Moderate agreement between intended and predicted selections (> 0.4) was achieved in nine out of ten participants.

The throughputs for all three sessions for all participants are shown in Figure 7. The average throughput for session I, II, and III across participants were 0.04 ± 0.05, 0.30 ± 0.14, and 0.32 ± 0.10 characters/minute, respectively.

Figure 8 depicts the edit distances for each session. Using a paired t-test, we compared the edit distances for the 10 participants at a rigorous significance level of 0.01. In the session I test block, there was no significant difference between edit distances for the no output case (|X^*| = 18) against distances when something was spelled (p = 0.619). In other words, the composed output was distant from the target output string. In session II, testing blocks 1 and 2 showed significant reduction in edit distances below that achieved in session I (p = 0.001; p = 0.005), though there was no significant difference between the edit distances of the two blocks (p = 0.019). In session III, testing blocks 1 and 2 again showed significant improvement over session I edit distances (p = 0.001; p < 0.001). In addition, there was no significant difference between edit distances of the two testing blocks in session III (p = 0.790). Finally, there was no significant difference between edit distances from the corresponding blocks of sessions II and III (p ≥ 0.114).

The correlation between tiredness levels and performance of all sessions were ascertained through Spearman's coefficient (r_s) (Brown and Hollander, 1977). There were no significant correlations between the tiredness levels and edit space or throughput. In addition, there were no significant correlations between tiredness and specificity or sensitivity. However, there was a negative trend on the tiredness of the participant before the session and the specificity of the testing blocks (r_s = −0.314, n = 20, p = 0.177).

The throughputs for sessions two and three improved beyond those of session one, approaching transmission rates of established BCI spelling devices (0.5 char/min) (Birbaumer et al., 1999). The observed combination of low throughput (Figure 7) and high kappa coefficient (Table 1) can be attributed to the cost (temporal penalty) of a false-negative. If a bin was unintentionally bypassed, the participant had to wait between 4 and 15 additional slides before the target bin would be presented again. This wait time can translate into a temporal penalty of several minutes for missing a selection, and is an inherent limitation of scanning keyboards. Additional practice may help to decrease response latency. Further, the Dynamic Keyboard interface could also be improved (e.g., context-specific word prediction) to enhance the speed and accuracy of letter/word selection.

Some features were consistently selected across all participants. For most participants, the left lateralization of the mental task was pronounced. This finding confirms previous reports of left hemispheric lateralization accompanying verbal fluency tasks (Vingerhoets and Stroobant, 1999; Myrden et al., 2011). Due to the inherent lateralization, bilateral features were selected more frequently, as shown in Figure 6, particularly those corresponding to differences between the mean velocities of the left and right MCAs. Nine out of ten participants had the two most frequent features (i.e., difference of MCA means at 5–10 and 10–15 s) selected at least in one session. The difference in means between 10 and 15 s was the most frequently selected feature, followed by the difference in means between 5 and 10 s. The feature representing the difference in the means between 0 and 5 s was seldom selected. This is likely due to the inherent 5–10 s hemodynamic delay post-mental activation (Harders et al., 1989; Szirmai et al., 2005).

All participants exhibited improved upon their session I performance in the latter 2 sessions. This improvement is attributable in part to the increase of training data available to the classifier. In addition, participants may have also become more familiar and comfortable with the study protocol and the user-interface. A longitudinal study of TCD-based BCI may help elucidate the effect of mental practice on functional performance.

Other factors (e.g., fatigue, extended trial duration or head motion) may have also impacted participant performance. For example, participant 4 reported a lack of concentration and physical fatigue, which may explain the lower accuracies for this individual.

The TCD-BCI was able to achieve an average bit-rate of 0.87 bits/min and a maximum of 1.53 bits/min. If the post-activation task 10 s recovery time was removed, the average bit-rate would improve to 1.10 bits/min. In addition, if we are able to bring a three-task TCD into an online setting, similar to the offline study by Aleem and Chau (2013), assuming equal priors, we can further increase the bit-rate to 4.38 bits/min. Due to a lack of published online TCD-BCIs at present, we compare our results to those of other hemodynamic BCIs. Recent fMRI BCI studies using two-task algorithm attained an average of 2 bits/min (~80% accuracy). Other fMRI BCI studies with a four-task algorithm attained bit rates between 0.9 and 1.5 bits/min (~90% accuracies) (Yoo et al., 2004; LaConte et al., 2007; Minati et al., 2012). Thus, our system achieved a comparable bit-rate with a much simpler set-up. At present, EEG-BCIs still offer the most compelling bit-rates, typically in the order of 15–30 bits/min (Donchin and Arbel, 2009; Kansaku et al., 2010).

The average throughput for session I was not significantly different from 0 characters/min at a significance level of 0.01 (p = 0.022), which could have been due to the lowered specificity and sensitivity across participants in session I. Throughput for sessions II and III were significantly different from that of session I (p = 0.001; p < 0.001) and from 0 characters/minute (p < 0.001; p < 0.001). This improvement may be due, in part, to the user's increasing familiarity with the keyboard, facilitating more skilled navigation through the user interface. Given the temporal resolution of TCD and the sequential nature of the dynamic keyboard, the throughput may have approached its theoretical limit at 0.3 characters/min. Without modification of the user interface and the temporal window of data acquisition, further improvement of the throughput might not be possible. Incidentally, the change in throughput from session II to III was not significant (p = 0.653), but this does not preclude further improvements over extended periods of practice.

Similar to throughput, the edit distances for both session II and III improved significantly beyond session I values. Within sessions II and III, the edit distances for the actual outputs did not differ significantly. This suggested that the duration of TCD usage did not affect the quality of the output as testing block 2 typically occurred an hour after initial TCD set-up. Therefore, prolonged TCD usage may be possible provided that breaks are provided every 15~20 min.

Feedback regarding the performance of the online TCD-BCI system was neutral to positive (except for the first sessions for participants 2 and 5). Participants 4 and 8 both indicated that they were “somewhat tired” prior to and “very tired” after every session. The lack of energy prior to the session may have impacted participant performance with the online TCD-BCI. The live feedback may have further frustrated the participants, exacerbating their fatigue and diminishing their concentration, thus forming a negative feedback loop that further impacted performance. However, the lack of significant overall correlation between tiredness levels and performance in terms of specificity, sensitivity, edit distance, and throughput suggest that user perceived fatigue did not directly impact overall user performance.

The inefficiency of the scanning keyboard undoubtedly constrained the observed BCI accuracies. Scanning keyboards are frequently used as an interface for assistive technology devices (Jans and Clark, 1994; Lesher et al., 1998). However, the existing keyboard interface was prone to long delays in the event of incorrect selections. For example, for individuals who achieved high accuracies (>85%), it was still difficult to spell the intended phrase within the allotted time. Further improvement of the Dynamic Keyboard is necessary to achieve more efficient communication in future studies. In addition, the required periodic cooling of the TCD probes introduces further delays in communication. Future improvements in TCD technology may minimize the required duration of probe cooling.

One of the major determinants for participant performance was their motivation and concentration. For participants who reported fatigue during specific sessions, (e.g., session 3, participant 4), the overall accuracy rates were lower compared to those of other participants. For participants who maintained concentration during the testing session, on the other hand, higher accuracies were observed (e.g., participant 7 and participant 9).

Despite the efforts to precisely locate the MCAs, unbalanced left and right CBFV magnitudes were occasionally observed. Probe placement errors may contribute to lower accuracies. Future TCD-BCI studies should endeavor to place the probes flush against the skin overlying the temporal bone and establish the same physiological insonation depth and sampling volume on either side of the head.

Another potential source of signal contamination for functional TCD studies is motion artifact. Conspicuous facial movements may shift the TCD probes, resulting in momentary or continuous deterioration of the recorded signals. Additionally, extensive body movements e.g., swinging of the arms, crossing and uncrossing the legs, and shifting body in the chair) may also introduce CBFV changes unrelated to the mental tasks at hand. Moderate movements (e.g., moving hands, and shifting feet) were observed in many participants during this experiment. However, the high classification accuracies achieved suggest a level of robustness to these motion artifacts.

Compared to EEG, TCD is robust to electrical artifacts but, like near infrared spectroscopy, subject to long hemodynamic time constants which are several orders of magnitude greater than their corresponding electrical counterparts. However, unlike near-infrared spectroscopy BCIs, TCD is immune to ambient lighting. Based on these relative merits and the findings reported herein, TCD may fulfill a niche need where users possess sufficient literacy skills to do mental spelling, but may be unable to use electrical and optical alternatives, due for example to excessive myogenic noise or light absorption by dark hair. By addressing the aforementioned limitations, a TCD-BCI may eventually provide a means of bedside communication to non-verbal individuals who have severe motor impairments.

Future research will however need to go beyond the controlled, distraction-free laboratory conditions of the present study to gauge feasibility in realistic environments such as the home or inpatient unit. Further, future work must engage clients with physical disabilities to ascertain tolerance for the instrumentation and feasibility of the task paradigm.

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.