The protocol for this review was registered and published in the International Prospective Register of Systematic Reviews (PROSPERO) with identification number CRD42020161039.

The Population, Intervention, Comparison, and Outcome (PICO) strategy of the Joanna Briggs Institute (JBI; Aromataris and Munn, 2017) was the basis for this systematic literature review. The main objective was to synthesize the effectiveness of cognitive training in individuals with SUD when there are improvements in memory, executive functioning, and processing speed. The research strategy aimed to identify published studies, as well as unpublished studies, written in English, Portuguese, and Spanish, from 1985 to 2019. The selected period was based on the first found article referring to cognitive training in individuals with SUD (Godfrey et al., 1985). It was also intended to include gray literature to limit the bias of the present review.

Initially, a general search was carried out in the JBI Database of Systematic Reviews and Implementation Reports, the Cochrane Database of Systematic Reviews, MEDLINE, Epistemonikos, and PROSPERO to confirm the absence of other systematic literature reviews with the same objectives as the present review. Subsequently, an exhaustive and limited search in four databases was performed, including PubMed, the Cochrane Library, Web of Science, and PsycINFO. Then, the titles were analyzed and the articles were summarized using the search terms.

The search terms originated from DeCS® and Medical Subject Headings (MeSH Browser®). These were also combined with the Boolean operators, as well as with the elements of the PICO strategy. Below are the keywords used in the search: Substance-Related Disorders, Addiction Medicine, Alcoholism, Alcohol Abuse, Alcohol Dependence, Substance Abuse, Addiction Treatment, Drug Abusers, Drug Abuse, Cocaine Abusers, Cocaine Dependent, Cocaine-Related Disorders, Marijuana Abuse, Opioid-Related Disorders, Amphetamine-Related Disorders, Hallucinogens, Substance-Use Disorders, Problem Substance Use, Drug Dependence, Cognitive Stimulation Program, Cognitive Stimulation, Cognitive Rehabilitation, Cognitive Training, Memory Training, Cognitive Intervention, Brain Training, Executive Training, Neurocognitive Training, Reasoning Training, Mental Training.

The Boolean operators were arranged as follows:

(Substance-Related Disorders OR Addiction Medicine OR Alcoholism OR Alcohol Abuse OR Alcohol Dependence OR Substance Abuse OR Addiction Treatment OR Drug Abusers OR Drug Abuse OR Cocaine Abusers OR Cocaine Dependent OR Cocaine-Related Disorders OR Marijuana Abuse OR Opioid-Related Disorders OR Amphetamine-Related Disorders OR Hallucinogens OR Substance-Use Disorders OR Problem Substance Use OR Drug Dependence) AND (Cognitive Stimulation Program OR Cognitive Stimulation OR Cognitive Rehabilitation OR Cognitive Training OR Memory Training OR Cognitive Intervention OR Brain Training OR Executive Training OR Neurocognitive Training OR Reasoning Training OR Mental Training). This survey was conducted between May and 31st of July 2020.

Lastly, the references of all selected studies were analyzed for the possibility of including new studies. The articles resulting from the bibliographic search, organized according to the steps previously described, were analyzed by two reviewers. First, the titles and abstracts of studies that could possibly be eligible for the literature review were evaluated, followed by the analysis of the full article.

The identified articles were independently evaluated by two reviewers, using the standardized JBI instruments. In this context, we used the JBI Critical Appraisal Checklist for Randomized Controlled Trials for randomized controlled trials and the JBI Critical Appraisal Checklist for Quasi-Experimental Studies (non-randomized experimental studies) for quasi-experimental studies (Tufanaru et al., 2017).

To assess the quality of a study, namely the risk of bias, we used the Cochrane risk-of-bias tool for the randomized controlled trials (Higgins et al., 2011). This checklist allowed us to perform a complete assessment of risk of bias that may affect the cumulative evidence of the review. Six bias domains were examined: selection, performance, detection, attrition, reporting, and other biases. The studies were classified as “unclear risk,” “low risk,” and “high risk” in each of the above domains. In turn, for the non-randomized studies, the Risk of Bias in Non-randomized Studies - of Interventions (ROBINS-I) tool was used (Sterne et al., 2016). The following domains were analyzed: baseline confounding, selection of participants, classification of intervention, deviation from intended intervention, missing data, measurement of outcomes, and selection of reported results. In this case, each study in question was classified as “low risk of bias,” “moderate risk of bias,” “serious risk of bias,” “critical risk of bias,” and “no information.”

In situations where the reviewers did not reach a consensus on the inclusion or exclusion of a study, a third reviewer intervened. All studies that met the inclusion criteria are included in this review, and any methodological weaknesses present in the selected studies are also discussed.

Data were extracted considering the Cochrane Handbook for Systematic Reviews (Li et al., 2020). Analysis considered the following items:

The data were extracted by two independent reviewers (TC; ER).

Due to the heterogeneity of the data, no meta-analysis was performed. Therefore, a narrative approach was used for data synthesis. There were significant differences between interventions, populations, comparators, and the presentation of outcome results, and thus it was not possible to make a direct comparison regarding the study results. Since statistical pooling was not viable, it was then decided to use tabular and narrative formats to present the results.

The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) flowchart (see Figure 1) shows the studies included and excluded from the present review. Through the research strategies identified above, a total of 467 studies were obtained (54 in PubMed, 124 in Web of Science, 100 in the Cochrane Library, and 189 in PsycINFO) and three studies using other research sources. After the elimination of duplicates, 319 studies remained for analysis. To determine the eligibility of the studies according to the inclusion criteria, their titles and abstracts were analyzed. Fifty studies were considered based on the eligibility criteria; they were analyzed in full by two reviewers (TC; ER). In case of discrepancies, a third reviewer intervened (TA). After this analysis, 24 studies were excluded (see Supplementary Material) and 26 studies met all inclusion criteria. Of the 26 studies included, 25 are controlled randomized clinical studies (Godfrey and Knight, 1985; Godfrey et al., 1985; Yohman et al., 1988; Wetzig and Hardin, 1990; Fals-Stewart and Lucente, 1994; Steingass et al., 1994; Peterson et al., 2002; Goldstein et al., 2005; Fals-Stewart and Lam, 2010; Rupp et al., 2012; Gamito et al., 2013, 2014, 2016, 2017; Eack et al., 2015; Rass et al., 2015; Bell et al., 2016, 2017; Hendershot et al., 2018; Zhu et al., 2018; Khemiri et al., 2019; Rezapour et al., 2019) and one is quasi-experimental (Hannon et al., 1989). The PRISMA guidelines were used to conduct this systematic literature review.

Table 1 summarizes the characteristics of the participants from the 26 included studies. The sample sizes ranged from 12 to 160 participants (with an average of 56.5 participants).

The characteristics of the studies (randomization, blindness, control group, and outcome measures) are provided in Table 2.

The intervention characteristics (programs, cognitive functions, follow-up, total number of sessions, duration and number of sessions per week, and difficulty level and type of training) are provided in Table 3.

Table 4 describes the main results for the 26 studies included in the review. In 16 of the 26 studies, the authors reported clear significant cognitive improvements in individuals who received cognitive training (Wetzig and Hardin, 1990; Fals-Stewart and Lucente, 1994; Steingass et al., 1994; Goldstein et al., 2005; Fals-Stewart and Lam, 2010; Rupp et al., 2012; Gamito et al., 2013, 2014, 2016, 2017; Bell et al., 2016, 2017; Gunn et al., 2018; Zhu et al., 2018; Khemiri et al., 2019; Rezapour et al., 2019). In another two studies, the authors reported marginally significant cognitive improvements (Yohman et al., 1988; Hendershot et al., 2018). From the total 18 studies that found some kind of cognitive improvement, two (Yohman et al., 1988; Khemiri et al., 2019), reported that they were not found in all the assessed cognitive functions. Yohman et al. (1988) reported that although the memory-training group showed no significant improvements in memory tests, the problem-solving group showed marginally significant improvements in problem-solving tests. Khemiri et al. (2019) indicated that there was no enhancement in visuospatial WM however there was a significant increase in the verbal WM ability.

Two studies presented somewhat ambiguous results. One study (Eack et al., 2015) reported significant improvement in neurocognition, but described the differences in the areas where the biggest changes were found (processing speed and verbal learning) as failing traditional significant thresholds. Another (Godfrey et al., 1985), reported significant improvements in memory functioning for both the training and active control groups, without presenting data on the statistical comparison between them.

One study (Rass et al., 2015) discriminated results regarding similar and dissimilar measures to the training tasks, reporting improvements in some measures of WM (visuospatial WM and digit span) similar to the training tasks, although no improvements in their dissimilar equivalent.

Finally, the efficacy/effectiveness of cognitive training was not supported in five studies (Godfrey and Knight, 1985; Hannon et al., 1989; Peterson et al., 2002; Brooks et al., 2016, 2017). Godfrey and Knight (1985) reported that the control and experimental groups showed the same improvement in terms of memory functioning. Hannon et al. (1989) concluded that the obtained results did not show sufficient support to confirm the objective of the study. However, there was still an increase in the Memory Matrix Test between the pre-test and the post-test. Peterson et al. (2002) did not confirm the efficacy of the Computerized Cognitive remediation program. Brooks et al. (2016) found that WM accuracy was improved in the experimental group, but that no near-transfer effects were found (no significant differences in the Trail Making Test). However, the experimental group did show more pronounced neural changes. Similarly, Brooks et al. (2017) reported a learning effect of 35% between pre and post-test, but no significant differences in executive measures (Trail Making Test).

In the present literature review, the risk of bias in randomized controlled trials was assessed using the Cochrane Risk of Bias Tool (Higgins et al., 2011). In turn, the non-randomized study (Hannon et al., 1989) was assessed for risk of bias using the ROBINS-I tool (Sterne et al., 2016). Since the methodological details of many of the studies included in the present review were incomplete or not sufficiently detailed (see Supplementary Material), we consider that the risk-of-bias assessment has limitations. However, we observed that the most common possible sources of bias in the randomized controlled trials studies selected for this review refers to the blinding of participants and personnel (performance bias) and blinding of outcome assessment (detection bias). There were also 14 studies in which we were unable to assess the type of concealment performed (Godfrey and Knight, 1985; Godfrey et al., 1985; Yohman et al., 1988; Wetzig and Hardin, 1990; Fals-Stewart and Lucente, 1994; Steingass et al., 1994; Peterson et al., 2002; Goldstein et al., 2005; Fals-Stewart and Lam, 2010; Rupp et al., 2012; Gamito et al., 2013, 2014; Eack et al., 2015; Brooks et al., 2017; Gunn et al., 2018) due to the lack of methodological information (as can be seen Supplementary Material). This lack of information is also a possible source of bias. With reference to low risk of bias, after complete analysis, only two studies (Hendershot et al., 2018; Khemiri et al., 2019) presented a low risk of bias in all the assessment domains (see Supplementary Material).

On the other hand, the quasi-experimental study included (Hannon et al., 1989) in the present review presented a moderated risk of bias on the baseline confounding, selection of participants and selection of reported results. There were also domains (deviation from intended information and missing data) where there can be possible risk of bias due to lack of information provided (see Supplementary Material).

The presented final assessment was discussed between the two reviewers (TC; CC) who examined the discrepancies between the performed evaluations. In situations where the reviewers did not reach a consensus, a third reviewer intervened (TA). The data found highlights selection bias, performance bias and detection bias as risk of bias for the cumulative evidence for the present review.

From all cognitive domains, memory was the domain most targeted in the reviewed studies. This is likely explained by the fact that memory is not only one of the areas most affected by substance use, but also one believed to impact treatment outcomes.

Significant improvements regarding memory could be found in studies with various SUD populations (i.e., substance of use). When considering overall memory capacity, positive and significant effects were found in participants who consumed both alcohol and other substances (Bell et al., 2016). Concerning WM specifically, significant improvements were shown for both participants who consumed only alcohol (Rupp et al., 2012; Gunn et al., 2018; Khemiri et al., 2019), and those who also used other substances (Hendershot et al., 2018). Khemiri et al. (2019) discriminated between verbal and visuospatial WM, and only found significant changes for the first. Delayed and verbal memory were also studied subdomains, and positive changes in these were found in alcohol-consuming participants (Rupp et al., 2012), and opioid-consuming participants (Gamito et al., 2017).

It is also important to analyse the studies that did not found significant memory improvements following cognitive training. From the studies that showed a clear lack of cognitive improvement after cognitive training, four studies focused on memory (Godfrey and Knight, 1985; Hannon et al., 1989; Brooks et al., 2016, 2017) with two of those specifically on WM (Brooks et al., 2016, 2017), and one considered a number of cognitive functions (e.g., visual-motor coordination, visual-spatial skills) including memory (Peterson et al., 2002). Regarding population, three of these studies explored the effectiveness of cognitive training in alcoholics (Godfrey and Knight, 1985; Hannon et al., 1989; Peterson et al., 2002), and two in methamphetamine users (Brooks et al., 2016, 2017).

Some of these studies presented significant limitations that may have affected the results, such as small sample size and/or high drop-out rate (Godfrey and Knight, 1985; Peterson et al., 2002), reported possible insensitivity of outcome measures (Godfrey and Knight, 1985; Hannon et al., 1989), and a lack of specificity in the training techniques (Godfrey and Knight, 1985). Moreover, Peterson et al. (2002), proposed that the lack of baseline cognitive impairment in their study participants may explain theirs result. They pointed out that cognitive training may be more effective on those with at least mild to moderate baseline cognitive impairment, something that would be interesting to consider in future research. Interestingly, two of these studies (Godfrey and Knight, 1985; Hannon et al., 1989) delivered the cognitive training intervention in a group setting.

Rass et al. (2015) and Brooks et al. (2016, 2017), presented results that justify a more in-depth look. Rass et al. (2015) had the only study that clearly discriminated results according to the measures' level of similarity to the training tasks. They found that there were significant improvements in some measures of WM similar to the training tasks, but no improvements in dissimilar measures. These results indicate the presence of “near” but not “far transfer” effects, and highlight the root of the on-going debate about cognitive training effectiveness. Brooks et al. (2016) too found that although WMT did not lead to significant changes in the cognitive measures used (i.e., Trail Making Test), it did increase memory accuracy (in the training tasks). In turn, memory accuracy showed itself to be connected with larger volume in the right middle frontal cortex and orbitofrontal cortex, both regions associated with WM ability and executive functioning. Brooks et al. (2017), found similarly that WMT did not lead to significant improvements in the cognitive measures used (i.e., Trail Making Test), but did lead to a learning effect of 35% and significant changes in self-report measures looking into impulsivity and self-regulation. These results are intriguing and bring up questions about the efficacy of cognitive measures in evaluating potential benefits of WMT, or cognitive training in general, and in adequately assessing “far transfer” effects. In a more recent review study, Brooks et al. (2020) reported that WMT can lead to significant neural effects often in the absence of behavioral changes. Moreover, various neuroimaging studies appeared to have found “far transfer” effects of WMT to other un-related cognitive domains, something that might be harder to measure.

Similarly to memory, executive functioning was also studied in different SUD populations (i.e., substance of use). Bell et al. (2017) found significant improvements on neurocognitive measures of executive functioning in participants who consumed both alcohol and other substances following 13 weeks (5 h/week) of cognitive training (both auditory and visual tasks). In line with these findings, Gamito et al. (2017) showed an improvement on the frontal lobe functions of opioid-consuming participants after 10 cognitive training sessions. Concerning mental flexibility specifically, significant improvements were found in alcohol-consuming participants (Gamito et al., 2014). Finally, problem-solving, which is a skill strongly associated with executive functioning, also showed significant positive effects in the same population (Yohman et al., 1988).

In comparison with memory and executive functions, there appears to be a lack of interest in studying the impact of cognitive training on processing speed. From the studies included in the review, only four targeted this cognitive domain (Gamito et al., 2014; Eack et al., 2015; Bell et al., 2017; Rezapour et al., 2019). And, from those, only Rezapour et al. (2019) reported significant improvements in the processing speed of individuals with opioid use disorder who received cognitive training. These improvements persisting for at least 6 months.

Cognitive training programs have suffered significant changes over the years as a result of technological advancement. When these programs first started to be used, they were administered with a paper-and-pencil modality, but today most new cognitive training programs created are computer- or even mobile-based. The studies included in this review reflected this tendency, with the majority of cognitive training programs used being computerized (Fals-Stewart and Lucente, 1994; Peterson et al., 2002; Fals-Stewart and Lam, 2010; Rupp et al., 2012; Gamito et al., 2013, 2014, 2016, 2017; Eack et al., 2015; Rass et al., 2015; Bell et al., 2016, 2017; Brooks et al., 2016, 2017; Gunn et al., 2018; Hendershot et al., 2018; Zhu et al., 2018; Khemiri et al., 2019).

Among the studies that used computerized programs, many created unique training regimes by adapting relevant cognitive tasks, while some used already recognized cognitive training programs. Cogmed was the most used program (Rass et al., 2015; Hendershot et al., 2018; Khemiri et al., 2019), followed by Posit Science (Bell et al., 2016, 2017), and the computer-based WM training program “Curb Your Addiction (C- Ya)” (Brooks et al., 2016, 2017). Other computerized training programs used were PSS CogRehab (Fals-Stewart and Lam, 2010), Cogpack (Rupp et al., 2012), NeurXerciseTM (Peterson et al., 2002), and Mobile-Based Computerized Cognitive Addiction Therapy (CCAT; Zhu et al., 2018).

Only five studies declared using paper-and-pencil training programs, and predictably four of those were among the oldest studies included in the review (Godfrey and Knight, 1985; Hannon et al., 1989; Wetzig and Hardin, 1990; Steingass et al., 1994). Interestingly, the fifth study (Rezapour et al., 2019), used the recently developed paper and pencil cognitive rehabilitation package NEuroCOnitiveREhabilitation for Disease of Addiction (NECOREDA).

There is another, more recent, type of cognitive training intervention that we did not considered in this review for lack of any studies that met the inclusion criteria-Virtual Reality programs. These type of interventions have shown promising results in other diseases and/or disorders that involve impairment of cognitive functions (Pedroli et al., 2018). However, to date, most studies that use virtual reality in the scope of SUD seek to understand the relationship between environmental stimuli and drug use (Bordnick et al., 2011; Hone-Blanchet et al., 2014). Indeed, studies that explore virtual reality as a cognitive training tool in SUD are scarce. To our knowledge, only Man (2018) has studied the effectiveness of this type of intervention on the improvement of cognitive functioning in individuals with substance abuse disorders. The results appear promising. As a drastically different form of delivering cognitive training, it is important that more research be conducted to study its effectiveness and compare it to the type of interventions used to date.

The presented results need to be interpreted taking into account this review's limitations. Of the 26 studies presented, 15 did not have an active control group (Yohman et al., 1988; Hannon et al., 1989; Steingass et al., 1994; Peterson et al., 2002; Rupp et al., 2012; Gamito et al., 2013, 2014, 2016, 2017; Eack et al., 2015; Bell et al., 2016, 2017; Brooks et al., 2016, 2017; Zhu et al., 2018), and 13 did not have a follow-up (Hannon et al., 1989; Wetzig and Hardin, 1990; Steingass et al., 1994; Peterson et al., 2002; Goldstein et al., 2005; Rupp et al., 2012; Gamito et al., 2013, 2016; Eack et al., 2015; Rass et al., 2015; Brooks et al., 2016, 2017; Zhu et al., 2018). There were also three studies that showed only a follow-up right after the intervention (Gamito et al., 2014, 2017; Khemiri et al., 2019). These limitations make it impossible to effectively account for possible placebo effects as well as infer the maintenance of any real effects over time.

It is also important to highlight the diversity of cognitive training programs (e.g., administration, duration, number of sessions, and hours of training) and populations (i.e., substance of use, time of abstinence) included in the reviewed studies. This heterogeneity, along with the lack of detailed information about the used interventions found in many studies, prevented us from analyzing the results more in-depth and from evaluating the real impact of these variables, for example on effect size. It also made it impossible to generalize about the improvements obtained in cognitive functions for the general population with SUD.

Finally, the lack of concealment concerning the researchers in most of the included studies in the present review stands out, along with the fact that some studies failed to provide information regarding the methodology used for concealment of the participants.