Computer-aided search was performed by four researchers (Susy Braun, Melanie Kleynen, Tessa van Heel, Nena Kruithof) using PubMed/Medline, PEDro, Science Direct, Cochrane Library, PsycINFO, Rehadat, Embase, and Picarta. The authors hand-searched reference lists of obtained articles (reference and author tracking). Key words used were: stroke, Parkinson's disease, multiple sclerosis, mental practice, movement and motor imagery, motor learning, rehabilitation, physical therapy, occupational therapy, activities of daily living. These search terms were used in Dutch and German articles as well and were translated if necessary. The detailed search strategy is available from the authors.

Screening on tittle and abstract was performed by two researchers (Susy Braun, Melanie Kleynen) independently. If based on the information in the abstract, it was not clear whether the study should be included the full-text of the article was assessed.

Methodologic quality assessment of the studies was assessed using the Amsterdam-Maastricht Consensus List for Quality Assessment (AMCL; Van Tulder et al., 2003). The AMCL was originally developed by van Tulder et al. for the Cochrane Collaboration Back Group and includes all criteria of other prominent quality scales like the Delphi List (Verhagen et al., 1998). It rates a study's internal validity and statistical reporting using an 11-point scale (12 criteria), with higher scores indicating higher quality. Each criterion was scored either positive (+, 1 point), negative (−, 0 points), or unclear (?, 0 points), leading to a maximum score of 11 points per study (1 point for the items 2–11; ½ point for the items 1a and 1b).

To increase uniformity in the assessment the validity criteria were defined and then discussed by the two researchers (Susy Braun, Melanie Kleynen). Each item of the AMCL was explained in a separate document that provided uniform operationalization of criteria. In the Appendix the definitions and cut-offs of the criteria of the AMCL are described (Table A1). For example “an acceptable percentage of withdrawals” (criterion 7) was defined as: 10% during the intervention period and from the remaining sample 10% during follow-up as suggested by Van Tulder et al. (2003) Compliance (criterion 5) was considered acceptable if participants themselves or therapists and relatives reported that the participants followed the given instructions. A follow-up period (criterion 10) of at least 3 months was considered clinically relevant for this type of intervention. For these last two mentioned criteria (criteria 5 and 10) reviews of other interventions within health care were used as standard, for generally accepted references in literature were not found (Huibers et al., 2003; Van Tulder et al., 2003). If disagreement on the scores persisted, a third researcher (Anna Beurskens) was approached to reach consensus. A study was defined as being of “sufficient quality” if the score was equal to or above six points. As standard references from the literature are missing, the cut-off was defined by the authors after references from other reviews in physical therapy were taken into account (van Tulder et al., 2001; Van Tulder et al., 2003; Huibers et al., 2003).

Authors of the included articles were contacted to clarify the items on which a question mark was scored. Both scores (blinded assessment as well as after contact with authors) are presented.

Information was extracted from each included trial on: (1) study and population (including number of participants and mean age); (2) type of intervention. We for instance wanted to know if an instruction period for therapists and participants was embedded within the mental practice intervention period (e.g., stepwise approach, tools to check compliance) and what the content of the mental imagery session would be (e.g., what activities were rehearsed and how the imagery was instructed e.g., tape, therapist.); (3) type of outcome measure for physical recovery (primary and secondary measurements, assessment time points, and follow-up period); (4) conclusion (is mental practice recommended and what are the (significant) effects on physical recovery). The conclusions were based on the results and conclusions in the articles but summarized by the independent researchers; (5) All included articles were screened on possibly reported effects on cognition or emotion as well as side-effects (quantitative and qualitative measures). If (secondary) measures were used to consciously search and systematically identify effects on cognition or emotion within the study design the results were categorized as “effects.” Side-effects are described as effects that were not intended, but were observed and reported. These side-effects could be therapeutic (positive) or adverse (negative). Both independent reviewers extracted data from the full papers by using a pre-structured standard form.

A meta-analysis was conducted using Review Manager version 5.1.6. (The Nordic Cochrane Centre TCC, 2011). Post intervention scores and if possible follow-up scores (at least 3 months) were used. Short- and long-term effects of the intervention were distinguished for two reasons: initial effects might extinguish over time and most studies did not perform a follow-up. Data from both measuring moments were analysed separately. Studies were excluded from the analyses if not all necessary data was provided in the article. No data was imputed.

If a study included two control groups, mental practice was compared to the group with the least chance of improvement (e.g., control group). If no significant differences were found between those groups, it was assumed that no differences would be found between the two experimental groups.

Studies with identical physical effect measurement instruments or studies with instruments measuring the same construct were pooled. The Mean Difference (MD) and 95% confidence interval (95% CI) was used if data was based on identical measurement instruments and the Standardized Mean Difference (SMD) for data based on different measurement instrument. Statistical heterogeneity was assessed using the I²-statistic. If I² was greater than 50% outcomes were pooled using SMD with a random effects model. If there was a big variance in Standard Deviations (SD) across studies, reflecting differences in the real variability of outcomes, we also used the SMD. Sensitivity analysis was done to investigate the influence of studies with ACML scores below 6 (lower quality studies). If for instance data were pooled from studies with both lower and high quality, the analysis was performed first with all studies and then repeated without the lower quality studies (Barclay-Goddard et al., 2011).

The scores on the AMCL (range 0–11 points) of the included studies varied from 3.5 to 8 points after blinded assessment of the reviewers. After additional information was retrieved through authors contact (directly or through earlier confirmed information by the authors in the Cochrane review (Barclay-Goddard et al., 2011)) to clarify the questions marks the scores ranged from 6 to 9 points (Table 1).

Based on the scores after assessment of the articles by the independent reviewers of the text only, 11 of the 16 studies scored 6 points or more and were considered to have sufficient methodologic quality (Page et al., 2001, 2005, 2009; Liu et al., 2004; Tamir et al., 2007; Liu, 2009; Bovend'Eerdt et al., 2010; Braun et al., 2011a, 2012; Ietswaart et al., 2011; Welfringer et al., 2011). After additional information was processed three more studies came to a total score above six points (Page et al., 2007; Riccio et al., 2010; Schuster et al., 2012). Of these 14 studies with at least sufficient quality, half (n = 7) showed overall positive effects of mental practice on arm-hand-function, activities of daily living and mobility of which six in stroke (Page et al., 2001, 2005, 2007, 2009; Liu et al., 2004; Riccio et al., 2010) and one in Parkinson's disease (Tamir et al., 2007). In three high quality studies in stroke positive results were found in favor of the experimental group but not on all outcome measures (Liu, 2009; Welfringer et al., 2011; Schuster et al., 2012) and four high quality studies reported similar effects in the control and experimental group, of which three in stroke (Bovend'Eerdt et al., 2010; Ietswaart et al., 2011; Braun et al., 2012) and one in Parkinson's disease (Braun et al., 2011a). Of the two remaining low quality studies in stroke, one study did not find significant differences between groups (Muller et al., 2007) and one study had mixed results (Liu et al., 2009).

Study characteristics of the included randomized controlled trials are shown in Table 2.

Age of the participants varied from 40 to 84 years. The time post-stroke differed greatly, ranging from 0 to 7 days after stroke (Liu et al., 2009) to the chronic phase of recovery (>6 months after stroke; Page et al., 2007; Schuster et al., 2012). The average time after the diagnosis of participants with Parkinson's disease was between 5.2 and 7.8 years. Based on these qualitative descriptions mental practice seems to have potential effects in all ages of participants and phases of stroke recovery. In participants with Parkinson's disease, effects of mental practice were more often reported in the two included studies in participants with Hoehn and Yahr stage 1 or 2.

Six studies embedded mental practice in therapy (Liu et al., 2004, 2009; Tamir et al., 2007; Bovend'Eerdt et al., 2010; Braun et al., 2011a, 2012), nine studies added mental practice to therapy (Page et al., 2001, 2005, 2007, 2009; Muller et al., 2007; Liu, 2009; Riccio et al., 2010; Ietswaart et al., 2011; Welfringer et al., 2011) and one study investigated both embedded and added mental practice (Schuster et al., 2012).

The intervention in the control group varied from a single intervention like relaxation therapy (Page et al., 2005, 2007), (general) information (Page et al., 2001), embedded therapy as usual (Muller et al., 2007; Tamir et al., 2007; Page et al., 2009; Bovend'Eerdt et al., 2010; Riccio et al., 2010; Braun et al., 2011a, 2012; Ietswaart et al., 2011; Welfringer et al., 2011; Schuster et al., 2012) to therapy according to the demonstration-then-practice method (Liu, 2009; Liu et al., 2004, 2009).

The activities or skills practiced in the intervention group could be restricted to only movements of the arm (e.g., drinking from a cup; Page et al., 2005) or could contain complex tasks involving the entire body (e.g., going to the park; Liu et al., 2009). Frequency of the intervention varied from two to five times a week, while the intervention lasted between 30 and 60 min per session and continued for 2 to 10 weeks. The included studies used different types of imaging (Mulder, 2007): participants were offered kinesthetic motor imagery (Page et al., 2005; Riccio et al., 2010; Ietswaart et al., 2011; Welfringer et al., 2011), or a combination of kinesthetic and visual motor imagery (Page et al., 2001, 2007, 2009; Liu et al., 2004, 2009; Muller et al., 2007; Tamir et al., 2007; Liu, 2009; Bovend'Eerdt et al., 2010; Braun et al., 2011a, 2012; Schuster et al., 2012).

Based on these qualitative descriptions it seems that different kind of interventions may have potential effects on activities (e.g., embedded and added mental practice were both reported effective and ineffective in different studies).

Five studies reported effects on cognition (Liu et al., 2004, 2009; Tamir et al., 2007; Liu, 2009; Welfringer et al., 2011) which were measured with the Stroop test (part B) in participants with Parkinson's disease (Tamir et al., 2007), the Color Trails Test (CTT; Liu et al., 2004) and the Cognistat (Liu, 2009) in participants after stroke (Table 2). Participants with Parkinson's disease seemed to have an increase in attention and concentration after the mental practice intervention period (Tamir et al., 2007). In the studies by Liu et al. (2004, 2009); Liu (2009) the mental practice intervention involved strategy training and participants after stroke seemed to be more able to form cognitive maps of routes and plan actions in unfamiliar surroundings compared to the participants in the control group. Earlier positive findings on the CTT were however not repeated in a later study using the Cognistat (Liu et al., 2004; Liu, 2009).

Positive observed side-effects reported in the stroke trials were increased autonomy (Braun et al., 2012) and increased sensations in and awareness of the left arm (Welfringer et al., 2011). In Parkinson's disease research there was some indication that imagery might increase motivation and arousal and reduce depression (Tamir et al., 2007).

Two studies reported acute adverse side-effects of mental practice (Braun et al., 2011a, 2012) like “too much effort,” “not worthwhile,” and “too confronting.” Some participants after stroke showed diminished concentration and signs of tiredness at the end of mental practice training sessions (Welfringer et al., 2011).

The meta-analysis was conducted using a selection of the studies in stroke in which the same physical outcome measurement instruments were used (Table 3).

A meta-analysis in participants with Parkinson's disease was not possible. Both studies use the Timed up and Go as an outcome measure, but Tamir et al. did not report the exact data (only figures provided; Tamir et al., 2007). Several outcome measures which had been used in at least two studies were excluded because of missing data (Motricity Index, Pinch/Hand force; Muller et al., 2007; Riccio et al., 2010). The study of Schuster et al. (2012) was excluded because of clinical important differences between groups at baseline. Six times data could pooled in a meta-analysis. No sensitivity test could be performed as all studies that could be pooled were of at least sufficient quality.

Data were available from two studies (Bovend'Eerdt et al., 2010; Braun et al., 2012) that randomized a total of 64 and 58 participants respectively. Pooling did not lead to significant effects assessed with the Rivermead Mobility Index directly after the intervention (p = 0.72; MD: −0.82; 95% CI: −3.04 to 1.41) nor at follow-up (p = 0.75; MD: −0.40; 95% CI: −2.90 to 2.10).

Data were available from seven studies (Page et al., 2001, 2005, 2007, 2009; Bovend'Eerdt et al., 2010; Ietswaart et al., 2011; Welfringer et al., 2011) that randomized a total of 197 participants. Due to heterogeneity in the SDs of outcomes SMD and random-effect model were used. Pooling led to significant short-term effects on the Action Research Arm Test (p = 0.03;SMD 0.62; 95% CI: 0.05 to 1.19). No data for long-term effects could be pooled.

Data of the Barthel Index were available from three studies (Bovend'Eerdt et al., 2010; Ietswaart et al., 2011; Braun et al., 2012) on short-term that randomized a total of 135 participants. Pooling did not show significant effects (p = 0.31; MD: 0.87; 95% CI: −0.80 to 2.53). No significant effects were found at follow-up either (p = 0.75; MD:0.46; 95% CI: − 2.36 to 3.27). Data for long-term effects were available from two studies (Bovend'Eerdt et al., 2010; Braun et al., 2012) that randomized a total of 57 participants. The study of Liu (Liu et al., 2009) used the modified Barthel Index and was therefore excluded from pooling in both meta-analyses.

Four studies (Liu et al., 2004, 2009; Liu, 2009; Braun et al., 2012) used a numeric rating scale to assess the performance of functional activities. Two studies were excluded from the analyses because they did not provide any point estimates (Liu, 2009; Liu et al., 2009). The data of the studies by Braun et al. and Liu et al. were pooled using SMD because Braun et al. (2012) used a 10-point scale whereas Liu et al. (2004) used a 7-point scale. Liu et al. provide data of the average score of five activities that was used for the analyses. Braun et al. provided scores of the NRS of drinking, walking, and two self-chosen activities. We used the data of the most promising result (biggest different between the experimental and the control group) this was the score of the self-chosen activity for the lower limb. Data of 78 participants could be pooled and a marginal significant overall effect on short-term was found. (p = 0.04; SMD 0.9; 95% CI: 0.04 to 1.77). No long-term data could be pooled.

There is a possibility that studies were missed due to inconsistency in terminology used in databases (e.g., mental practice, motor imagery, movement imagery).

The varied clinical populations in this review can be seen as a limitation. This review however does summarize the existing information in neurorehabilitation about a widely used intervention, which will facilitate the exchange of existing knowledge and evidence between professionals working with different target populations. Reviews covering the evidence for specific interventions in multiple target groups, like the recently published one by Newman and Barker (2012) on supported standing, will help professionals get a better understanding of the intervention and potential (side-) effects.

Using assessment scales in general and therefore also the AMCL for rating methodologic quality leads to some practical issues. Blinding of therapists and patients is often not possible in therapeutic interventions like mental practice. If therapists instruct the patients they are not blind to the type of intervention they are providing. The same accounts for the patients when they are asked to actively participate in an intervention. In randomized controlled trials in which therapeutic interventions (e.g., physiotherapy and occupational therapy) are researched the assessment of the randomized controlled trials with any assessment scale will be lower than in for instance pharmaceutical studies. The highest possible score on the AMCL of 11 points will decrease in many therapeutic studies by 2 points, as double blinding is often not possible.

Different assessment tools were used to rate the quality of the included studies (PEDro (Nilsen et al., 2010; Barclay-Goddard et al., 2011), AMCL (Braun et al., 2006), JADAD (Cha et al., 2012)) in earlier reviews. The PEDRO and AMCL are derived from the Delphi list and therefore interrelated (Olivo et al., 2008). The JADAD is a shorter list, most used even though it was not originally developed for therapeutic studies (Olivo et al., 2008). Sometimes the studies within the reviews were categorized into lower and high quality studies (Braun et al., 2006; Zimmermann-Schlatter et al., 2008; Cha et al., 2012) and sometimes the authors of trials were contacted to provided additional information (Barclay-Goddard et al., 2011).

We contacted the authors to clarify the criteria on which a question mark was scored after the blinded reviewing assessment by the independent reviewers was performed. The quality assessment of identical studies may therefore vary within the different reviews as scores on assessment tools normally go up after additional information is retrieved. In one review the quality criteria for assessment of the identified studies were chosen by the reviewers (Zimmermann-Schlatter et al., 2008). These differences make it harder to compare the results and recommendations from the reviews.

Good reporting of trials is important to understanding changes and effects of mental practice and therefore ongoing attention to high quality study reports is required (Barclay-Goddard et al., 2011). Guidelines, like the CONSORT statements are essential to achieve this.

The biggest problem of researching mental practice is the lack of consensus on the definition and concept of the intervention. Heterogeneity within the intervention protocols and outcomes makes it impossible to conduct an overall pooling and thus to come to an overall conclusion.

Results from pooling based on identical outcome measures should be interpreted with caution because of the heterogeneity in study populations. Also, results from meta-analyses depend very much on the data (and models) used. We decided to base the decision on which model to use on the measurement instruments (identical instruments or instruments measuring the same construct) and on the variance in SDs across the included studies. The downside of this flexibility in data/model choice is that it is harder for the reader to follow what has been done in the analysis. The biggest and in our opinion more important advantage is that the outcome is less misleading. Big variation in SDs across studies reflect differences in the real variability of outcomes and the use of MD would in our case suggest potential effects which are probably not there. The study by Ietswaart et al. (2011) with the largest population would for instance have the lowest weight in the meta-analysis and studies with relatively small sample sizes would determine outcome for more than 80%. We tried to correct for this heterogeneity in the analysis and we used change scores instead of effect sizes which might explain to some extend why our results are less optimistic than the meta-analysis by Cha et al. (2012). In the meta-analysis by Cha et al. mental practice combined with exercise therapy had an even bigger effect size (ES 0.51; moderate) then augmented therapy alone (Cha et al., 2012). Two other reviews (Nilsen et al., 2010; Barclay-Goddard et al., 2011) performed statistical analyses to synthesize the evidence of six (Barclay-Goddard et al., 2011) and four (Nilsen et al., 2010) studies. Differences in statistical analysis approaches should be taken into account when interpreting and comparing the results.

Publication bias is a potential weakness in all systematic reviews, as positive or statistically significant findings are more likely to be published than small trials with non-significant or negative findings (Thornton and Lee, 2000). The funnel plot of the ARAT showed indication for publication bias (results not presented) and should therefore be interpreted with care. Barclay-Goddard et al. (2011) identified some risk of bias with regard to concealment of allocation and blinding. Cha et al. (2012) did not report any significant publication bias in their investigation.

Determining effects of complex interventions like mental practice is complicated (Braun et al., 2011b). A systematic way of assessing the potential of mental practice could be through the four steps suggested by the Medical Research Council (Craig et al., 2008). Until now, most research has been performed in the first two steps of this model: “determining the working mechanisms” and “piloting.” Fundamental research has shown that mental practice can be performed in patients with neurological conditions and showed that the underlying mechanism is also working in at least parts of the patient populations.

The past 5 years more research has been published on techniques that might assist in monitoring and implementing imagery treatments, like tests (e.g., chronometry, hand-rotation-test; Malouin et al., 2008a; Simmons et al., 2008) and questionnaires (e.g., KVIQ; Malouin et al., 2007, 2008b). Mental practice has been explored in different clinical situations and contexts and a range of different types of intervention, assessed with different measures, have been studied. However, the predictive value of these tests has not been established yet. So we do not know for sure if people who can image according to questionnaires and tests will also benefit from it and whether participants who are at first unable to image, are able to learn and potentially benefit from imagery. In addition, if imagery tests are used as a selection tool patients who are unable to perform these tests are often excluded from research. That is why the study by Welfringer et al. (2011) is of value. Although the results should be interpreted with great caution because participants in the control group did not receive supplementary therapy on top of therapy as usual to control for mental practice, it is until now the only mental practice randomized controlled trial in patients with neglect. Researching feasibility and effects in sub groups that are normally excluded from research will tell us more about whether mental practice can be taught and who might benefit.

There are general methodologic issues in rehabilitations trials (Dobkin, 2007) that also should be considered in mental practice studies. The main problem is that almost all studies are underpowered, increasing the chance of type-2 errors. Especially in mono-centered, small trials the samples are not likely to reflect the real-world sample.

Recent negative trials have shown that not all participants with stroke and Parkinson's disease benefit from mental practice. At this point we do not know how to identify the people who might benefit from mental practice. Sample sizes of future trials should be large enough to enable sub group and dose-response analyses. For dose-response analyses adherence, attendance, and compliance should be reported (Barclay-Goddard et al., 2011). Especially adherence and compliance are difficult to assess as mental practice is an intervention that takes place in the mind and remains covert for the therapist. For adherence to mental practice it is essential that participants can engage in movement imagery. However, there is no perfect test to assess this ability. Combining some tests might provide indicators which then might be related to outcome. Therefore, the imagery ability of every participant should be assessed before and/or after the mental practice intervention.

The mental practice intervention should be well described. Both, short- and long-term effects should be measured with predefined measure instruments to enable comparison of results among different studies. Effects should not only be sought at the physical level, but also on emotion and cognition (Nilsen et al., 2010; Barclay-Goddard et al., 2011). Reporting the opinions on and experiences with mental practice of people with neurological diseases, care-givers, and care professionals will provide valuable information on how to optimize and tailor the mental practice intervention to the patients' needs and abilities. Mixed methods are needed to assess these different components.

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.