Apparatuses involving arm and/or hand movements for probing approach-avoidance tendencies may differ in their sensitivity to valence (Krieglmeyer and Deutsch, 2010). Results with the button stand of Rotteveel and Phaf (2004), which was not investigated by Krieglmeyer and Deutsch, may even yield qualitatively different results, due to the movement in the vertical direction instead of in the sagittal direction as with joystick movements. One type of experimental setup only measures approach-avoidance behavior on an abstract level, not involving any physical arm movements (De Houwer et al., 2001). Participants control a manikin on the computer screen that appears randomly above or below a stimulus. By means of manual key presses they either move the manikin toward or away from the stimulus. This setup is referred to here as the abstract-manikin task. Because also more abstract setups are involved in this moderator variable, it will be labeled “task.”

In the joystick task, approach-avoidance behavior is operationalized as horizontally pulling or pushing a vertically positioned control stick. This may involve flexion and extension of the arm, but also pulse and shoulder movements. The same applies to a lever when used as approach-avoidance task (Chen and Bargh, 1999). Because joystick responses suffer from more sideways variability than lever responses, the results from the latter may be slightly more accurate than from the former. The joystick and lever measures are, however, treated as the same task.

The feedback-joystick task, which was discussed above (cf. Seibt et al., 2008), should be considered a separate task. It was shown that due to the visual feedback the task is resistant to cognitive reinterpretations (Rinck and Becker, 2007). When a stimulus-reference point was induced by rephrasing instructions (i.e., pull the joystick away from the picture, push the joystick toward the picture), the compatibility effect did not reverse. Therefore, the crucial aspect in the feedback-joystick task seems to be the visual reinforcement that the stimuli come closer or disappear.

Another distinct task is the vertical three-button stand (Rotteveel and Phaf, 2004). Movements are only made in the vertical direction by either flexing the arm with the biceps muscle or extending the arm with the triceps muscle. Instructions are typically not contaminated with explicit references to approach and avoidance behaviors (i.e., toward and away). Compatibility effects can generally only be observed in the initiation times (the interval between the start of stimulus presentation and resting button release), which reflect the time needed for response preparation (Rotteveel and Phaf, 2004). With respect to the button stand thus only initiation times are involved in this meta-analysis.

The relatively small number of studies that investigated approach-avoidance tendencies toward affective stimuli with whole-body movements (e.g., Stins et al., 2011) did not serve as an extra level to this moderator variable. This would have resulted in a large number of empty cells and deviates somewhat from the lines set out by the seminal Solarz (1960) and Chen and Bargh (1999) studies using arm/hand movements to probe approach and avoidance tendencies. Nevertheless, the Stins et al. results conceptually replicated the results of the latter studies by showing that the initiation of forward steps took more time when evaluating angry than happy faces.

A central question in the literature is whether affective information processing automatically triggers flexion (i.e., approach) and extension (i.e., avoidance), independent from the intention to evaluate the stimuli. Three different instructions were compared to investigate this question. With explicit instructions, as in Solarz (1960) and in Experiment 1 of Chen and Bargh (1999), participants are asked explicitly to evaluate the stimuli with the approach-avoidance task on a positive-negative dimension. They are, for instance, told to pull the joystick toward them or push it away from them when they judge the stimulus as positive or negative, respectively. Implicit instructions do not require participants to attend to stimulus valence. Instead, they are instructed to react to a task-irrelevant feature (e.g., the background color or the gender of a face). The most extreme form of implicitness can be found in research where also the affective valence of the stimuli is implicit (i.e., not consciously recognized by the participant). There are, however, only very few examples of such studies (e.g., Phaf and Rotteveel, 2009; Jones et al., 2011). Nevertheless, “Valence” (explicit vs. implicit) will be included as a separate moderator variable in the meta-analysis. In experimental comparisons explicit instructions tend to yield larger effect sizes than implicit instructions (e.g., Krieglmeyer and Deutsch, 2010).

The third type of instruction will be termed explicit-converted (i.e., relative to the Solarz, 1960, and Chen and Bargh, 1999, studies). Here, the instructions require explicit evaluation, but the meaning of the arm/hand movements is changed, usually by reversing the reference point of flexion and extension (e.g., from the self to the object), or by a relabeling of the end-points of the movements. With object reference flexion of the arm now corresponds to avoidance, whereas extension reflects an approach movement. Other types of instructions are also included in the explicit-converted level, as in Experiment 3 of Eder and Rothermund (2008), where movements to the right or to the left are labeled affectively. Although it would in principle be possible to obtain converted results in implicit evaluation conditions, these conditions have only been tested in combination with explicit evaluation instructions, which means that participants were instructed to respond with the arm-hand movements to the valence of stimuli.

By comparing explicit and explicit-converted conditions, the question can be addressed whether action-tendencies to approach and avoid are context-independent movements consisting of specific motor patterns (i.e., of arm flexion and extension). Furthermore, non-zero effects with implicit instructions would argue for an automatic link between affective information processing and approach-avoidance behavior that does not depend on the intention to evaluate the affective meaning of stimuli.

In most studies, the same participants were tested with both positively and negatively valenced stimuli. This introduces interdependence between reaction times. In order to extract multiple effect sizes from the same study, positive and negative stimuli were analyzed separately, when possible. Some studies, however, only reported reaction times pooled across compatible (i.e., approach positive and avoid negative stimuli) and incompatible trials (i.e., avoid positive and approach negative stimuli). A third type of analysis was added with the pooled reaction times, where only studies were included that did not report the separate means and standard deviations, or did not provide them upon request. A moderator variable labeled “Design” was also included for comparing within-participants with between-participants manipulations of compatibility.

In the meta-analysis we only included studies using strongly affective stimuli that favored the direct link hypothesis of Chen and Bargh (1999). A primary category of such stimuli may be evolutionary prepared (e.g., emotion faces), which might be more suitable than others (e.g., words) to automatically elicit affect and thus may provide a better opportunity for investigating the automaticity of approach-avoidance tendencies. Emotion words are less likely to be evolutionary prepared, because across languages words denoting the same emotion generally do not share the same perceptual characteristics (e.g., see Phaf and Kan, 2007). Within each analysis (i.e., positive, negative, or both affects), four types of stimuli were investigated. The first type concerned words with an emotional content. These are words that have been selected on the basis of their strong affective valence and thus can be explicitly evaluated on a positive-negative dimension. For this reason studies using individually relevant (e.g., addiction-related, which may be affectively ambiguous; Wiers et al., 2011), and/or weakly valenced stimuli (e.g., social exemplars, Castelli et al., 2004; homophobic words, Clow and Olson, 2010; goal and temptation related words, Fishbach and Shah, 2006) were excluded. In addition, we considered pictures depicting emotional scenes, mostly selected from the International Affective Picture System (IAPS) (Lang et al., 1996). The third type consisted of emotional facial expressions, which are presumably evolutionary prepared stimuli and might therefore be processed more automatically (Öhman, 1986) than for instance words. To ensure comparability across studies, only happy and angry facial expressions were included. The fourth stimulus type involved personally relevant stimuli. Approach-avoidance tasks are often used to assess action-tendencies with stimuli that are relevant to some individual concern. These stimuli were predominantly spider pictures that are tested with participants suffering from a spider phobia.

In sum, we investigated the direct hypothesis in three separate analyses, for positive, negative, and both affects. Krieglmeyer and Deutsch (2010) already provided a direct experimental comparison of some frequently used measures of approach-avoidance, but the field still lacks a quantitative review of the available data. Divergent research results may be caused by confounds of subsidiary factors, such as instruction, apparatus, design, and stimulus, differentially moderating the affective influences on approach-avoidance behaviors. The goal of the current meta-analysis is to provide an estimation of the overall effect size and to shed light on the role of potential moderators. In contrast to an experimental study, however, there is no control in a meta-analysis over the number of replications at a given level of a moderator variable. In addition, the meta-analysis may thus also reveal research areas in need of further study with the approach-avoidance task. Finally, we will investigate whether the literature in this field is subject to publication bias (Rothstein et al., 2005).

A literature search for relevant studies was conducted (until June 2012) across four databases (ISI Web of Science, PsycINFO, PubMed, and Google Scholar) using the search string “(approach-avoidance behavior OR approach-avoidance task OR compatibility) AND evaluation,” with OR and AND representing Boolean operators. The search in PsycINFO resulted in 325 references. In addition, cited reference searches were conducted in ISI Web of Science to search for studies that referred to studies representative for the joystick-lever (Chen and Bargh, 1999, 283 references), the feedback-joystick (Rinck and Becker, 2007, 41 references), and button stand tasks (Rotteveel and Phaf, 2004, 49 references). Additional studies were identified by manual search which consisted of the screening of studies we already knew from our prior research on the approach-avoidance task, and the references therein. Because these provide the best guarantees for study quality, only peer-reviewed, published studies were included in the meta-analysis.

Studies were included according to the following criteria: (1) Studies investigated healthy participants (i.e., not patients). (2) To maximize the chances of finding support for the direct-link hypothesis, only studies with clearly positive and/or negative stimuli were included. (3) Studies involving longer-term moods (e.g., by mood-induction procedures instead of by emotional stimuli) were excluded, but we did include data from control conditions and behavioral assessments prior to a to-be-excluded manipulation (e.g., Roelofs et al., 2005). (4) Studies should employ the joystick/lever, feedback-joystick, abstract tasks, or the button stand as the dependent measure. This resulted in the exclusion of studies that had whole-body movements as the dependent measure (e.g., Stins et al., 2011), or that investigated the reverse effect of arm movements on evaluation of stimuli (e.g., Cacioppo et al., 1993). (5) Studies should report relevant means and standard deviations (or standard errors), which according to Dunlap et al. (1996) should be used rather than t-values and other test statistics to compute effect sizes for correlated designs. To prevent a potential inflation of effect sizes, studies that did not report these statistics and from which we could not retrieve the necessary information from the authors were excluded. In a previous meta-analysis (Phaf and Kan, 2007), moreover, we noticed that the discrepancies between effect sizes calculated in the two different manners would sometimes be much larger (in the most extreme case they differed by a factor 10) than suggested by Dunlap and collaborators, possibly due to errors in the statistical analysis. To limit the potential for publication bias due to statistical error, which may be quite prevalent (Bakker and Wicherts, 2011), we therefore had to exclude a number of classical studies that did not report, and where the authors did not provide us with, means and standard deviations (e.g., Duckworth et al., 2002; Seibt et al., 2008; Proctor and Zhang, 2010). Also studies that did not report the results for different levels of a moderator variable separately (e.g., words and pictures, Bamford and Ward, 2008) could not be included in the meta-analysis.

All studies were published between 1999 and mid-2012. Altogether, the meta-analysis included 29 usable studies, from which 81 effect sizes were obtained (Combined N = 1538). A detailed overview, listing the studies by moderators, is provided in the supplementary material.

Effect sizes were computed in terms of Cohen's d (see Equation 1).

Cohen's d refers to the standardized mean difference between experimental conditions (Hedges and Olkin, 1985; Borenstein et al., 2009), an incompatible condition (INC) and a compatible condition (COMP), divided by the pooled standard deviation (i.e., Equation 4.4 from Borenstein et al., 2009). Compatible conditions refer to situations where participants approached positive stimuli or avoided negative stimuli, incompatible conditions to situations where participants avoided positive stimuli or approached negative stimuli. In the both affects analysis the reaction times were pooled for the two conditions. With explicit-converted instructions, the coupling between valence and the flexor and extensor movements is usually reversed. In this case, flexor movements to negative stimuli and extensor movements to positive stimuli were considered compatible and the inverse coupling incompatible. This changes the sign with respect to the explicit condition. In the study by Eder and Rothermund (2008; Experiment 3), however, left and right movements were labeled positively and negatively, respectively. These instructions were also coded as explicit-converted instructions in the current meta-analysis, in the sense that this response-label assignment is different from what we refer to as standard explicit instructions.

Cohen's d has a slight bias in small samples (Hedges and Olkin, 1985), so we transformed it into Hedges' g, using the correction factor J (Equation 4.22 from Borenstein et al., 2009). This unbiased estimator, Hedges' g (Hedges and Olkin, 1985), was used for subsequent analyses and the correction factor was also applied to sampling variances (Equation 4.24 from Borenstein et al., 2009). Because the majority of studies had repeated measures designs (k = 26), the variance of g was computed with Equation 4.28 from Borenstein et al. (2009), which requires the correlation (r) between pairs of observations (see Dunlap et al., 1996). For those studies using an independent groups (i.e., between-participants) design, the sampling variance of g was computed with Equation 4.20 from Borenstein et al. (2009). All studies in the current meta-analysis with independent groups had equal sample sizes per group.

Whenever necessary, authors were contacted to gather means and standard deviations in order to compute effect sizes. It is not common practice to also report r, the correlation between pairs of observations in repeated-measures designs. Therefore, r was estimated from paired t-tests according to Equation 2.

It could, similarly, be calculated from repeated measures ANOVAs according to Equation 3.

For some studies (Phaf and Rotteveel, 2009; Seidel et al., 2010a,b), r could be computed from the raw data and compared to estimations derived from the test-statistics. The estimations turned out to be fairly accurate, validating the use of the above formulas. For the remaining studies that did not report the relevant test-statistics the average of all available correlations, weighted by individual sample sizes, was imputed as the correlation for that individual study. If means and standard deviations were not provided, and if the correlation between measures was not reported, nor could be estimated appropriately, the study was excluded as recommended by Dunlap et al. (1996).

Analyses were performed in the statistical software package R (version 2.14.1) (R Development Core Team, 2010) with the metafor package (Viechtbauer, 2010). Due to expected heterogeneity, all analyses were computed within the random-effects model. The proportion of systematic unexplained variance (τ²) was estimated using restricted maximum-likelihood estimation, which is approximately unbiased and quite efficient (Viechtbauer, 2010). Cochran's Q-test (Hedges and Olkin, 1985) served to test the null hypothesis of homogeneity of the effect sizes. A significant Cochran's Q-test indicated study heterogeneity. Influence analysis (i.e., the exclusion of single studies) was performed to identify influential studies based on Cook's distance and residual heterogeneity.

Separate analyses were performed for positive affect, negative affect, and both affects. Moderating variables were defined a priori. Hypothesized categorical moderators were (1) task: vertical button stand, joystick/lever, feedback-joystick, abstract-manikin task; (2) instruction: explicit (i.e., task-relevant), explicit-converted (i.e., task-relevant), implicit (i.e., task-irrelevant); (3) stimulus type: emotional facial expressions, emotional words, emotional pictures, personally relevant stimuli: (4) design: repeated measures design, independent groups design; (5) valence: explicitly valenced stimuli, implicitly valenced stimuli. The statistic Q_m served as an omnibus test for differences between levels, Q_e as a test for residual heterogeneity.

An important issue in meta-analyses is the occurrence of a publication bias (Rothstein et al., 2005). Studies with statistically significant effects and positive treatment outcomes are more likely to be published than null results. If a publication bias is present, the studies included in the meta-analysis are not representative of all valid studies undertaken in the field, leading to an over-estimation of the effect. If studies with non-significant results remain unpublished, this may be reflected in an asymmetric funnel plot and an excess of significant findings (Sterne and Egger, 2005; Ioannidis and Trikalinos, 2007; Bakker et al., 2012; Francis, 2012). In the graph a measure of the accuracy of the study is plotted against the effect size. In the absence of publication bias, studies should be scattered symmetrically around the most accurate studies in a pyramid fashion. In the present meta-analysis, the occurrence of publication bias was tested by conducting a regression test for funnel plot asymmetry within relatively homogenous subsets of studies (Egger et al., 1997). To correct for a possible publication bias, the trim-and-fill method was applied to the same subsets (Duval and Tweedie, 2000). This method estimates the number of missing studies and provides an adjustment of the overall effect size.

The effect sizes (k = 27) ranged from g = −0.08 to 1.29. The random effects model yielded a significant average effect size (g = 0.307; p < 0.0001; 95% CI = 0.200, 0.414). The majority of the effect sizes were in the expected direction (k = 25). Twelve of these positive effect sizes were significant. Two studies showed an effect in the opposite direction, one of which was significant. The estimated amount of heterogeneity was equal to τ² = 0.057; 95% CI = 0.029, 0.148. There was a clear indication of heterogeneity in effect sizes (Q = 183.24, df = 26, p < 0.0001). Influence analysis identified three outliers, g = 1.29 (standardized residual = 0.983; Markman and Brendl, 2005; A), g = 1.06 (standardized residual = 0.755; Markman and Brendl, 2005; B), g = 0.8 (standardized residual = 0.501; Phaf and Rotteveel, 2009; Experiment 2A). The exclusion of the three outliers reduced the average effect size (g = 0.216, p < 0.0001; 95% CI = 0.141, 0.292). The unexplained variance component was reduced (τ²= 0.0172), but the test of heterogeneity was still significant (Q = 93.03, df = 23, p < 0.0001). All moderator analyses were conducted under exclusion of the three outliers, except for the analysis of the moderator valence, because one outlier (Phaf and Rotteveel, 2009; A) was the only study using stimuli with implicit valence.

Four moderator variables (task, instruction, stimulus type, design) were included in a mixed effects model. The estimated amount of residual heterogeneity was equal to τ² = 0.0000; 95% CI = 0.0000, 0.0044, suggesting that at least 74% of the variance in effect sizes could be accounted for by including the moderators (Q_m = 83.4099, df = 7, p < 0.0001). The test for residual heterogeneity was not significant (Q_e = 9.6185, df = 16, p = 0.8858).

The results of the moderator analyses are provided in Table 1. The abstract-manikin task did not occur in the studies investigating positive affect separately. The test of the moderator task was significant (Q_m = 6.54, df = 2, p = 0.038). The average effect size was significantly different from zero for the vertical stand (g = 0.272; p < 0.001; 95% CI = 0.113, 0.432), as well as for the joystick/lever (g = 0.251; p < 0.0001; 95% CI = 0.165, 0.337). The feedback-joystick did not yield a significant effect (g = 0.047; p = 0.519; 95% CI = −0.095, 0.189), which is probably a consequence of many studies using the feedback-joystick in combination with implicit instructions. There was a significant difference between the feedback-joystick and the vertical stand (p = 0.039), and between the feedback-joystick and the joystick/lever (p = 0.016). The moderator task explained 35% of the variance (τ² = 0.011). The test for residual heterogeneity was significant (Q_e = 38.70, df = 21, p = 0.011).

The test of the moderator instruction was significant (Q_m = 17.62, df = 2, p = 0.0001). The average effect size differed significantly from zero for explicit instructions (g = 0.287; p < 0.0001; 95% CI = 0.204, 0.369) and for explicit-converted instructions (g = 0.287; p < 0.0001; 95% CI = 0.146, 0.429), but not for implicit instructions (g = 0.028; p = 0.572). The average effect size was significantly smaller for implicit instructions than for explicit instructions (p < 0.0001) and for explicit-converted instructions (p = 0.003). The moderator instruction explained 66% of the variance. The test for residual heterogeneity was not significant (Q_e = 27.75, df = 21, p = 0.147).

Considering the levels moderator stimulus type separately, the average effect size was significant for emotional words (g = 0.339; p < 0.0001; 95% CI = 0.211, 0.467), as well as for emotional pictures (g = 0.203; p = 0.028; 95% CI = 0.022, 0.383), and for emotional facial expressions (g = 0.148; p = 0.002; 95% CI = 0.056, 0.241). Only the difference between emotional words and facial expressions was significant (p = 0.018), which might also be due to many studies using facial expressions in combination with implicit instructions. However, the omnibus test of the moderator was not significant (Q_m = 5.62, df = 2, p = 0.060).

The test of the moderator design was not significant (Q_m = 1.34, df = 1, p = 0.247), presumably due to the low number of studies having an independent groups design. The test of the moderator valence was significant (Q_m = 12.27, df = 1, p < 0.001). Only one study used implicitly affective stimulus material (i.e., arrows) and showed a significantly larger effect (g = 0.808; p < 0.0001; 95% CI = 0.486, 1.130) than all other studies (g = 0.216; p < 0.0001; 95% CI = 0.141, 0.292). The moderator valence explained 45% of the variance. The test for residual heterogeneity was significant (Q_e = 93.03, df = 23, p < 0.0001).

Because the above moderator analyses indicated no significant effect for implicit instructions, some effect sizes may have been underestimated, when relatively many studies at this moderator level had such instructions. It therefore seems worthwhile to examine interactions between moderators. For this purpose, however, there needs to be at least one observation for every combination of moderator levels, which was not the case in this dataset, particularly for the implicit instructions. Therefore, we performed analyses under exclusion of all seven studies using implicit instructions.

After exclusion of studies with implicit instructions, the random effects model yielded a significant average effect size (g = 0.283; p < 0.0001; 95% CI = 0.216, 0.348). The estimated amount of heterogeneity equaled τ² = 0.0037; 95% CI = 0, 0.0211. The test for heterogeneity was not significant (Q = 17.74, df = 16, p = 0.340), confirming that most of the heterogeneity was due to differences in average effect sizes between task-irrelevant instructions (implicit) and task-relevant instructions (explicit and explicit-converted).

As expected, moderator analyses for the subset of task-relevant instructions (see Table 2) showed that no moderator was significant (except for the moderator stimulus valence). This suggests that differences in effect size between levels of the moderators appeared due to the inclusion of implicit instructions. Specifically, the average effect size was now significant for the feedback-joystick (g = 0.233, p = 0.035, 95% CI = 0.016, 0.451). The average effect size was not significantly different from the joystick/lever (p = 0.690) and the vertical stand (p = 0.463). The same applies to the moderator stimulus type. There was no statistically significant difference in effect size between emotional facial expressions and emotional words (p = 0.707).

Effect sizes (k = 32) ranged from g = −0.13 to 1.85. Four studies showed an effect in a direction opposite to the expected one. However, none of these effect sizes was significant. The remaining effect sizes were in the expected direction (k = 28). Similar to the analysis of positive affect, the random effects model yielded a significant average effect size (g = 0.304; p < 0.0001; 95% CI = 0.174, 0.435). The estimated amount of heterogeneity was equal to τ² = 0.122; 95% CI = 0.082, 0.306. There was a clear indication of heterogeneity in effect sizes (Q = 189.97, df = 31, p < 0.0001). Influence analysis identified two outliers, g = 1.85 (standardized residual = 1.543; Markman and Brendl, 2005; D), g = 1.76 (standardized residual = 1.457; Markman and Brendl, 2005; C). The exclusion of the two outliers resulted in a reduced average effect size (g = 0.217; p < 0.0001; 95% CI = 0.141, 0.292). The unexplained variance component was reduced (τ² = 0.029), but the test for heterogeneity was still significant (Q = 104.32, df = 29, p < 0.0001). Moderator analyses were conducted under exclusion of the two outliers.

All five moderator variables (task, instruction, stimulus type, design, valence) were included in a mixed effects model. The abstract-manikin level again did not occur in the studies that investigated negative affect separately. The results of the moderator analyses are provided in Table 3. The estimated amount of residual heterogeneity was equal to τ² = 0.0234. However, the test of the moderators was not significant (Q_m = 13.7224, df = 9, p = 0.1325). There was substantial residual heterogeneity (Q_e = 56.8393, df = 20, p < 0.0001). Separate analyses of each moderator confirmed that no moderator was significant, which means that none of the levels differed significantly from the other levels of the same moderator.

The test of the moderator task was not significant (Q_m = 0.26, df = 2, p = 0.876). The average effect size differed significantly from zero for the joystick/lever (g = 0.235; p < 0.0001; 95% CI = 0.126, 0.344), for the feedback-joystick (g = 0.212; p = 0.007; 95% CI = 0.057, 0.367), and for the vertical stand (g = 0.184; p = 0.027; 95% CI = 0.021, 0.347).

The omnibus test for moderation due to different instructions was close to significance (Q_m = 5.91, df = 2, p = 0.052). Effect sizes differed significantly from zero for explicit-converted instructions (g = 0.389; p = 0.001; 95% CI = 0.155, 0.624) and for explicit instructions (g = 0.249; p < 0.0001; 95% CI = 0.159, 0.339). Implicit instructions did not yield a significant effect (g = 0.103; p = 0.0959). Results indicated a significant difference between implicit and explicit-converted instructions (p = 0.034) and there was a trend toward a significant difference between implicit and explicit instructions (p = 0.059). There was no difference between explicit and explicit-converted instructions (p = 0.276). This pattern is consistent with the results of the analysis of positive affect. In order to increase power, the two levels of explicit and explicit-converted instructions were combined to form the level task-relevant instructions. The test of this moderator was significant (Q_m = 4.74, df = 1, p = 0.030). Task-relevant instructions showed a significant average effect size (g = 0.267; p < 0.0001; 95% CI = 0.183, 0.351), whereas task-irrelevant (implicit) instructions did not (g = 0.103; p = 0.096; 95% CI = −0.018, 0.225).

The omnibus test of the moderator stimulus type was not significant (Q_m = 5.58, df = 3, p = 0.134). The effect size was significantly different from zero for personally relevant stimuli (g = 0.348; p = 0.005; 95% CI = 0.107, 0.589), for emotional pictures (g = 0.322; p < 0.001; 95% CI = 0.143, 0.502), for emotional words (g = 0.277; p < 0.001; 95% CI = 0.119, 0.434), and for emotional facial expressions (g = 0.134; p = 0.009; 95% CI = 0.034, 0.235). The test of the moderator design was not significant (Q_m = 0.49, df = 1, p = 0.4824). One study used implicitly affective stimulus material (i.e., arrows; Phaf and Rotteveel, 2009). The effect size for this study was not significantly different from the effect size of all other studies (Q_m = 2.84, df = 1, p = 0.092).

After recoding the moderator instruction, results showed a significant difference between task-relevant and implicit instructions. Although all other moderators were not significant, the magnitude of the average effect size for each level might depend on which instructions were used. Therefore, subset analyses were performed under exclusion of all studies employing implicit instructions.

After exclusion of 10 studies with implicit instructions the random effects model yielded a significant average effect size (g = 0.265; p < 0.0001; 95% CI = 0.188, 0.343). The estimated amount of heterogeneity was equal to τ² = 0.018; 95% CI = 0.006, 0.062. The test for heterogeneity in effect sizes was significant (Q = 60.92, df = 19, p < 0.0001).

Excluding studies with implicit instructions did not affect the significance of any of the omnibus moderator tests. The magnitude of some average effect sizes was increased, however, due to the exclusion of implicit instructions (see Table 4). Differences between levels of the moderator task were numerically reduced. One study using personally relevant stimuli, moreover showed a considerably larger effect size (g = 0.769, p = 0.006, 95% CI = 0.222, 1.316).

In order to investigate the impact of the correlation between pairs of observations, sampling variances were computed based on the two most extreme correlations. This was done separately for each analysis. For the analysis of positive affect, the lowest correlation derived from test-statistics was equal to r = 0.479 (Seidel et al., 2010b; A). The highest correlation was equal to r = 0.797 (Van Dantzig et al., 2008; A). The results from the random effects model with the lowest correlation (g = 0.310; 95% CI = 0.201, 0.420; Q = 181.17; τ² = 0.0584) were very similar to those with the highest correlation (g = 0.299; 95% CI = 0.199, 0.400; Q = 192.65; τ² = 0.0546).

For the analysis of negative affect, the lowest correlation derived from test-statistics was equal to r = 0.403 (Rinck and Becker, 2007; Study 1). The highest correlation was equal to r = 0.966 (Van Dantzig et al., 2008; B). The results from the random effects model with the lowest correlation (g = 0.313; 95% CI = 0.172, 0.453; Q = 166.76; τ² = 0.1297) were very similar to those with the highest correlation (g = 0.300; 95% CI = 0.170, 0.430; Q = 440.74; τ² = 0.1267). In sum, these sensitivity analyses suggest that the estimated mean effect sizes are hardly affected by the use of alternative imputations of the correlations between pairs of observations in within-participants designs. All studies in the both affects analysis reported the relevant test statistic, so that the correlations and effect sizes could be estimated here with a relatively large precision.

The effect sizes (k = 22) ranged from g = 0.002 to 0.87. All effect sizes were in the expected direction. Eighteen of these effect sizes were significant. Influence analysis identified no outliers. The random effects model yielded a significant average effect size (g = 0.308; p < 0.0001; 95% CI = 0.205, 0.410). The estimated amount of heterogeneity was equal to τ² = 0.045; 95% CI = 0.0213, 0.1107. The test for heterogeneity was significant (Q = 133.13, df = 21, p < 0.0001).

The same five moderators were analyzed in a mixed effects model. The results of the analyses are provided in Table 5. The estimated amount of residual heterogeneity was equal to τ²= 0.0007; 95% CI = 0, 0.0436, suggesting that 98.4% of the systematic variance in effect sizes could be accounted for by including the moderators (Q_m = 104.4251, df = 9, p < 0.0001). The test for residual heterogeneity was not significant (Q_e = 15.8157, df = 12, p = 0.1998).

The test of the moderator task was not significant (Q_m = 3.44, df = 3, p = 0.328). The average effect size differed significantly from zero for the vertical stand (g = 0.662; p = 0.004; 95% CI = 0.212, 1.11), the joystick/lever (g = 0.305; p < 0.0001; 95% CI = 0.187, 0.423), and the abstract-manikin task (g = 0.281; p = 0.028; 95% CI = 0.030, 0.532). The feedback-joystick task did not yield a significant effect (g = 0.094; p = 0.659).

The test of the moderator instruction was significant (Q_m = 23.71, df = 2, p < 0.0001). The average effect size differed significantly from zero for explicit-converted instructions (g = 0.433; p < 0.0001; 95% CI = 0.295, 0.571) and explicit instructions (g = 0.403; p < 0.0001; 95% CI = 0.286, 0.521). Their difference was not significant (p = 0.747). Implicit instructions did not yield a significant effect (g = 0.076; p = 0.148). The average effect size was significantly smaller with implicit instructions than with explicit instructions (p < 0.0001) and explicit-converted instructions (p < 0.0001). The moderator instruction explained 67% of the variance (τ² = 0.0150). The test for residual heterogeneity was significant (Q_e = 50.56, df = 21, p = 0.0001).

The test of the moderator stimulus type was not significant (Q_m = 1.70, df = 2, p = 0.429). The average effect size differed significantly from zero for emotional words (g = 0.343; p < 0.0001; 95% CI = 0.215, 0.472), and for emotional pictures (g = 0.321; p = 0.012; 95% CI = 0.071, 0.571), but not for emotional facial expressions (g = 0.146; p = 0.286). The test of the moderator design was not significant (Q_m = 2.49, df = 1, p = 0.115). Two studies employed implicitly affective stimulus material. The average effect size for these studies was not significantly different from the average effect size of all other studies (Q_m = 0.420, df = 1, p = 0.517).

The moderator analyses so far have demonstrated consistently that task-relevant (explicit or explicit-converted) instructions are required to find an effect of affective information processing on approach-avoidance behaviors. Accordingly, the analysis of both affects also showed a non-significant effect for task-irrelevant (implicit) instructions. Again, subset analyses were performed under exclusion of all studies using implicit instructions in order to investigate how they might affect the results (see Table 6). However, inferences are based on even fewer studies and should therefore be treated with caution. After exclusion of the six studies with implicit instructions the random effects model yielded a significant medium average effect size (g = 0.425; p < 0.0001; 95% CI = 0.317, 0.533). The estimated amount of heterogeneity was equal to τ² = 0.0283; 95% CI = 0.007, 0.088. The test for heterogeneity in effect sizes was significant (Q = 44.98, df = 15, p < 0.0001).

The test of the moderator task now reached significance (Q_m = 6.22, df = 2, p = 0.045). The level feedback-joystick was dropped, because the only study in this level used implicit instructions. There was only one study that used the abstract-manikin task (g = 0.729, p < 0.0001, 95% CI = 0.373, 1.084). This effect size was almost significantly larger (p = 0.056) than for the joystick/lever (g = 0.368, p < 0.0001, 95% CI = 0.2681, 0.468). The moderator task explained 42% of the variance (τ² = 0.0164). The test for residual heterogeneity was significant (Q_e = 27.00, df = 13, p = 0.0124). The test of the moderator stimulus type was not significant (Q_m = 2.85, df = 2, p = 0.240). Based on two studies, the average effect size for emotional facial expressions was still not significant (g = 0.210, p = 0.138, 95% CI = 0.067, 0.487).

So far, instruction seems to be a crucial factor. When combining effect sizes from the three affect levels, task-relevant instructions (i.e., explicit and explicit-converted) showed a medium effect size (g = 0.3182) and task-irrelevant instructions (i.e., implicit) had a negligible effect size (g = 0.0644). To investigate the possibility of the former being caused by a publication bias, we prepared funnel plots for the three affect analyses, excluding the implicit-instruction effects and the previously identified outliers (see Figures 1–3). Visual inspection already suggests that the plots are asymmetrical with more studies with a large effect and a large standard error to the right of the mean than the left of the mean. To test for a publication bias, we followed the approach used by Bakker et al. (2012; cf. Francis, 2012), which involves the use of Egger's regression test and Ioannidis and Trikalinos (2007) test of an excess of significant outcomes. In addition, we applied the trim and fill method (Duval and Tweedie, 2000) to correct for potential funnel plot asymmetry due to publication bias.

The funnel plot of positive-affect studies (k = 17) is given in Figure 1 and appears to be asymmetric. Indeed, Egger's regression test was significant at α = 0.10 (which is the commonly used nominal significance level for these analyses; Bakker et al., 2012; Francis, 2012): Z = 1.87, p = 0.062. The use of trim and fill suggested seven missing studies on the left-hand side of the funnel plot, which lowered the estimated effect to 0.212 (95% CI: 0.136, 0.288). Power computations on the basis of the estimated effect size (i.e., g = 0.282) showed that the average power of the 17 studies was 0.54. On the basis of this power calculation, one would expect 9.2 significant outcomes. Given that nine of studies showed a significant outcome, there does not appear to be an excess of significant outcomes in this set of studies.

In the analysis of negative affect (k = 20; see Figure 2), Egger's regression test also indicated funnel plot asymmetry: Z = 2.22, p = 0.027. The use of trim and fill suggested three missing studies, which lowered the estimated effect from 0.260 to 0.231 (95% CI: 0.148, 0.314). Power computations on the basis of the estimated effect size (i.e., 0.260) provided a mean power over studies of 0.61. On the basis of that power calculation the expected number of significant outcomes was 12.2. Given that 13 of the studies concerning negative affect showed a significant outcome, there is no clear excess of significant outcomes in this set of studies.

Figure 3 depicts the funnel plot with effect sizes related to both affects (k = 16). Egger's regression test again highlighted an asymmetric funnel plot: Z = 1.73, p = 0.083. Trim and fill suggested five missing studies on the left-hand side of the funnel, which led to an estimated effect of 0.344 (95% CI: 0.230, 0.458). In this subset, the power analyses on the basis of the uncorrected mean effect size (g = 0.425) showed that the power averaged 0.78. On this basis, 12.4 significant outcomes are to be expected, which compares well to the dozen significant outcomes in this subset of studies. Hence, there does not appear to be an excess of significant outcomes in this analysis.

Taken together, there are some indications of publication bias in all three sets of effects. The trim and fill corrections led to lower estimate effect sizes, which all remained significant and equaled 0.21 for positive effect, 0.23 for negative affect, and 0.34 for both affects.

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.