Continuous Predictors of Pretest-Posttest Change: Highlighting the Impact of the Regression Artifact - Farmus, Linda; Arpin-Cribbie, Chantal A.; Cribbie, Robert A. (Frontiers in Applied Mathematics and Statistics, 2019)
Researchers are often interested in exploring predictors of change, and commonly use a regression based model or a gain score analysis to compare degree of change across groups. Methodologists have cautioned against the use of the regression based model when there are non-random group differences at baseline because this model inappropriately corrects for baseline differences. Less research has addressed the issues that arise when exploring continuous predictors of change (e.g., a regression model with posttest as the outcome and pretest as a covariate). If continuous predictors of change correlate with pretest scores, the modeled relationship between predictors and change may be an artefact. This two-part study explored the statistical artefact that may arise when continuous predictors of change are included in pretest-posttest regression based models. Study 1 revealed that the problematic regression based model is currently being applied in psychology literature more often than non-problematic models, and that the conditions leading to the artefact were met in a significant amount of studies reviewed. Study 2 demonstrated that the artefact arises in conditions common within psychological research, and directly impacts Type I error rates. Recommendations are provided regarding which regression based models are appropriate when pretest scores are correlated with continuous predictors.