Model-Free RL or Action Sequences? - Morris, Adam; Cushman, Fiery (Frontiers in Psychology, 2019)
The alignment of habits with model-free reinforcement learning (MF RL) is a success story for compu- tational models of decision making, and MF RL has been applied to explain phasic dopamine responses (Schultz, Dayan, & Montague, 1997), working memory gating (O’Reilly & Frank, 2006), drug addiction (Redish, 2004), moral intuitions (Cushman, 2013; Crockett, 2013), and more. Yet, the role of MF RL has recently been challenged by an alternate model—model-based selection of chained action sequences—that produces similar behavioral and neural patterns. Here, we present two experiments that dissociate MF RL from this prominent alternative, and present unconfounded empirical support for the role of MF RL in human decision making. Our results also demonstrate that people are simultaneously using model-based selection of action sequences, thus demonstrating two distinct mechanisms of habitual control in a common experimental paradigm. These findings clarify the nature of habits and help solidify MF RL’s central position in models of human behavior.