Learning Generative State Space Models for Active Inference - Çatal, Ozan; Wauthier, Samuel; De Boom, Cedric; Verbelen, Tim; Dhoedt, Bart (Frontiers in Computational Neuroscience, 2020)
In this paper we investigate the active inference framework as a means to enable intelligent behaviour in artificial agents. Active inference might underpin the way organisms act and observe in the real world. In active inference, agents act in order to minimize their so called free energy, or prediction error. Besides being biologically plausible, active inference has been shown to solve hard exploration problems in various simulations. However, these simulations typically require handcrafting a generative model for the agent. Therefore we propose to use recent advances in deep artificial neural networks to learn generative state space models from scratch, using only observation-action sequences. This way we are able to scale active inference to new and challenging problem domains, whilst still building on the theoretical backing of the free energy principle. We validate our approach on the mountain car problem to illustrate that our learnt models can indeed trade-off instrumental value and ambiguity. Furthermore, we show that generative models can also be learnt using high-dimensional pixel observations, both in the OpenAI Gym car racing environment and a real-world robotic navigation task.