Multimodal bipedal locomotion generation with passive dynamics via deep reinforcement learning - Koseki, Shunsuke; Kutsuzawa, Kyo; Owaki, Dai; Hayashibe, Mitsuhiro (Frontiers in Neurorobotics, 2023)
Generating multimodal locomotion in underactuated bipedal robots requires control solutions that can achieve stability with input from a limited number of active actuators and facilitate motion patterns for drastically different dynamical modes, which is an extremely challenging optimization problem in locomotion-learning tasks. Thus far, multimodal locomotion exploiting body dynamics has not been achieved. This study provides a framework that reproduces multimodal bipedal locomotion through deep reinforcement learning (DRL). An underactuated bipedal model was developed based on a passive walker, and a controller was designed using DRL. By carefully planning the weight parameter settings of the DRL reward function during the learning process based on a curriculum learning method, the bipedal model successfully learned to walk, run, and perform gait transitions by adjusting a single command input. These results indicate that DRL can be applied to underactuated bipedal robots to generate various gaits with effective use of body dynamics. Despite being limited to physical simulation, this study provides evidence that a bipedal model is able to learn various motions, including walking, running, and gait transition, through DRL. Therefore, we believe that this study provides a framework that will enable walking and running with the efficient use of body dynamics in bipedal robots.