Koopman Operator–Based Knowledge-Guided Reinforcement Learning for Safe Human–Robot Interaction - Sinha, Anirban; Wang, Yue (Frontiers in Robotics and AI, 2022)
We develop a novel framework for deep reinforcement learning (RL) algorithms in task constrained path generation problems of robotic manipulators from the human demonstrated trajectories. The main contribution of this paper is to design a reward function that can be used with generic RL algorithms by utilizing the Koopman operator theory to build a human intent model from the human demonstrated trajectories. In order to ensure the developed reward function produces the correct reward, the demonstrated trajectories are further used to create a trust domain within which Koopman operator-based human intent prediction is considered. Otherwise, the proposed algorithm asks for human feedback to receive rewards from the human expert. The designed reward function is incorporated inside the deep Q-learning (DQN) framework, which results in a modified DQN algorithm. The effectiveness of the proposed learning algorithm is demonstrated using a simulated robotic arm to learn paths for constrained end-effector motion and considering the safety of the human in the surroundings of the robot.