Research on reinforcement learning-based safe decision-making methodology for multiple unmanned aerial vehicles - Yue, Longfei; Yang, Rennong; Zhang, Ying; Zuo, Jialiang (Frontiers in Neurorobotics, 2023)
A system with multiple cooperating unmanned aerial vehicles (multi-UAV) can exploit the cooperative advantages of multiple UAVs to accomplish complicated tasks. Recent developments in deep reinforcement learning (DRL) offer good prospects for decision-making for multi-UAV systems. However, the safety and training efficiency of DRL still need to be improved before practical use. This paper presents a transfer-safe soft actor-critic (TSSAC) for multi-UAV decision-making. Decision-making by each UAV is modeled with constrained Markov decision process (CMDP), in which safety is constrained to maximize the return. The soft actor-critic-Lagrangian (SAC-Lagrangian) algorithm is combined with a modified Lagrange multiplier in the CMDP model. Moreover, parameter-based transfer learning is used to enable cooperative and efficient training of the tasks for the multi-UAV. Simulation experiments indicate that the proposed method can improve safety and training efficiency and allow the UAVs to adapt to a dynamic scenario.