Prioritized experience replay in path planning via multi-dimensional transition priority fusion - Cheng, Nuo; Wang, Peng; Zhang, Guangyuan; Ni, Cui; Nematov, Erkin (Frontiers in Neurorobotics, 2023)
Deep deterministic policy gradient (DDPG)-based path planning algorithms for intelligent robots struggle to discern the value of experience transitions during training due to their reliance on a random experience replay. This can lead to inappropriate sampling of experience transitions and overemphasis on edge experience transitions. As a result, the algorithm's convergence becomes slower, and the success rate of path planning diminishes. We comprehensively examines the impacts of immediate reward, temporal-difference error (TD-error), and Actor network loss function on the training process. It calculates experience transition priorities based on these three factors. Subsequently, using information entropy as a weight, the three calculated priorities are merged to determine the final priority of the experience transition. In addition, we introduce a method for adaptively adjusting the priority of positive experience transitions to focus on positive experience transitions and maintain a balanced distribution. Finally, the sampling probability of each experience transition is derived from its respective priority. Experimental results indicated that the determined experience transition priority accurately gauges the significance of distinct experience transitions for path planning algorithm training. This method enhances the utilization rate of transition conversion and the convergence speed of the algorithm and also improves the success rate of path planning.