A review of the applications of multi-agent reinforcement learning in smart factories - Bahrpeyma, Fouad; Reichelt, Dirk (Frontiers in Robotics and AI, 2022)
Smart factory as the heart of Industry 4.0, is the new paradigm for establishing advanced manufacturing systems and realize modern manufacturing objectives such as mass customization, automation, efficiency and self-organization all at once. Such manufacturing systems, however, are characterized by dynamic and complex environments where a large number of decisions should be made for smart components such as production machines and the material handling system in a real-time and optimal manner. AI offers the key intelligent control approaches in order to realize efficiency, agility and automation all at once. One of the most challenging problems faced in this regard is the presence of uncertainty, meaning that due to the dynamic nature of the smart manufacturing environments sudden seen or unseen events occur that should be handled in real-time. Due to the complexity and the high-dimentionality of smart factories, it is not possible to predict all the possible events and prepare appropriate scenarios to respond. Reinforcement learning is an AI technique that provides the intelligent control process to deal with such uncertainties. Due to the distributed nature of smart factories and the presence of multiple decision making components, Multi-agent Reinforcement Learning (MARL) should be incorporated instead of single agent reinforcement learning (SARL), which due to the complexities involved in the development process, attracted less attention. In this research, we will review the literature of the applications of MARL to tasks within a smart factory, and then demonstrate a mapping connecting smart factory attributes to the equivalent MARL features based on which we suggest MARL as one of the most effective approaches to implement the control mechanism for smart factories.