An adaptive control framework based multi-modal information-driven dance composition model for musical robots - Xu, Fumei; Xia, Yu; Wu, Xiaorun (Frontiers in Neurorobotics, 2023)
Currently, most robot dances are precompiled, requirement of manual adjustment of relevant parameters and meta- action to change another type of music would greatly reduce its function. To overcome the gap, this study proposed a dance composition model for mobile robots based on multimodal information. The model consists of three parts. (1) Extraction of multimodal information. The temporal structure feature method of structure analysis framework is used to divide audio music files into music structures; then, a hierarchical emotion detection framework is used to extract information (rhythm, emotion, and tension, etc.) for each segmented music structure; calculating the safety of the current car and surrounding objects in motion; finally, extracting the stage color of the robot's location, corresponding to the relevant atmosphere emotions. (2) Initialize dance library. Dance composition is divided into four categories based on the classification of music emotions; in addition, each type of dance composition is divided into skilled composition and general dance composition. (3) The total path length can be obtained through combining multimodal information on the basis of different emotions, initial speeds, and music structure periods; then, target point planning can be carried out based on the specific dance composition selected. An adaptive control framework based on Cerebellar Model Articulation Controller (CMAC) and compensation controllers is used to track the target point trajectory, and finally the selected dance composition is formed. Mobile robot dance composition provides a new method and idea for humanoid robot dance composition.