AUTHOR=Wang Meifang , Liang Zhange 

TITLE=Cross-modal self-attention mechanism for controlling robot volleyball motion

JOURNAL=Frontiers in Neurorobotics

VOLUME=Volume 17 - 2023

YEAR=2023

URL=https://www.frontiersin.org/journals/neurorobotics/articles/10.3389/fnbot.2023.1288463

DOI=10.3389/fnbot.2023.1288463

ISSN=1662-5218

ABSTRACT=This study focuses on the field of robot control, exploring the application of robots in volleyball tasks through a dynamic neural network-driven cross-modal self-attention mechanism. The emergence of cross-modal perception and deep learning technologies has had a profound impact on the field of modern robotics. The primary objective of this study is to achieve precise control of robots in volleyball tasks by effectively integrating information from different sensors using a crossmodal self-attention mechanism. Our approach utilizes a cross-modal self-attention mechanism to integrate information from various sensors, providing robots with a more comprehensive scene perception. Additionally, we employ Generative Adversarial Networks (GANs) to synthesize realistic volleyball scenarios, enhancing the diversity and practicality of robot training. Furthermore, we leverage transfer learning to incorporate knowledge from other sports datasets, enriching the process of skill acquisition for robots. To validate the feasibility of this approach, we simulate robot volleyball scenarios using multiple volleyball-related datasets and measure metrics such as accuracy, recall, precision, and F1 score using quantitative data from experiments. Experimental results indicate a significant enhancement in the performance of our approach in robot volleyball tasks. This study provides valuable insights into the application of multi-modal perception and deep learning in the field of sports robotics, while also opening up new possibilities for humanrobot collaboration and athletic performance improvement.