Adaptive visual–tactile fusion recognition for robotic operation of multi-material system - Ding, Zihao; Chen, Guodong; Wang, Zhenhua; Sun, Lining (Frontiers in Neurorobotics, 2023)
The application of robots in various industries is gradually shifting from mechanization to intelligence and high precision. The systems are typically composed of parts of different materials and place greater demands on the accuracy and comprehensiveness of target identification. Humans recognize the material world with a highly diverse perceptual system and quickly identify deformable objects through vision and touch to prevent slipping or excessive deformation during grasping. However, robot recognition technology mainly relies on visual sensors and lacks critical information such as object material, resulting in incomplete cognition. Therefore, it is believed that multimodal information fusion is the key to the development of robot recognition. Firstly, a method of converting tactile sequences to images is proposed to deal with the obstacles of information exchange between different modalities for vision and touch, which overcomes the problems of the noise and instability of tactile data. Subsequently, a visual-tactile fusion network framework based on an adaptive dropout algorithm is constructed, together with an optimal joint mechanism between visual information and tactile information established, to solve the problem of mutual exclusion or unbalanced fusion in traditional fusion methods. Finally, experiments show that the proposed method effectively improves robot recognition ability, and the classification accuracy is as high as 99.3%.