Multimodal Human-Exoskeleton Interface for Lower Limb Movement Prediction Through a Dense Co-Attention Symmetric Mechanism - Shi, Kecheng; Mu, Fengjun; Huang, Rui; Huang, Ke; Peng, Zhinan; Zou, Chaobin; Yang, Xiao; Cheng, Hong (Frontiers in Neuroscience, 2022)
A challenging task for the biological neural signal-based human-exoskeleton interface is to achieve accurate lower limb movement prediction of hemiplegic patients in rehabilitation training scenarios. The human-exoskeleton interface based on single-modal biological signals such as electroencephalography (EEG) is currently not mature in predicting movements, due to its unreliability. Multimodal human-exoskeleton interface is a very novel solution to this problem. This kind of interface normally combines the EEG signal with surface electromyography (sEMG) signal. However, their use for the lower limb movement prediction are still limited --- the connection between sEMG and EEG signals and the deep feature fusion between them are ignored. In this paper, a Dense con-attention mechanism-based Multimodal Enhance Fusion Network (DMEFNet) is proposed for predicting lower limb movement of hemiplegic patients. The DMEFNet introduces the con-attention structure to extract the common attention between sEMG and EEG signal features. To verify the effectiveness of DMEFNet, a sEMG and EEG data acquisition experiment and an incomplete asynchronous data collection paradigm are designed. The experimental results show that DMEFNet has a good movement prediction performance in both within-subject and cross-subject situations, reaching an accuracy of 82.96% and 88.44% respectively.