AUTHOR=Alsuradi Haneen , Eid Mohamad 

TITLE=An ensemble deep learning approach to evaluate haptic delay from a single trial EEG data

JOURNAL=Frontiers in Robotics and AI

VOLUME=Volume 9 - 2022

YEAR=2022

URL=https://www.frontiersin.org/journals/robotics-and-ai/articles/10.3389/frobt.2022.1013043

DOI=10.3389/frobt.2022.1013043

ISSN=2296-9144

ABSTRACT=Haptic technologies are becoming increasingly valuable for teleoperation systems as they provide means of physical interaction with a remote or virtual environment. . One of the persistent challenges in teleoperation systems is the synchrony of the delivered haptic information with the rest of the sensory modalities. Delayed haptic feedback can have serious implications on the user performance and overall experience. Limited research efforts have been devoted to studying the implication of haptic delay on the human neural response and relating it to the overall haptic experience. Deep learning could offer autonomous brain activity interpretation in response to a haptic experience such as haptic delay. In this work, we propose an ensemble of 2D CNN and transformer models that is capable of detecting the presence and amount of a perceived haptic delay from a single-trial EEG data. We design and conduct two EEG-based experiments involving visuo-haptic manipulation tasks. The first aims to collect data for detecting the presence of haptic delay during discrete force feedback stimulation, while the second aims to collect data for detecting the level (none, mild, moderate, severe) of the perceived haptic delay during a continuous force feedback stimulation. The ensemble model showed a promising performance with an accuracy of 0.9142±0.0157 and 0.6625±0.0067 for the first and second experiments, respectively. These results were obtained based on training the model with raw EEG data as well as their wavelet transform using several wavelet kernels.