AUTHOR=Bolarinwa Joseph , Eimontaite Iveta , Mitchell Tom , Dogramadzi Sanja , Caleb-Solly Praminda 

TITLE=Assessing the Role of Gaze Tracking in Optimizing Humans-In-The-Loop Telerobotic Operation Using Multimodal Feedback

JOURNAL=Frontiers in Robotics and AI

VOLUME=Volume 8 - 2021

YEAR=2021

URL=https://www.frontiersin.org/journals/robotics-and-ai/articles/10.3389/frobt.2021.578596

DOI=10.3389/frobt.2021.578596

ISSN=2296-9144

ABSTRACT=A key challenge in achieving effective robot tele-operation is minimising tele-operator cognitive workload and fatigue. We set out to investigate the extent to which gaze tracking data can reveal how tele-operators interact with a system. In this paper we present an analysis of gaze tracking, captured as a robot tele-operator completes a multi-stage task: grasping and emptying the contents of a jar into a container. The task was repeated with different combinations of visual, haptic and verbal feedback. Our aim was to determine if tele-operation workload can be inferred by combining gaze duration, fixation count, task completion time, and the complexity of robot motion (measured as the sum of robot joint steps) at different stages of the task. Visual information of the robot workspace was captured using four cameras, positioned to capture the robot workspace from different angles. These camera views (top, right, front and left) were displayed through four quadrants (top-left, top-right, bottom-left, bottom-right quadrants) of the tele-operator video feedback computer screen, respectively.

We found that the gaze duration and fixation count were highly dependent on the stage of the task and the feedback modality utilized. The results revealed that combining feedback modalities reduced the cognitive workload (inferred by investigating the correlation between gaze duration, fixation count, task completion time, success or failure of task completion, and robot gripper trajectories), particularly in the task stages that require more precision. There was a significant positive correlation between gaze duration and complexity of robot joint movements. Participants’ gaze outside the areas of interest (distractions) was not influenced by feedback modalities. A learning effect was observed in the use of the controller for all participants as they repeated the task with different feedback combination scenarios. To design a system for tele-operation, applicable in healthcare, analysis of tele-operators’ gaze can help to understand the interaction between tele-operators and the system.