A Method for Measuring Contact Points in Human–Object Interaction Utilizing Infrared Cameras - Hakala, Jussi; Häkkinen, Jukka (Frontiers in Robotics and AI, 2022)
This article presents a novel method for measuring contact points in human–object interaction. Research in multiple prehension-related fields, e.g., action planning, affordance, motor function, ergonomics, and robotic grasping, benefits from accurate and precise measurements of contact points between a subject’s hands and objects. During interaction, the subject’s hands occlude the contact points, which poses a major challenge for direct optical measurement methods. Our method solves the occlusion problem by exploiting thermal energy transfer from the subject’s hand to the object surface during interaction. After the interaction, we measure the heat emitted by the object surface with four high-resolution infrared cameras surrounding the object. A computer-vision algorithm detects the areas in the infrared images where the subject’s fingers have touched the object. A structured light 3D scanner produces a point cloud of the scene, which enables the localization of the object in relation to the infrared cameras. We then utilize the localization result to project the detected contact points from the infrared camera images to the surface of the object 3D model. Data collection with the method is fast, unobtrusive, contactless, markerless, and automated. The method enables accurate measurement of contact points in non-trivial complex objects. Furthermore, the method is extendable to measuring surface contact areas, or patches, instead of contact points. In this article, we present the method and sample grasp measurement results with publicly available objects.