AUTHOR=Li Bin , Cao Hu , Qu Zhongnan , Hu Yingbai , Wang Zhenke , Liang Zichen 

TITLE=Event-Based Robotic Grasping Detection With Neuromorphic Vision Sensor and Event-Grasping Dataset

JOURNAL=Frontiers in Neurorobotics

VOLUME=Volume 14 - 2020

YEAR=2020

URL=https://www.frontiersin.org/journals/neurorobotics/articles/10.3389/fnbot.2020.00051

DOI=10.3389/fnbot.2020.00051

ISSN=1662-5218

ABSTRACT=Robotic grasping plays an important role in the field of robotics. The current state-of-the-art robotic grasping detection systems are usually built on the conventional vision, such as RGB-D camera. Compared to traditional frame-based computer vision, neuromorphic vision is a small and young community of research. Currently, there are limited event-based datasets due to the troublesome annotation of the asynchronous event stream. Annotating large scale vision dataset often takes lots of computation resources, especially the troublesome data for video-level annotation. In this work, we consider the problem of detecting robotic grasps in a moving camera view of a scene containing objects. To obtain more agile robotic perception, a neuromorphic vision sensor (DAVIS) attaching to the robot gripper is introduced to explore the potential usage in grasping detection. We construct a robotic grasping dataset named Event-Stream Dataset with 91 objects. For each object, there are $4020$ successive grasping annotations in different views with a time resolution of $1$ ms. A spatio-temporal mixed particle filter (SMP Filter) is proposed to track the led-based grasp rectangles which enables video-level annotation of a single grasp rectangle per object. As leds blink at high frequency, the Event-Stream dataset is annotated in a high frequency of 1 kHz. Based on the Event-Stream dataset, we develop a deep neural network for grasping detection which consider the angle learning problem as classification instead of regression. The method performs high detection accuracy on our Event-Stream dataset with 93% precision at object-wise level. This work provides a large-scale and well-annotated dataset, and promotes the neuromorphic vision applications in agile robot.