AUTHOR=Wei Ming , Zhu Ming , Zhang Yaoyuan , Wang Jiarong , Sun Jiaqi 

TITLE=Real-time depth completion based on LiDAR-stereo for autonomous driving

JOURNAL=Frontiers in Neurorobotics

VOLUME=Volume 17 - 2023

YEAR=2023

URL=https://www.frontiersin.org/journals/neurorobotics/articles/10.3389/fnbot.2023.1124676

DOI=10.3389/fnbot.2023.1124676

ISSN=1662-5218

ABSTRACT=The fusion of multiple sensors is an inevitable trend in the development of automatic driving. The depth image obtained by stereo matching of binocular camera is easily influenced by environment and distance. The point cloud of LiDAR is more penetrating. However, it is much sparser than binocular images. LiDAR-Stereo fusion can neutralize the advantages of the two sensors and maximize the acquisition of reliable three-dimensional information to improve the safety of automatic driving. Cross-sensor fusion is the key issue. At present, the effective LiDAR-Stereo fusion network often regards the two as independent individuals and adopts 3D convolution to realize the fusion of different positions, which makes the time cost relatively large. In this paper, a real-time LiDAR-Stereo depth completion network without 3D convolution is proposed to fuse point clouds and binocular images by injection guidance. At the same time, a kernel connection spatial propagation network is utilized to refine the depth to obtain dense and effective 3D information that can be utilized in autonomous driving. The results of experiment on the KITTI autonomous driving dataset show that our method is real-time and effective. Further, we demonstrated our solution to the sensor defect problem and terrible environment on the p-KITTI dataset.