E2VIDX: improved bridge between conventional vision and bionic vision - Hou, Xujia; Zhang, Feihu; Gulati, Dhiraj; Tan, Tingfeng; Zhang, Wei (Frontiers in Neurorobotics, 2023)
Common RGBD, CMOS, and CCD-based cameras produce motion blur and incorrect exposure under high-speed and improper lighting conditions. The event camera developed according to the bionic principle has the advantages of low delay, high dynamic range and no motion blur.However, due to its unique data representation, it encounters significant obstacles in practical applications. The image reconstruction algorithm based on an event camera solves the problem by converting a series of "events" into common frames to apply existing vision algorithms. Thanks to the rapid development of neural networks, this field has made significant breakthroughs in past years. Based on the most popular Events-to-Video (E2VID) method, this paper designs a new network called E2VIDX. The proposed network includes group convolution and sub-pixel convolution, which not only achieves better feature fusion but also the network model size is reduced by 25%. Futhermore, we propose a new loss function. The loss function is divided into two parts, first which calculates the high level features and the second which calculates the low level features of the reconstructed image. The experimental results clearly outperform against the state-of-the-art. Compared with the original method, Structural Similarity (SSIM) increases by 1.3%, Learned Perceptual Image Patch Similarity (LPIPS) decreases by 1.7%, Mean Squared Error (MSE) decreases by 2.5%, and it runs faster on GPU and CPU. Additionally, we evaluate the results of E2VIDX with application to image classification, object detection and instance segmentation. The experiments show that conversions using our method can help event cameras directly apply existing vision algorithms in most scenarios.