A Novel Rain Removal Approach for Outdoor Dynamic Vision Sensor Event Videos - Cheng, Long; Liu, Ni; Guo, Xusen; Shen, Yuhao; Meng, Zijun; Huang, Kai; Zhang, Xiaoqin (Frontiers in Neurorobotics, 2022)
As bio-inspired vision devices, dynamic vision sensors (DVS) are being applied in more and more applications. Unlike normal cameras, pixels in \dvs independently respond to the luminance change with asynchronous output spikes. Therefore, removing raindrops and streaks from \dvs event videos is a new but challenging task as the conventional deraining methods are no longer applicable. In this paper, we propose to perform the deraining process in the width and time (W-T) space. This is motivated by the observation that rain steaks exhibit discontinuity in the width and time directions while background moving objects are usually piecewise smooth along with both directions. The W-T space can fuse the discontinuity in both directions and thus transforms raindrops and streaks into approximately uniform noise that is easy to remove. The non-local means filter is adopted as background object motion has periodic patterns in the W-T space. A repairing method is also designed to restore edge details erased during the deraining process. Experimental results demonstrate that our approach can better remove rain noise than the four existing methods for traditional camera videos. We also study how the event buffer depth and event frame time affect the performance and investigate the potential implementation of our approach to classic RGB images. A new real-world database for DVS deraining is also created and shared for public use.