GaborNet Visual Encoding: A Lightweight Region-Based Visual Encoding Model With Good Expressiveness and Biological Interpretability - Cui, Yibo; Qiao, Kai; Zhang, Chi; Wang, Linyuan; Yan, Bin; Tong, Li (Frontiers in Neuroscience, 2021)
Computational visual encoding models play a key role in understanding the stimulus-response characteristics of neuronal populations in the brain visual cortex. However, building such models typically faces challenges in the effective construction of nonlinear feature spaces to fit the neuronal responses. In this work, we propose the GaborNet visual encoding (GaborNet-VE) model, a novel end-to-end encoding model for the visual ventral stream. This model is composed of a Gabor convolutional layer, two regular convolutional layers, and a fully-connected layer. The key design principle for the GaborNet-VE model is to replace regular convolutional kernels in the first convolutional layer by Gabor kernels with parameters whose values can be learned through model training. The GaborNet-VE model efficiently and simultaneously encodes all voxels in one region of interest (ROI) of functional magnetic resonance imaging (fMRI) data. The experimental results show that the proposed model achieves state-of-the-art prediction performance for the primary visual cortex. Moreover, the visualizations demonstrate the regularity of ROI fitting to the visual features and the estimated receptive fields (RFs). These results suggest that the lightweight region-based GaborNet-VE model based on combining hand-crafted and deep features exhibits good expressiveness and biological interpretability.