Recurrent Spiking Neural Network Learning Based on a Competitive Maximization of Neuronal Activity - Demin, Vyacheslav; Nekhaev, Dmitry (Frontiers in Neuroinformatics, 2018)
Spiking neural networks (SNNs) are believed to be highly computationally and energy efficient 5 for specific neurochip hardware real-time solutions. However, there is a lack of learning algorithms 6 for complex SNNs with recurrent connections, comparable in efficiency with the back-propagation 7 techniques and capable of unsupervised training. Here we suppose that each neuron in a 8 biological neural network tends to maximize its activity in competition with other neurons and 9 put this principle at the basis of a new SNN learning algorithm. In such a way a spiking network 10 with the learned feed-forward, reciprocal and intralayer inhibitory connections is introduced for 11 the MNIST database digit recognition. It is demonstrated that this SNN can be trained without a 12 teacher after a short supervised initialization of weights by the same algorithm. Also, it is shown 13 that neurons are grouped into families of hierarchical structure corresponding to different digit 14 classes and their associations. This property is expected to be useful for reducing the number 15 of layers in deep neural networks and modeling the formation of various functional structures in 16 a biological nervous system. Comparison of the learning properties of the suggested algorithm 17 with those of the Sparse Distributed Representation approach shows similarity in coding but also 18 some advantages of the former. The basic principle of the proposed algorithm is believed to be 19 practically applicable to the construction of much more complicated and diverse task solving 20 SNNs. We refer to this new approach as ‘Family-Engaged Execution and Learning of Induced 21 Neuron Groups’, FEELING for short.