AUTHOR=Schubert Fabian , Gros Claudius 

TITLE=Local Homeostatic Regulation of the Spectral Radius of Echo-State Networks

JOURNAL=Frontiers in Computational Neuroscience

VOLUME=Volume 15 - 2021

YEAR=2021

URL=https://www.frontiersin.org/journals/computational-neuroscience/articles/10.3389/fncom.2021.587721

DOI=10.3389/fncom.2021.587721

ISSN=1662-5188

ABSTRACT=Recurrent cortical network dynamics plays a crucial role for sequential information processing in the brain. While reservoir computing provides a conceptual basis for the understanding of recurrent neural computation, it requires manual adjustments of global network parameters, in particular of the spectral radius of the recurrent synaptic weight matrix. However, the spectral radius is not accessible to biological neural networks, which adhere to the principle that information about the network state is either encoded in local intrinsic dynamical quantities, or transmitted via synaptic connectivity. 
We present two synaptic scaling rules for echo state networks that rely on locally accessible variables. Both rules work online, in the presence of a continuous stream of input signals. The first rule, termed flow control, is based on a local comparison between the mean recurrent membrane potential and the activity of the neuron itself. It is derived from a global scaling condition on the dynamic flow of neural activities and requires the separability of external and recurrent input currents. The second rule, variance control, directly regulates the variance of neural activities by locally scaling the recurrent synaptic weights. The target set point of this homeostatic mechanism is dynamically determined as a function of the variance of the locally measured external input. 
The effectiveness of the presented mechanisms was tested numerically using different external input protocols. Network performance after adaptation was evaluated by training the network to perform a time delayed XOR operation on binary sequences. As our main result, we found that flow control can reliably regulate the spectral radius under different input statistics, but precise tuning is negatively affected by interneural correlations. Furthermore, it showed a consistent task performance over a wide range of input strengths/variances. Variance control, however, did not yield the desired spectral radii with the same precision. Moreover, task performance was less consistent across different input strengths. Given the better performance and simpler mathematical form of flow control, we concluded that a local control of the spectral radius via an implicit adaptation scheme is a realistic alternative to approaches using classical "set point" homeostatic feedback controls of neural firing.