Neuromorphic Systems Design by Matching Inductive Biases to Hardware Constraints - Muller, Lorenz K.; Stark, Pascal; Offrein, Bert Jan; Abel, Stefan (Frontiers in Neuroscience, 2020)
Neuromorphic systems are designed with careful consideration of the physical properties of the computational substrate they use. Specifically neuromorphic engineers often exploit physical phenomena to directly implement a desired functionality, enabled by ``the isomorphism between physical processes in different media'' (Douglas, 1995). This bottom-up design methodology could be described as matching computational primitives to physical phenomena. In this paper we propose a top-down counterpart to the bottom-up approach to neuromorphic design. Our top-down approach, termed `bias matching', is to match the inductive biases required in a learning system to the hardware constraints of its implementation; a well-known example is enforcing translation equivariance by tying weights (replacing vector-matrix multiplications with convolutions). We give numerous examples from the literature and explain how they can be understood from this perspective. Furthermore we propose novel network designs based on this approach in the context of collaborative filtering. Our simulation results underline our central conclusions: Additional hardware constraints can improve the predictions of a Machine Learning system and understanding the inductive biases that mediate these performance gains can be useful in finding applications for a given constraint.