Communication Sparsity in Distributed Spiking Neural Network Simulations to Improve Scalability - Fernandez-Musoles, Carlos; Coca, Daniel; Richmond, Paul (Frontiers in Neuroinformatics, 2019)
In the last decade there has been a surge in the number of big science projects interested in achieving a holistic understanding of the functions of the brain, using Spiking Neuronal Network (SNN) simulations to aid discovery and experimentation. Such an approach increases the computational demands on SNN simulators: if natural scale brain-size simulations are to be realised, it is necessary to use parallel and distributed models of computing. Communication is recognised as the dominant part of distributed SNN simulations. As the number of computational nodes raises, the proportion of time the simulation spends in useful computing (computational efficiency) is reduced and therefore applies a limit to scalability. This work targets the three phases of communication to improve overall computational efficiency in distributed simulations: implicit synchronisation, process handshake and data exchange. We introduce a connectivity-aware allocation of neurons to compute nodes by modelling the SNN as a hypergraph. Partitioning the hypergraph to reduce interprocess communication increases the sparsity of the communication graph. We propose dynamic sparse exchange as an improvement over simple point-to-point exchange on sparse communications. Results show a combined gain when using hypergraph-based allocation and dynamic sparse communication, increasing computational efficiency by up to \replaced[id=CFM]{40.8 percentage points}{20%} and reducing simulation time by up to \replaced[id=CFM]{73%}{44%}. The findings are applicable to other distributed complex system simulations in which communication is modelled as a graph network.