Studying the Evolution of Neural Activation Patterns During Training of Feed-Forward ReLU Networks - Hartmann, David; Franzen, Daniel; Brodehl, Sebastian (Frontiers in Artificial Intelligence, 2021)
The ability of deep neural networks to form powerful emergent representations of complex statistical patterns in data is as remarkable as imperfectly understood. For deep ReLU networks, these are encoded in the mixed discrete-continuous structure of linear weight matrices and non-linear, binary activations. Our paper develops a new technique for instrumenting such networks to efficiently record activation statistics, such as information content (entropy) and similarity of patterns, in real-world training runs. We then study the evolution of activation patterns during training for networks of different architecture and using different training and initialization strategies. As a result, we see characteristic, general as well as architecture-related behavioral patterns: In particular, most architectures form structure bottom-up, with the exception of highly-tuned state-of-the-art architectures and methods (PyramidNet, FixUp), where layers appear to converge more simultaneously. We also observe intermediate dips in entropy in conventional CNNs that are not visible in residual networks. A reference implementation is provided under a free license under https://github.com/JGU-VC/activation-pattern-analysis.