NN-Poly: Approximating common neural networks with Taylor polynomials to imbue dynamical system constraints - Zhu, Frances; Jing, Dongheng; Leve, Frederick; Ferrari, Silvia (Frontiers in Robotics and AI, 2022)
Recent advances in deep learning have bolstered our ability to forecast the evolution of dynamical systems, but uninterpretable ”black box” models are left wanting in safety-critical applications. This contribution addresses this concern by proposing a Neural Network to Polynomial (NN-Poly) approximation, a method that furnishes algorithmic guarantees while retaining state prediction accuracy. To achieve these goals, this paper shows how to represent a trained fully-connected perceptron, convolution, and recurrent neural networks of various activation functions as Taylor polynomials of arbitrary order. This solution is not only analytic in nature, but also least squares optimal. The NN-Poly system identification or state prediction method is evaluated against a single layer neural network and a polynomial trained on data generated by dynamic systems. Across our test cases, the proposed method maintains minimal root-mean-squared state error, requires few parameters to form, and enables model structure for verification and safety. Future work will incorporate safety constraints into state predictions with this new model structure and test high-dimensional dynamical system data.