GAPCNN with HyPar: Global Average Pooling convolutional neural network with novel NNLU activation function and HYBRID parallelism - Habib, Gousia; Qureshi, Shaima (Frontiers in Computational Neuroscience, 2022)
Abstract: With the increasing demand for deep learning in the last few years, CNNS haven been widely used in many application areas and gained interest in many classifications, regression, and image recognition tasks. The training of these deep neural networks is compute-intensive and takes even days or weeks to train the model from scratch. The compute-intensive nature of these deep neural networks sometimes limits the practical implementation of the CNNS in real-time applications. Therefore, the computational speedup in these networks is of utmost importance, which gains interest towards DNN acceleration. Much research is going on to meet the computational requirement and make its feasibility for real-time applications. Because of simplicity, data parallelism is used primarily, but it performs worst sometimes. In most cases, researchers prefer model parallelism instead of data parallelism, but it isn’t always the best choice. Therefore, in this work, we will implement a hybrid of both data and model parallelism for improving the computational speed without compromising accuracy. There is only a 1.5% accuracy drop in our proposed work with an increased speed up of 3.62X. Also, the novel activation function Normalized nonlinear activation unit NNLU is proposed, to introduce the nonlinearity in the model. The activation unit is non-saturated and helps avoid the model’s over-fitting. The activation unit is free from the vanishing gradient problem. Also in the proposed CNN model, the fully connected The layer is entirely replaced by the Global average pooling layers (GAP) to have enhanced accuracy and computational performance of the model. The model, when tested on a bio-medical image data-set, achieved an accuracy of 98.89% and required a training time of only 1 sec. The model performs the classification of medical images into different categories of Tumours such as glioma, meningioma, and pituitary tumours. The model is compared with existing network models and is observed that the proposed model outperforms as far as classification accuracy and computational speed is concerned. Also, the accuracy results are observed on the varying optimizer, a varying range of learning rates, and several epochs.