Lightweight ViT Model for Micro-Expression Recognition Enhanced by Transfer Learning - Liu, Yanju; Li, Yange; Yi, Xinhai; Hu, Zuojin; Zhang, Huiyu; Liu, Yanzhong (Frontiers in Neurorobotics, 2022)
Micro-expressions, as opposed to macro-expressions, are subtle and not easily detectable emotional expressions, often containing rich information about mental activities, and effective recognition of micro-expressions is important in fields such as interrogation and healthcare. Neural networks are currently one of the most common approaches to micro-expression recognition, but neural networks often increase their complexity when improving accuracy, and overly large neural networks require extremely high hardware requirements for running equipment. In recent years vision transformers based on self-attentive mechanisms have achieved accuracy in the field of image recognition and classification that is no less than that of neural networks, but the drawback is that without the image-specific biases inherent to neural networks, the cost of improving accuracy is an exponential increase in the number of parameters. This paper presents the application of an optimised MobileViT model to the micro-expression recognition task. First, the model is transfer learnt using three macro-expression datasets to preserve the weights at the highest recognition accuracy. Secondly, the CASME II, SAMM and SMIC datasets were combined into a compound dataset, apex frames of each expression sample were extracted, facial regions were extracted, and cropped to a fixed size after grey-scale processing. Finally, the pre-processed samples were fed into the optimised MobileViT model, which was trained and classified using a support vector machine classifier. An accuracy of 84.27% was obtained in the experiments, and the time to process a single sample was 35.4 milliseconds. The results show that the optimised MobileViT model significantly improves the recognition accuracy of micro-expressions using the self-attention mechanism in the vision transformer without a significant increase in resource consumption.