Spatiotemporal Features Fusion From Local Facial Regions for Micro-Expressions Recognition - Aouayeb, Mouath; Soladie, Catherine; Hamidouche, Wassim; Kpalma, Kidiyo; Seguier, Renaud (Frontiers in Signal Processing, 2022)
Facial Micro-Expressions (MiEs) analysis has applications in a variety of fields, including emotional intelligence, psychotherapy, and police investigation. However, since MiEs are fast, subtle, and local reactions, it is a challenge for humans as well as for machines to detect and recognize them. In this article, we propose a deep learning approach that addresses the locality and the temporal aspects of MiE by learning spatiotemporal features from local facial regions. Our proposed method is particularly unique in that we use two fusion-based Squeeze and Excitation (SE) strategies to drive the model to learn the optimal combination of extracted spatiotemporal features from each area. The proposed architecture enhances a previous solution of an automatic system for Micro-Expression Recognition (MER) from local facial regions using a composite deep learning model of Convolutional Neural Network (CNN) and Long Short Term Memory (LSTM). Experiments on three spontaneous MiE datasets show that the proposed solution outperforms state-of-the-art approaches. We open-source our code at https://github.com/MouathAb/AnalyseMiE-CNN_LSTM_SE