Spontaneous Facial Expressions and Micro-expressions Coding: From Brain to Face - Dong, Zizhao; Wang, Gang; Lu, Shaoyuan; Li, Jingting; Yan, Wenjing; Wang, Su-Jing (Frontiers in Psychology, 2022)
Facial expressions are a vital way for humans to show theirs perceive emotions. It is convenient for detecting and recognizing expressions or micro-expressions by annotating a large number of data in deep learning. However, the study of video-based expressions or micro-expressions requires coders to have professional knowledge and be familiar with action unit (AU) coding, leading to considerable difficulties. This paper aims to alleviate this situation. We deconstruct facial muscle movements from the motor cortex and systematically sort out the relationship among facial muscles, AU, and emotion to make more people understand coding from the basic principle: 1. We derived the relationship between AU and emotion based on a data-driven analysis of 5,000 images from the RAF-AU database, along with the experience of professional coders. 2. We discussed the complex facial motor cortical network system that generates facial movement properties, detailing the facial nucleus and the motor system associated with facial expressions. 3. The supported physiological theory for AU labeling of emotions is obtained by adding facial muscle movements patterns. 4. We present the detailed process of emotion labeling and the detection and recognition of AU. Based on the above research, the video’s coding of spontaneous expressions and micro-expressions is concluded and prospected.