Contextual emotion detection in images using deep learning - Limami, Fatiha; Hdioud, Boutaina; Oulad Haj Thami, Rachid (Frontiers in Artificial Intelligence, 2024)
Computerized sentiment detection is based on artificial intelligence and computer vision, which have become essential in recent years. Thanks to developments in deep neural networks, this technology can now take account of environmental, social, and cultural factors, as well as facial expressions. We aim to create more empathetic systems that can be used for various purposes, from medicine to interpreting emotional interactions on social media. To develop this technology, we combined authentic images from various databases, including EMOTIC (ADE20K, MSCOCO), EMODB_SMALL, and FRAMESDB, to train our models. We have developed two sophisticated algorithms based on deep learning techniques, DCNN and VGG19, and by optimizing the hyperparameters of our models to improve performance, we analyze context and body language to improve our understanding of human emotions in images in different contexts. We merge the 26 discrete emotional categories with the three continuous emotional dimensions to identify emotions in context. The proposed pipeline is then completed by fusing our models. We adjusted the parameters to outperform previous methods in capturing various emotions in different contexts. Our study showed that the Sentiment_recognition_model and VGG19_contexte increased mAP by 42.81% and 44.12%, respectively, surpassing the results of previous studies. This groundbreaking research could significantly improve contextual emotion recognition in images. The implications of these promising results are far-reaching, extending to diverse fields such as social robotics, affective computing, human-machine interaction, and human-robot communication.