MIDAS: Mixing Ambiguous Data With Soft Labels for Dynamic Facial Expression Recognition

Abstract

Dynamic facial expression recognition (DFER) is an important task in the field of computer vision. To apply automatic DFER in practice, it is necessary to accurately recognize ambiguous facial expressions, which often appear in data in the wild. In this paper, we propose MIDAS, a data augmentation method for DFER, which augments ambiguous facial expression data with soft labels consisting of probabilities for multiple emotion classes. In MIDAS, the training data are augmented by convexly combining pairs of video frames and their corresponding emotion class labels, which can also be regarded as an extension of mixup to soft- labeled video data. This simple extension is remarkably effective in DFER with ambiguous facial expression data. To evaluate MIDAS, we conducted experiments on the DFEW dataset. The results demonstrate that the model trained on the data augmented by MIDAS outperforms the existing state-of-the-art method trained on the original dataset.

Publication
Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision
Hideaki Hayashi
Hideaki Hayashi
Associate Professor

Hideaki Hayashi is an associate professor with Institute for Datability Science, Osaka University. His research interests focus on neural networks, machine learning, and medical data analysis.

Noriko Takemura
Noriko Takemura
Guest Associate Professor

She is working on ambient intelligence and gait recognition using pattern recognition and machine learning.

Hajime Nagahara
Hajime Nagahara
Professor

He is working on computer vision and pattern recognition. His main research interests lie in image/video recognition and understanding, as well as applications of natural language processing techniques.