Spontaneous Facial Expression Recognition Based on RGB-D Dynamic Sequences

Different from traditional facial expression recognition methods based on 2D static images, a spontaneous facial expression recognition algorithm is proposed for RGB-D image sequences. After preprocessing on image alignments and normalization, 4D spatio-temporal texture data are extracted as dynamic features. Then Slow Feature Analysis method is applied to detect the apex of the expression, so that 3D facial geometrical model of the apex image is built and used as the static feature. With the combination of these two kinds of features and the dimensional reduction by PCA, Conditional Random Fields is applied to train and classify the features in the end. A lot of experiments were performed based on BU-4DFE facial expression database. It has been verified that our algorithm not only outperforms traditional static facial expression recognition methods and many other dynamic facial expression recognition methods, but also could recognize spontaneous expression automatically, which makes it possible for further practical applications.