Convolutional Neural Network Fusing Ranking and Regression for Expression Intensity Estimation

Facial expression intensity estimation is an important part of facial expression analysis.It is also the key technology to realize nature human-machine emotional interaction.The main problem faced by the expression intensity estimation is the lack of abundant labeled data,which makes it difficult to estimate facial expression intensity by supervised methods.Although some ranking based methods can address this problem,these methods only estimate the relative intensity instead of an absolute intensity.To solve the above problems,a convolution neural network fusing rank and regression is proposed for facial expression intensity estimation.The rank-CNN with a Siamese network structure learns the relative relationship between the pair wise data in the sequence.Each subnetwork in the Siamese network adopts a regression-CNN,which is used to learn the samples with intensity labels and to estimate the absolute intensity of expression.In order to verify the effectiveness of the proposed method,experiments are carried out on PAIN and CK+datasets.The experimental results show that the performance of the proposed method in weak supervised learning(PCC,ICC and MAE on PAIN data set are 0.6551,0.5293 and 0.9241 respectively,and PCC,ICC and MAE on CK+data set are 0.7391,0.7216 and 0.1875 respectively)is superior to the state of art.