A Review of Body Gesture Based Affective Computing

Research on the theory and method of affective computing has been a hot topic in the field of human-computer interaction in recent years.At present,the common research on affective computing in related fields focuses on facial expression,speech,text,human gesture,and other directions.There are both single-modality research and multi-modality comprehensive research.Among them,the researches based on facial expressions and speech modalities are the majority,and the research based on human gesture is relatively few.In this paper,we conduct a research survey on several key problems faced by gesture-based affective computing,including the emotional psychological model,human pose estimation,body emotional feature extraction method,emotion classification and labeling method,gesture-emotion dataset,and gesture-based emotion recognition algorithm.Specifically,we first introduce several commonly used emotional computing psychology models,review the applications of various models.Then we summarize the common methods of human pose estimation from two perspectives of human detection and pose estimation,and discuss the application prospects of 2 D and 3 D pose estimation.For feature extraction methods,we analyze feature extraction methods based on body movements of the whole body and the upper body.In the aspect of emotion annotation,we introduce the emotion annotation methods of performance data and non-performance data.We also point out that semi-automatic or automatic labeling of non-performance data would be one of the important development trends in the future.For the posture and emotion datasets,we list 14 most commonly used datasets in recent years classified by performance or non-performance data,data dimensions,static or dynamic poses,full-body or non-full-body data.In terms of gesture-based emotion recognition algorithms,we mainly introduce emotion recognition algorithms based on artificial neural networks,pointing out the advantages and disadvantages of different methods and their applicable datasets.This article reviews and summarizes the classic and cutting-edge research work in related fields,hoping to provide a good research basis for researchers in similar directions.