Continuous Sign Language Recognition Based on 3D Hand Skeleton Data

In sign language recognition,it is necessary to eliminate the visual problems caused by interference factors such as background and light.Therefore,an end-to-end continuous sign language recognition model is designed using low-redundant skeleton data.Firstly,the shape and trajectory features are extracted from intra frame and inter frame respectively,which can reduce the discreteness of the original samples.Secondly,a se-ries of parallel two-stream residual networks are constructed to fuse shape and trajectory features,further gen-erate the spatial-temporal feature sequence.Finally,the attention-based encoder-decoder network is used to re-alize the mapping of the fused feature sequence to the translated text.In addition,a new skeleton-based sign language dataset using Leap Motion is collected named LMSLR.Experimental results on the LMSLR dataset and the public CSL dataset show that the proposed model has higher accuracy and lower computational com-plexity than most models based on video processing.