Abstract: This paper extended and improved the algorithm, a lightweight planar 2D hand gesture recognition algorithm, and realized a 3D gesture recognition algorithm based on projection and single rotation. The gesture template library was divided into two types: spatial planar gesture and spatial stereo gesture. For the templates of the spatial planar gesture, the input gesture trajectory needed to be projected onto the corresponding coordinate plane before the template matching. Based on the matrix singular value decomposition method, the optimal transposing rotation matrix between the normalized input gesture and the templates was calculated. With the transposing matrix, only a rigid transpose operation was needed for matching degree calculation of a template. Experiments show that this method has a high recognition rate for 3D gesture recognition, and the recognition ability is more balanced for 3D plane and stereo gestures.