Abstract: In the process of interacting with the virtual environment, the gesture recognition accuracy and efficiency will directly influence the operator's sense of immersion and success rate. The available methods are harder to keep greater accuracy, and meet the real-time request at the same time. Aiming at this problem, a gesture recognition method based on the shape of the hand feature is proposed in this paper. Firstly, we capture a serial of original data using data glove and construct the gesture base by modifying the original gesture. Then we extract the hand-type characteristic of each gesture. At last, we make the gesture recognition by the improved feature point set template match algorithm. Experimental results show that 98.9% accuracy is achieved when the number of categories of gesture is larger(up to twenty-five Categories). Moreover, the proposed method is simple enough to meet the real-time requirements.