Abstract: In the marker based optical motion capture system, the marker occlusion and various factors can easily lead to a failure of pose reconstruction. This paper proposes a deep learning model based on spatio-temporal constraints for real-time recovery of continuous missing marker sequences. The deep learning network model is based on the time reversal symmetry of human motion and uses the bi-directional long short-term memory network as the backbone of the network. In the process of model training, the combined loss function was proposed to limit the movement range of the key joints, the rigid structure between the markers on the same bone and the time continuity of the markers’ movement track, so as to ensure that the recovered marker sequence conforms to the spatio-temporal constraints of human movement. The experimental results on the HDM05 dataset show that the average error of the proposed method is reduced by more than 14% when compared with the existing method, under the condition that different number of marker sequences and different time spans are missing.