Abstract: A visual tracking method using stereo camera and markers is presented in this paper, to improve the accuracy and stability in a tracking-based glass-free 3D display system. First, two cameras with infrared pass-only filters are used to capture images of users wearing infrared markers in real time; then speckles corresponding to infrared markers are detected and reconstructed using calibrated parameters of the stereo camera; the reconstructed results are next passed to a tracking-based 3D display system for dynamic image mapping, and finally 3D images are projected to user's eyes via LC lens gratings. Experimental results show that the proposed method is more accurate and stable than monocular face tracking method used in traditional tracking-based 3D display systems, which makes it more suitable to areas like medical endoscopy operations where accuracy and stability are critical.