Abstract: Optical flow is a widely used method for points tracking.However,the performance of optical flow decreases drastically when the illumination changes or partly occlusion happens.To solve this problem,we present a novel method for outlier rejections from optical flow tracking points.By measuring the distance between the source point and the destination point obtained from its complements' projection relation,our method automatically determines the point with largest error.We prove the correctness of our method and then give the camera pose matrix on video sequence.The experimental comparison of our results to those of random sample consensus(RANSAC) shows the robustness of our method.