Abstract: Occlusions existing in long image sequences of a rigid scene cause missing entries in the measure matrix.A novel online multi-frame correspondence estimation algorithm is proposed in this paper to recover missing frames.The trajectory matrix and displacement matrix,which is weighted by image gradient statistics,reside in a low-dimensional linear subspace.Firstly,a complete sub-matrix selected from the displacement matrix is constrained by the rank of the low-dimensional linear subspaces and reorganized into a corresponding trajectory matrix.The trajectory matrix is then constrained by the corresponding rank and decomposed into a base matrix and a coefficient matrix.Secondly,to solve the aperture problem,the displacement components are estimated from the base and coefficient matrix and then backfilled into multi-frame displacement matrix.Thirdly,as subsequent frames acquired,the new data are integrated into the multi-frame correspondence estimation by using incremental SVD.Lastly,after processed all of the frames,the remaining missing entries are processed with the nonlinear optimization algorithm.We mathematically proved the feasibility of the algorithm.Compared with Irani’s correspondence estimation algorithm,our experimental results show that the proposed algorithm is faster and more effective in error-control and under different valid entry ratios and error levels.