Abstract: To overcome the limitation of existing methods for segmenting moving object from the dynamic background,a moving object detection algorithm using multi-frame homography constraint and Markov Random Fields(MRF) model is proposed.The proposed method is based on long term motion trajectory which is tracked across the whole video sequences.And the proposed method consists of two stages:trajectory level separation and pixel level labeling.In trajectory separation stage,the background motion is modeled as multi-frame homography constraint.Based on this constraint,the motion trajectory can be divided into foreground and background ones using designed cumulative acknowledgment strategy.In pixel labeling stage,per frame superpixels are acquired by over-segmenting method.Taken the superpixel set as node,the Markov Random Fields model is introduced to convert the sparse trajectory classification to dense pixel labeling.An energy function of Markov Random Fields detection model is designed which has combined motion trajectory separation cue and spatio-temporal constraints of each superpixel.The labeling of each superpixel can be obtained by finding the minimum of the energy function.Compared with traditional methods using long term motion trajectory,the proposed method doesn't need the assumption of affine camera model.And the problem caused by equal length constraint of motion trajectory is also avoided.Meanwhile,the proposed method can effectively handle videos from stationary camera and moving camera.Extensive experiments are conducted on publicly available videos from various sources.And the qualitative and quantitative results show that the proposed algorithm outperforms previous algorithms significantly in trajectory level separation accuracy,trajectory point density and pixel level labeling accuracy.