Abstract: Focusing on strengthening the robustness of vedio object tracking, an algorithm via sparse collaborative model was proposed. In the discriminative model, the prior visual information was exploited to learn an over-complete dictionary based on the SIFT feature, the dictionary was used to represent the object and train the classifier which separated the object from the background. In the generative model, it extracted the local feature and calculated the occlusion information of the object to construct the object templates, and then the tracking was implemented by computing the similarity between the candidates and the templates. Eventually, the multiplicative formula was exploited to joint the two models to acquire final tracking result. Both qualitative and quantitative evaluations on challenging image sequences demonstrate that the proposed algorithm performs favorably against several state-of-the-art methods.