Abstract: The wide use of fisheye cameras makes target tracking on fisheye video get increasing attention. However, the serious distortion caused by the special imaging principle of the fisheye lens brings the target tracking negative effect. Aiming at weakening the interference of the distortion, this paper proposes a novel fisheye video target tracking method based on response template and feature integration. Firstly, the proposed method synthesizes the response template based on the responses of multiple samples as well as constructs a classifier based on the response template, and then extracts the object’s HoG feature and Color Name feature respectively to train the corresponding classifiers. The responses of two classifiers are considered jointly to determine the target location. For further optimizing the tracker, imaging model is used to correct the deformed target before the training of the classifier. Finally, the evaluation results on the constructed fisheye video dataset validate that the proposed method can greatly reduce the negative impact of the image distortion and the target deformation while keeping the real-time performance.