Abstract: When targets’ scale change or occlusions occur, the performance of kernelized correlation filters tracking algorithm degrades. To cope with this issue, this paper proposes a block-based kernelized correlation filters tracking algorithm. Firstly, targets are divided into several blocks based on their appearance characteristics. Then these blocks are tracked separately and only valid blocks are used to guide target model updating process. Finally, as the tracking results of each block overlap or separate with the change of targets’ scale, we obtain targets’ overall positions based on the tracking results of valid blocks. Experimental results on 30 videos show that the proposed algorithm has better performance than original algorithm when the targets’ scale change or occlusions occur. Furthermore, the algorithm runs at a high speed about 100 frames per second.