Aiming at the problem of disease area calibration of the Ming Dynasty Murals in Zhilin Temple, Jianshui, Yunnan Province, this paper proposes an automatic calibration algorithm of crack and flaking disease. Firstly, the algorithm detects the multi-dimensional gradient of the mural image in the HSV space, and extracts the texture and line features of the mural. Then the guided filtering is used to suppress the painting lines in the mural, and the filtered image is segmented by an adaptive threshold to generate the initial mask of the mural disease areas. Next, the discontinuous edge curves in the initial mask are connected by using a tensor voting method, and the complete mural disease area mask is obtained by using morphological hole filling. Finally, the disease mask is added to the original mural image to achieve automatic calibration of the mural cracks and flaking disease areas. Experimental results show that the proposed algorithm achieves the best calibration effect on the self-made 48 Zhilin Temple mural datasets. The precision and recall reach 78.9 % and 69.5 % respectively. The F-Measure value is 18 % higher than the optimal comparison method. Moreover, the proposed algorithm does not require human-computer interaction, and has higher computational efficiency.