Abstract: To improve the detection accuracy of highway pavement cracks, a crack detection method combining ef-fective multi-scale feature fusion structure and cascade optimization is proposed for the problems of crack polymorphism and noise interference. Then, aiming at the focus of encoder-decoder on feature information, spatial and channel attention mechanisms are introduced to suppress background noise interference and enhance crack feature expression. Finally, the joint optimization learning of different levels of decoding features is carried out to strengthen the use of low-level feature information and improve the coherence of crack segmentation lines. The experimental results on data sets such as Crack500 show that the visual crack results detected by the proposed method have clear and complete texture, and the F1 score and average in-tersection over union on the comprehensive test set are 90.07% and 82.07%, which has better recognition effect and robustness, and can provide a deep learning method for automatic crack defect detection tasks.