Abstract: Leather defects with variable morphology and high local similarity are of difficulty in extracting features comprehensively and accurately. In this work, a refined surface defect segmentation method based on improved U-Net network is proposed. On the encoder side, a cascaded dilated convolution module is embedded to obtain the global features while preserving the detail information of the original image, and a feature fusion module is added to the jump connection to reduce local features loss caused by directly splicing of the high-level and low-level feature tensor; on the decoder side, a decoding module based on the channel attention mechanism,which can guide the network to adaptively focus on defective regions, is used to replace the original convolutional layer; to further integrate high-level information, a global average pooling module is embedded as the semantic guide to improve the discrimination capability of the network from similar defects at the decoding end. The experimental results conducted on a leather dataset containing 7 kinds of defects show that the proposed method achieves 99.17%, 93.27%, 98.39%, and 88.88% in PA, MPA, FWIoU, and MIoU, which is 0.28, 2.78, 0.53, and 4.03 percentage points better than that of U-Net. The qualitative and quantitative analysis results demonstrate that the algorithm proposed has remarkable ability to refine the segmentation in leather defect recognition.