Semantic Segmentation of Remote Sensing Image via Self-Attention-Based Multi-Scale Feature Fusion

 In order to solve the problems of incomplete and low accuracy of semantic segmentation of remote sensing images due to complex contents, large differences in object scales, and uneven distribution of remote sensing images, we propose one semantic segmentation algorithm of remote sensing images with self-attention multi-scale feature fusion. The main body of the algorithm is based on the encoder-decoder structure, where the encoder uses the Swin-Transformer model to extract complex multi-scale features, and the decoder consists of a self-attention multi-scale feature fusion module and a feature pyramid network. The extracted multi-scale features are firstly adjusted to the same scale and then fed into the self-attention multi-scale feature fusion module to fuse the multi-scale features to ensure that the feature information at different scales which can be fully utilized in the semantic segmentation. Afterwards, the results of the self-attention multi-scale feature fusion are further superimposed and fused from top to down using the feature pyramid. Finally, the results are predicted. The experimental results show that the proposed algorithm achieves the mean intersection over union is 52.77% under the single-scale strategy, which is 1.42% better than the suboptimal result and the mean intersection over union is 54.19% under the multi-scale strategy, which is 1.47% better than the suboptimal result. The experiment demonstrates that the proposed algorithm can effectively fuse the multi-scale features to improve the segmentation accuracy.