Abstract: To solve the problems of acquisition of label image and spectral distortion in the current remote sensing image fusion,a semi-supervised remote sensing image fusion method using Siamese structure is proposed.This method adopted a generative adversarial network structure composed of generator and discriminator,in which the generator contains two encoders and a decoder.First,the multispectral image is amplified and converted into HSV color space.Then,the V channel of the multispectral image and panchromatic images are respectively input into the Siamese network of the encoder,and the image features are extracted through the convolutional layer and the multi-skip connection layer model.Third,the obtained feature map is input to the decoder for image reconstruction.And the fused V channel image is identified by the discriminator,so as to obtain the optimal fusion result.Finally,the fused V channel is concatenated with the H and S channels of the multispectral image to obtain the final fused image.In addition,a compound loss function is designed.Experiments on QuickBird satellite remote sensing image dataset show that this method can effectively improve spatial details and color information in fused images.Compared with the contrast algorithms,the fusion images have certain advantages in subjective visual quality and objective evaluation index.