Image Harmonization Method Based on Deep Forgery Detection

In computer vision and augmented reality fields, it is an important and challenging task to fuse foreground objects into the background scene and achieve image harmonization. Most of the current mainstream harmonization methods adjust the appearance of the image foreground to make it compatible with the background visually. However, most of these methods are limited to improving the convolutional layer of the harmonization network, and the judgment of whether the image is harmonious depends on relatively subjective human visual angle or global reconstruction error. Therefore, there is limited room for improvement in the harmonization effect. Based on the current work of image harmonization, this paper proposes an image harmonization authenticity identification network. The network identifies whether the image is a synthetic image based on deep learning. Construct a GAN model with the results of the forgery detection network as the judgment indicator and the existing encoder-decoder harmonization network using the generative adversarial mechanism. The two networks compete with each other to achieve the result that the authenticity identification network cannot recognize the reconstruction result of the harmonization network as a synthetic image. To further ensure channel correlation and lighting consistency during image harmonization, this paper adds an image difference module and an image lighting module to the network. The difference module uses internal channels to calculate the difference information, and the illumination module uses an encoder network to extract features and parse local features and global features. Then it performs feature fusion and linear prediction to obtain the illumination information of the image. Finally, the feature extraction and fusion module comprehensively identifies the harmonization result of the image. This paper conducted multiple qualitative and quantitative experiments on the public benchmark dataset iHarmony. The experimental results show that this method has better MSE and PSNR evaluation index values than existing methods, and this method has achieved excellent performance in image harmonization tasks.