Image Inpainting with Multi-Modal Attention Mechanism Generative Networks

Image inpainting has important application value in the practical fields of old photo restoration, target removal and video editing. However, the results of existing single-modal attention-based methods show the problems of blurry texture and lack of semantics. We proposed an image inpainting method based on the multi-modal attention mechanism generative networks. Firstly, we adopted a U-Net as the backbone to finish the encoding, decoding and jump connection of damaged images. Secondly, in the encoding and decoding stages, the feature extraction block and image inpainting block based on multi-modal attention mechanism are constructed respectively, which can achieve more fine-grained content completion through multi-scale feature fusion. Finally, combining three image damage rates and three evaluation metrics(SSIM, PSNR and L₁), the experiments on Paris StreetView, CelebA and Places2 dataset show that, compared with the other 4 comparison methods, the proposed method achieves 20 higher, 1 same, and 6 slightly lower results in a total of 27 comparison items, which verifies the effectiveness of the proposed method.