Image Inpainting with Multi-modal Attention Mechanism Generative Networks

Image inpainting has important application value in the practical fields of old photo restoration, target removal and video editing. However, the results of existing single-modal attention-based methods show the problems of blurry texture and lack of semantics. In this work, we proposed an image inpainting method based on the multi-modal attention mechanism generative networks. Firstly, we adopted a U-Net as the backbone to finish the encoding, decoding and jump connection of damaged images; Then, in the encoding and decoding stages, the feature extraction block and image inpainting block based on multi-modal attention mechanism are constructed respectively, which can achieve more fine-grained content completion through multi-scale feature fusion. Finally, combining three image damage rates and three evaluation metrics(, and), the experiments on Paris Streetview, CelebA and Places2 dataset show that, compared with the other 4 comparison methods, the proposed method achieves 20 higher, 1 same, and 6 slightly lower results in a total of 27 comparison items, which verifies the effectiveness of the proposed method.