基于Transformer的东巴画超分辨率重建
Transformer-based Super-Resolution Reconstruction of Dongba Paintings
-
摘要: 纳西族东巴画线条繁杂、色彩丰富, 直接采用现有的方法对真实场景下的低分辨率东巴画图像进行超分辨率重建, 存在线条不够清晰、局部区域过度平滑、缺少细节等问题. 为了解决上述问题, 提出一种基于Transformer的东巴画超分辨率重建方法. 首先, 生成器采用卷积层和残差密集Swin Transformer块提取东巴画图像的浅层和深层特征, 并通过重建模块融合特征, 重建出高分辨率图像; 其次, 判别器采用U-Net评估每个像素的真实性, 增强重建图像的纹理细节; 最后, 采用像素损失、感知损失和对抗损失训练生成器生成自然清晰的东巴画图像. 在自建的东巴画测试集上与其他8种方法进行对比, 结果表明, 所提方法的重建结果具有更好的视觉效果; 在放大2倍、4倍和8倍时, 平均PIQE分别为22.749 3, 20.264 9和18.378 0, 平均ENIQA分别为0.091 7, 0.063 9和0.068 4, 均优于其他方法; 所提方法具有良好的扩展性, 在自然图像上进行实验也能获得更清晰的结果.Abstract: Naxi Dongba paintings have complex lines and rich colors. Directly using the existing methods to perform super-resolution reconstruction of low-resolution Dongba painting images in real scenes has problems such as unclear lines, excessive smoothness in local areas and lack of details. To solve the above problems, we propose a Transformer-based super-resolution reconstruction method for Dongba paintings. Firstly, the generator uses a convolutional layer and the residual dense Swin Transformer blocks to extract the shallow and deep features of the Dongba painting image, and fuses the features through the reconstruction module to reconstruct a high-resolution image. Secondly, the discriminator uses U-Net to evaluate the realness of each pixel to enhance the texture details of the reconstructed image. Finally, the generator is trained using pixel loss, perceptual loss and adversarial loss to generate natural and clear Dongba painting images. Compared with the other 8 methods on the self-built Dongba painting testing set, the results show that the reconstruction results of the proposed method have better visual effects. The average PIQE is 22.749 3, 20.264 9 and 18.378 0, and the average ENIQA is 0.091 7, 0.063 9 and 0.068 4 at magnifications of 2×, 4×, and 8×, respectively, all of which are superior to other methods. The proposed method has good scalability and achieves clearer results on natural images as well.