高级检索
时新月, 胡文瑾, 乔浪, 康文东. 融合特征位置编码和误差修正的唐卡图像风格迁移模型[J]. 计算机辅助设计与图形学学报. DOI: 10.3724/SP.J.1089.2023-00068
引用本文: 时新月, 胡文瑾, 乔浪, 康文东. 融合特征位置编码和误差修正的唐卡图像风格迁移模型[J]. 计算机辅助设计与图形学学报. DOI: 10.3724/SP.J.1089.2023-00068
XinYue SHI, WenJin HU, Lang QIAO, WenDong KANG. A Style Transfer Model for Thangka Images Based on Fusion of Feature Position Coding and Error Correction[J]. Journal of Computer-Aided Design & Computer Graphics. DOI: 10.3724/SP.J.1089.2023-00068
Citation: XinYue SHI, WenJin HU, Lang QIAO, WenDong KANG. A Style Transfer Model for Thangka Images Based on Fusion of Feature Position Coding and Error Correction[J]. Journal of Computer-Aided Design & Computer Graphics. DOI: 10.3724/SP.J.1089.2023-00068

融合特征位置编码和误差修正的唐卡图像风格迁移模型

A Style Transfer Model for Thangka Images Based on Fusion of Feature Position Coding and Error Correction

  • 摘要: 唐卡是藏族文化中一种独具特色的艺术风格,通过对唐卡图像进行二次创作能够帮助人们了解唐卡风格特征,促进文化传承与保护。然而,目前唐卡图像风格迁移技术生成新图像仍存在不足,如生成图像存在伪影、局部图像模糊和细节信息处理不到位等。针对以上问题,提出一种融合特征位置编码和误差修正的唐卡风格迁移模型(FPC-EI),该模型利用Transformer捕获图像特征远程依赖关系的能力实现唐卡风格迁移。首先,设计了突出特征位置信息的编码器,利用二维相对位置编码获取内容序列与风格序列之间的对应位置信息,实现图像特征之间的对齐和匹配。其次,编码器中还加入了坐标注意力机制模块,强化利用二维相对位置编码增强编码器对于唐卡以及内容图像纹理细节信息的捕获能力。最后,为了提高生成图像质量,加入密集深度反投影网络,使用不断上下采样层的模块进行误差修正,从而指导模型提高生成图像质量。实验结果表明,本文提出的方法与AdaIN、WCT等风格迁移模型相比,该方法在保留内容结构、特征融合和图像局部细节方面都取得更好的效果。

     

    Abstract: Thangka is a unique art style in Tibetan culture, and secondary creation of Thangka images can help people understand the characteristics of Thangka style and promote cultural inheritance and preservation. However, there are still shortcomings in the generation of new images by the current thangka image style migration technique, such as artifacts in the generated images, blurred local images and poor processing of detail in-formation. To address the above problems, a Tangka style migration model (FPC-EI) that incorporates feature location coding and error correction is proposed, which utilizes the Transformer's ability to capture the remote dependencies of image features to achieve Tangka style migration. First, an encoder highlighting feature position information is designed to obtain the corresponding position information between content sequences and style sequences using two-dimensional relative position encoding to achieve alignment and matching between image features. Secondly, a coordinate attention mechanism module is also added to the encoder to enhance the ability of capturing texture detail information of the tangka and content images using two-dimensional relative position encoding. Finally, in order to improve the quality of the generated im-ages, a dense depth inverse projection network is added to guide the model to improve the quality of the generated images by using a module that constantly up samples and down samples layers for error correction. Experimental results show that the method proposed in this paper achieves better results in terms of preserving content structure, feature fusion and local details of images compared with style migration models such as AdaIN and WCT.

     

/

返回文章
返回