高级检索
白静, 田栋文, 张霖, 杨宁. 跨域变分对抗自编码器[J]. 计算机辅助设计与图形学学报, 2020, 32(9): 1402-1410. DOI: 10.3724/SP.J.1089.2020.18115
引用本文: 白静, 田栋文, 张霖, 杨宁. 跨域变分对抗自编码器[J]. 计算机辅助设计与图形学学报, 2020, 32(9): 1402-1410. DOI: 10.3724/SP.J.1089.2020.18115
Bai Jing, Tian Dongwen, Zhang Lin, Yang Ning. Cross-Domain Variational Adversarial Autoencoder[J]. Journal of Computer-Aided Design & Computer Graphics, 2020, 32(9): 1402-1410. DOI: 10.3724/SP.J.1089.2020.18115
Citation: Bai Jing, Tian Dongwen, Zhang Lin, Yang Ning. Cross-Domain Variational Adversarial Autoencoder[J]. Journal of Computer-Aided Design & Computer Graphics, 2020, 32(9): 1402-1410. DOI: 10.3724/SP.J.1089.2020.18115

跨域变分对抗自编码器

Cross-Domain Variational Adversarial Autoencoder

  • 摘要: 现有跨域图像生成算法通常要求用户提供成对数据,且生成能力有限,往往仅支持一对一的跨域图像生成.针对以上问题,提出了一种跨域变分对抗自编码器框架,在不提供任何成对数据的前提下,实现了跨域图像的一对多连续变换.假定来自不同域的图像共享相同的内容属性,且拥有独立的风格属性,则跨域图像一对多连续变换问题可转换为图像内容属性和风格属性的解耦、编码、拟合和跨域拼接.首先利用编码器解耦建立跨域数据的内容编码和风格编码;然后利用对抗操作和变分操作分别去拟合图像的内容编码和风格编码;最后通过拼接单域图像的内容编码和风格编码实现图像重构,通过交叉拼接不同域的内容编码和风格编码得到跨域图像的一对多连续变换.在标准数据集MNIST和SVHN上进行的有监督跨域图像生成结果同时满足真实性和多样性,且在分类准确率和域自适应性的定量评价中优于其他跨域图像生成算法;在人脸数据集NIR-VIS和草图数据集Edges-Shoes上实现了无监督跨域图像一对一生成,其可视化结果充分说明了生成图像的特征分布和源特征分布的一致性.以上实验全面验证了变分对抗自编码器框架的可行性和有效性.

     

    Abstract: The existing cross domain image generation algorithms usually require users to provide pairwise data,and the generation ability is limited,which only supports one-to-one cross domain image generation.To solve the above problems,a cross-domain variational adversarial autoencode framework is proposed,which realizes one-to-many continuous transformation of cross-domain images without providing any paired data.Assuming that images from different domains share the same content attributes and have independent style attributes,the one-to-many continuous transformation of cross-domain images can be converted into four steps,including decoupling,encoding,fitting and cross-domain stitching of content attributes and style attributes.Firstly,a content code and a style code of the cross-domain data are decoupled using the proposed encoder.Then the two kinds of codes are fitted by the adversarial operation and the variational operation respectively.Finally,the image reconstruction is realized by stitching the content code and the style code of the single-domain image,and the one-to-many cross-domain continuous transformation is obtained by cross-stitching the content code and the style code of the different-domain images.The supervised cross domain image generation results on the standard datasets MNIST and SVHN satisfy both the authenticity and diversity,and are superior to other cross domain image generation algorithms in the quantitative evaluation of classification accuracy and domain self-adaptability.In addition,the unsupervised cross domain image one-to-one generation is realized on the face dataset NIR-VIS and the sketch dataset Edges-Shoes,which visualization results present that the feature distribution of the generated image is consistent with the feature distribution of the source images.The above experiments demonstrates the feasibility and effectiveness of the proposed framework.

     

/

返回文章
返回