高级检索
谢志峰, 梁佳佳, 夏世宇, 李梦甜, 马利庄. 基于自适应特征校准的区域感知图像和谐化[J]. 计算机辅助设计与图形学学报. DOI: 10.3724/SP.J.1089.2023-00348
引用本文: 谢志峰, 梁佳佳, 夏世宇, 李梦甜, 马利庄. 基于自适应特征校准的区域感知图像和谐化[J]. 计算机辅助设计与图形学学报. DOI: 10.3724/SP.J.1089.2023-00348
Zhifeng Xie, Jiajia Liang, Shiyu Xia, Mengtian Li, Lizhuang Ma. Region-aware Image Harmonization via Adaptive Feature Calibration[J]. Journal of Computer-Aided Design & Computer Graphics. DOI: 10.3724/SP.J.1089.2023-00348
Citation: Zhifeng Xie, Jiajia Liang, Shiyu Xia, Mengtian Li, Lizhuang Ma. Region-aware Image Harmonization via Adaptive Feature Calibration[J]. Journal of Computer-Aided Design & Computer Graphics. DOI: 10.3724/SP.J.1089.2023-00348

基于自适应特征校准的区域感知图像和谐化

Region-aware Image Harmonization via Adaptive Feature Calibration

  • 摘要: 为进一步提升图像和谐化质量, 本文提出基于自适应特征校准的区域感知图像和谐化方法. 该方法采用多层次的编码器解码器结构, 并在跳跃连接处设计自适应特征校准模块. 该模块利用部分卷积分别提取前景和背景特征, 通过自适应实例归一化对前景特征校准, 从而降低局部含偏特征对解码器重构过程的干扰. 同时, 该方法在编解码器之间使用由Swin Transformer改进的区域感知Transformer, 不仅增强上下文信息交互, 而且在不同尺度上利用背景特征的统计信息调制前景特征. 本文方法在公开的图像和谐化iHarmony4数据集上进行评估, 在PSNR、MSE和fMSE指标上显著优于目前的主流方法.

     

    Abstract: To enhance the quality of image harmonization, we propose a region-aware image harmonization method based on adaptive feature calibration. Our method adopts a multi-level encoder-decoder structure and designs an adaptive feature calibration module at the skip connection. This module leverages partial convolution to extract foreground and background features separately, and calibrates the foreground features with adaptive instance normalization, reducing the impact of locally biased features on the decoder reconstruction process. In addition, we utilize the region-aware Transformer improved by the Swin Transformer, which not only enhances the interaction of contextual information, but also modulates foreground features at different scales according to the statistical information of background features. Our method is evaluated on the publicly available iHarmony4 dataset, and outperforms current state-of-the-art methods significantly in terms of PSNR, MSE, and fMSE metrics.

     

/

返回文章
返回