高级检索
陈羽中, 林闽沪, 陈友昆, 牛玉贞. 基于多重注意力机制的图像雨滴去除方法[J]. 计算机辅助设计与图形学学报. DOI: 10.3724/SP.J.1089.2023-00409
引用本文: 陈羽中, 林闽沪, 陈友昆, 牛玉贞. 基于多重注意力机制的图像雨滴去除方法[J]. 计算机辅助设计与图形学学报. DOI: 10.3724/SP.J.1089.2023-00409
Yuzhong Chen, Minhu Lin, Youkun Chen, Yuzhen Niu. Multi-Attention Mechanism for Raindrop Removal from a Single Image[J]. Journal of Computer-Aided Design & Computer Graphics. DOI: 10.3724/SP.J.1089.2023-00409
Citation: Yuzhong Chen, Minhu Lin, Youkun Chen, Yuzhen Niu. Multi-Attention Mechanism for Raindrop Removal from a Single Image[J]. Journal of Computer-Aided Design & Computer Graphics. DOI: 10.3724/SP.J.1089.2023-00409

基于多重注意力机制的图像雨滴去除方法

Multi-Attention Mechanism for Raindrop Removal from a Single Image

  • 摘要: 图像雨滴去除任务的目标是对于给定的雨滴图像去除其镜头上遮挡的附着雨滴, 还原出真实的干净图像, 其在计算机视觉下游任务中有着至关重要的作用. 由于已有的图像雨滴去除方法没有考虑雨滴所具有的空间位置局部性和尺度多样性, 导致雨滴去除效果不理想. 为解决上述问题, 本文提出一种基于多重注意力机制的图像雨滴去除方法. 首先为了适应雨滴的空间位置局部性和尺度多样性, 本文提出结合多尺度特征提取模块和多重注意力模块构建编码器-解码器架构, 其中多重注意力模块融合了像素、通道和空间注意力, 能够自适应地匹配雨滴的空间位置局部性. 此外, 本文设计了一种迭代式图像特征融合模块, 首先融合解码器特征和雨滴图像得到初步去雨图像, 然后用初步去雨图像加强解码器特征得到进一步的细化特征, 最后融合初步去雨图像和细化特征得到最终去雨图像. 实验结果表明本文所提出的方法能够有效地去除图像中的雨滴, 且性能超越了当前先进的图像雨滴去除方法.

     

    Abstract: The image raindrop removal task aims to remove raindrops attached to the lens from a given rainy image, restoring a clean image, which plays a crucial role in downstream tasks of computer vision. Because the existing image raindrop removal methods not taking the spatial locality and scale diversity of raindrops into account, the raindrops are usually not completely removed from the images. To address the above issues, this paper proposes a raindrop removal method from a single image based on a multi-attention mechanism. First, in order to adapt to the spatial locality and scale diversity of raindrops, we combine a multi-scale feature extraction module and a multi-attention module to construct an encoder-decoder architecture. The multi-attention module integrates pixel, channel, and spatial attention, which can match the spatial locality of raindrops adaptively. In addition, this paper designs an iterative image feature fusion module. First, the features from the decoder and raindrop images are fused to obtain a preliminary raindrop removal image. Then, the decoder features are enhanced with the preliminary raindrop removal image to obtain further refined features. Finally, the preliminary raindrop removal image and refined features are fused to obtain a final raindrop removal image. The experimental results show that our method can effectively remove raindrops from images, and its performance surpasses the state-of-the-art image raindrop removal methods.

     

/

返回文章
返回