基于多重注意力机制的图像雨滴去除方法
Multi-Attention Mechanism for Raindrop Removal from a Single Image
-
摘要: 图像雨滴去除任务的目标是对于给定的雨滴图像去除其镜头上遮挡的附着雨滴, 还原出真实的干净图像,其在计算机视觉下游任务中有着至关重要的作用. 由于已有的图像雨滴去除方法没有考虑雨滴所具有的空间位置局部性和尺度多样性, 雨滴去除效果不理想. 为缓解上述问题, 提出一种基于多重注意力机制的图像雨滴去除方法. 首先, 为了适应雨滴的空间位置局部性和尺度多样性, 提出结合多尺度特征提取模块和多重注意力模块构建编码器-解码器架构, 其中多重注意力模块融合了像素、通道和空间注意力, 能够自适应地匹配雨滴的空间位置局部性. 此外,设计了一种迭代式图像特征融合模块, 在融合解码器特征和雨滴图像得到初步去雨图像后, 采用初步去雨图像加强解码器特征, 得到进一步的细化特征, 再融合初步去雨图像和细化特征得到最终去雨图像. 在雨滴图像测试集Raindrop 上实验结果表明, 与其他方法相比, 所提方法能够有效地去除图像中的雨滴, 进一步提升雨滴去除的性能,比对比方法中最优的方法在 PSNR 指标上提升了 0.25 dB.Abstract: The image raindrop removal task aims to remove raindrops attached to the lens from a given rainy image, restoring a clean image, which plays a crucial role in downstream tasks of computer vision. Because the existing image raindrop removal methods not taking the spatial locality and scale diversity of raindrops into account, the raindrops are usually not completely removed from the images. To alleviate the above issues, this paper proposes a raindrop removal method from a single image based on a multi-attention mechanism. First, in order to adapt to the spatial locality and scale diversity of raindrops, a multi-scale feature extraction module and a multi-attention module are combined to construct an encoder-decoder architecture. The multi-attention module integrates pixel, channel, and spatial attention, which can match the spatial locality of raindrops adaptively. In addition, this paper designs an iterative image feature fusion module. The features from the decoder and raindrop images are fused to obtain a preliminary raindrop removal image; then, the decoder features are enhanced with the preliminary raindrop removal image to obtain further refined features; the preliminary raindrop removal image and refined features are fused to obtain a final raindrop removal image. The experimental results on Raindrop image test set show that the proposed method can effectively remove raindrops from the image compared with other methods, further improving the performance of raindrop removal, and improving the PSNR by 0.25 dB compared to the best method in the comparison methods.