高级检索

基于向量符号架构-域适应网络的低照度图像语义分割方法

Low-Light Image Semantic Segmentation Method Based on Vector Symbolic Architecture-Domain Adaptation Network

  • 摘要: 针对语义分割中由于低照度图像光照不足, 存在图像亮度、对比度、信噪比低等现象, 导致低照度图像语义分割难度增大, 提出一种基于向量符号架构(vector symbolic architectures, VSA)-域适应网络的低照度图像语义分割方法. 该方法首先将基于向量符号架构的图像翻译网络和域适应语义分割网络融合在一起, 将正常照度图像翻译为低照度图像, 减少域之间的风格差异, 从而提高低照度图像的分割精度; 然后在图像翻译网络中提出分层特征映射(layeredfeature mapping, LFM)模块, 它能更好地将语义分割网络编码器的多层特征向量映射到 VSA 空间, 使翻译图像与低照度图像更相似, 进而提高分割网络的分割精度; 最后在域适应语义分割网络中提出跨域稀有类混合(cross domain rare classmixing, CDRCM)方法, CDRCM 在域适应的过程中根据低照度图像的伪标签分布情况得到低照度域的稀有类, 再在跨域混合时偏向这些稀有类, 从而提高低照度域中稀有类的分割精度. 在 Cityscapes→Dark Zurich 和 Cityscapes→ACDC-night上的实验结果表明, 文中方法比基线 DAFormer 在平均交并比上分别提高了 8.2 个百分点和 7.2 个百分点, 实验结果表明了其有效性.

     

    Abstract: In view of the insufficient illumination of low-light images in semantic segmentation, there are phenomena such as low image brightness, contrast, and signal-to-noise ratio, which makes semantic segmentation of low-light images more difficult. A low-light image semantic segmentation method based on vector symbolic architectures domain adaptation network is proposed. The method first integrates the image translation network based on vector symbolic architectures and the domain adaptive semantic segmentation network. The image translation network translates normal illumination images into low-light images, reducing style differences between domains and improving the segmentation accuracy of low-light images. Then, a layered feature mapping (LFM) module is proposed in the image translation network. The LFM module can better map the multi-layer feature vectors of the semantic segmentation network encoder to the VSA space, making the translated image more similar to the low-light image, thereby improving the segmentation accuracy of the segmentation network. Finally, a cross domain rare class mixing (CDRCM) method is proposed in the domain adaptive semantic segmentation network. CDRCM obtains rare classes in the low-light domain based on the pseudo- label distribution of low-light images during domain adaptation, and biases towards these rare classes during cross-domain mixing, thereby improving the segmentation accuracy of rare classes in the low-light domain. The experimental results on Cityscapes→Dark Zurich and Cityscapes→ACDC-night indicated that the proposed method in the paper achieved improvements of 8.2 percentage points and 7.2 percentage points, respectively, in mean intersection-over-union compared to the baseline DAFormer, the experimental results show the effectiveness of the method.

     

/

返回文章
返回