深度指导的无监督领域自适应语义分割
Depth Guidance Unsupervised Domain Adaptation for Semantic Segmentation
-
摘要: 为了提高语义分割精度, 解决模型在不同数据域上泛化性差的问题, 提出基于深度信息的无监督领域自适应语义分割方法.首先, 深度感知自适应框架通过捕捉深度信息和语义信息的内在联系, 减小不同域之间的差异; 然后, 设计了一个轻量级深度估计网络来提供深度信息, 通过跨任务交互策略融合深度和语义信息, 并在深度感知空间对齐源域和目标域的分布差距; 最后, 提出基于深度信息的域内自适应策略弥合目标域内部的分布差异, 将目标域分为子源域和子目标域, 并缩小子源域和子目标域分布差距.实验结果表明, 所提方法在SYNTHIA-2-Cityscapes和SYNTHIA-2-Mapillary跨域任务上的平均交并比分别为46.7%和73.3%, 与同类方法相比, 该方法在语义分割和深度估计精度上均有显著提升.Abstract: To improve the segmentation performance and solve the problem of poor generalization of the model in different data domains, we propose a method based on depth information for semantic segmentation in the context of unsupervised domain adaptation. It includes a Depth-aware Adaptation Framework(DAF) and a Intra-domain Adaptation(IDA) strategy. Firstly, DAF is proposed to adapt domains by capitalizing on the inherent correlations of semantic and depth information. Then a novel light-weight depth estimation network is designed provide additional depth information, and we fuse semantic and depth information by cross-task interaction, then align the distribution in depth-aware space between source and target domains. Finally, IDA strategy is proposed to bridge the distribution gap inside the target domain. To this end, a depth-aware ranking strategy is presented to separate target domain into sub-source domain and sub-target domain, and then we perform the alignment between sub-source domain and sub-target domain. Experiments on SYNTHIA-2-Cityscapes and SYNTHIA-2-Mapillary cross-domain tasks show that our method achieves significant improvement(46.7% mIoU and 73.3% mIoU, respectively) compared with the similar methods.