高级检索

多辅助任务下的单幅深度图像修复

Single Depth Map Completion with Multiple Auxiliary Tasks

  • 摘要: 深度图像修复是由稀疏深度图像恢复出密集深度图像. 针对目前单幅深度图像修复算法存在边界模糊、语义信息缺失等问题, 提出一种多辅助任务下的单幅深度图像修复算法. 算法采用由粗到精的修复模式, 粗修复网络采用核选择卷积有效提取输入信息, 精修复网络由深度图像修复主任务和相关辅助任务构成; 灰度重建辅助任务旨在从灰度图像中学习到丰富的语义信息, 并由特征融合分支将所学语义信息传递到深度图像修复主任务, 有效解决深度图像修复中细节缺失、结构混乱等问题; 边界预测辅助任务侧重于提高密集深度图像的边界准确性和清晰度; 深度图像修复主分支和灰度重建辅助分支间的特征融合分支主要使用空间和通道注意力机制实现多任务特征的自适应融合, 以增强相关特征和抑制无关特征. 在NYUv2数据集上的实验结果表明, 提出算法的修复视觉效果良好, 客观评价指标RMSE和REL在200采样点数下, 分别取得了0.199和0.033的结果, 均优于对比算法.

     

    Abstract: The aim of depth map completion is to restore dense depth maps from sparse depth maps. To address the problems of edge blurring and semantic information missing in current algorithms, we propose a single depth map completion algorithm based on multiple auxiliary tasks. The completion strategy used in our algorithms is coarse-to-refine, where the coarse completion sub-network uses selective kernel convolution to effectively extract the input information, and the refine completion sub-network consists of a depth map completion master task and related auxiliary tasks. The grayscale reconstruction auxiliary task aims to learn rich semantic information from grayscale images, and then transfer the learned semantic information to depth map completion master task by a feature fusion branch to alleviate the problems of details missing and structure blending. The edge prediction auxiliary task focuses on improving accuracy and sharpness of depth edge. The feature fusion branch between the depth map completion master task and grayscale reconstruction auxiliary task mainly uses spatial and channel attention mechanisms to achieve adaptive fusion of multi-task features to enhance relevant features and suppress irrelevant features. The experiments on the NYUv2 dataset show that the proposed algorithm has better visual completion performance. And our RMSE and REL in the case of 200 sampling points is 0.199 and 0.033, respectively, which outperforms the comparison algorithms.

     

/

返回文章
返回