Single Depth Map Completion with Multiple Auxiliary Tasks

The aim of depth map completion is to restore dense depth maps from sparse depth maps. To address the problems of edge blurring and semantic information missing in current algorithms, we propose a single depth map completion algorithm based on multiple auxiliary tasks. The completion strategy used in our algorithms is coarse-to-refine, where the coarse completion sub-network uses selective kernel convolution to effectively extract the input information, and the refine completion sub-network consists of a depth map completion master task and related auxiliary tasks. The grayscale reconstruction auxiliary task aims to learn rich semantic information from grayscale images, and then transfer the learned semantic information to depth map completion master task by a feature fusion branch to alleviate the problems of details missing and structure blending. The edge prediction auxiliary task focuses on improving accuracy and sharpness of depth edge. The feature fusion branch between the depth map completion master task and grayscale reconstruction auxiliary task mainly uses spatial and channel attention mechanisms to achieve adaptive fusion of multi-task features to enhance relevant features and suppress irrelevant features. The experiments on the NYUv2 dataset show that the proposed algorithm has better visual completion performance. And our RMSE and REL in the case of 200 sampling points is 0.199 and 0.033, respectively, which outperforms the comparison algorithms.