Depth Map Super-Resolution Reconstruction Based On Hybrid Multimodal Attention Fusion Correction

To solve the problems of blurring and low resolution in depth maps captured by low-cost depth cameras, a hybrid multimode attention fusion correction network (HMAFCN) is proposed by combining high-resolution color images in the same scene. In order to effectively use the structure information of color image, HMAFCN uses the attention mechanism to iteratively integrate the depth features and color structure features, and corrects the fused features through the color structure features and perceptual edge map. Firstly, in the deep edge aware blocks, the high-resolution depth edge map is extracted by combining low-resolution depth map and color edge map; Secondly, the side-window edge preserving blocks is used to extract the useful structure information in the guide color image to reduce the influence of redundant texture information; Finally, the multimodal attention fusion correction blocks uses the attention mechanism to extract the structural feature distribution of the depth feature and the color structure feature, so that the two modal features can be integrated, and uses the spatial attention weight of the color structure feature and the high-resolution depth edge map to enhance the supervision of the depth edge and overcome the limitation of ambiguity and artifacts. The average test results of root mean square error in NYU v2 datasets and Middlebury datasets were 2.96 and 1.44, respectively, which were reduced by 0.1 and 0.09 compared with the suboptimal method. The experimental results show that the proposed method can reconstruct high-resolution depth maps with clearer edges and fewer artifacts.