Depth Map Super-Resolution Reconstruction Based on Hybrid Multimodal Attention Fusion Correction

To solve the problems of blurring and low resolution in depth maps captured by low-cost depth cameras, a hybrid multimode attention fusion correction network is proposed by combining high-resolution color images in the same scene. To effectively leverage the structure information of color images, the attention mechanism is employed to iteratively align and integrate depth features with color structural features, and the fused features are corrected through the incorporation of color structure features and edge aware maps. Initially, the deep edge aware block combines low-resolution depth maps with color edge maps to extract high-resolution depth edge maps. Then, the side-window edge preserve block is utilized to extract useful structural information from the color image, reducing the impact of extraneous texture information. Finally, multimodal attention fusion correction block utilizes attention mechanisms to extract the structural attention distribution of both depth features and color structural features, aligning and fusing these modalities. The spatial attention weights of the color structural features and high-resolution depth edge maps are used to enhance the supervision of the depth edges, reducing blurring and artifacts. Experimental results on the NYU v2 dataset and Middlebury dataset (2005) show that the average root mean square error test results are 2.96 and 1.45, respectively, which are reduced by 0.10 and 0.08 compared to suboptimal methods. The network can reconstruct high-resolution depth maps with clearer edges and fewer artifacts.