Abstract: Multi-view 3D reconstruction of natural scenes has long been a standard topic in computer vision with various applications. With the prevalence of depth capturing devices, how to effectively use multiple depth maps for 3D scene reconstruction becomes an important problem. This paper proposes a multi-view depth map sampling method for 3D reconstruction of natural scenes to automatically eliminate depth errors in the input depth maps, so that high-quality scene models can be recovered. We first do non-uniform sampling on depth maps to obtain a set of 3D points for each frame, and eliminate the 3D points with severe depth errors. Then, a depth confidence estimation algorithm is employed to integrate the sampled 3D points among multiple frames and eliminate redundant ones, so as to obtain the complete 3D point cloud of the scene. Finally, the complete scene geometry model can be generated based on the fused 3D point cloud. A variety of complicated natural scenes are experimented to demonstrate the accuracy and robustness of the proposed method, which can well handle both small objects and large-scale scenes.