Abstract: Image-based 3D reconstruction is an important research topic in computer vision.The current deep neural networks cannot effectively eliminate outliers from point correspondences across multiple images.To address this problem,a robust deep convolutional neural network is proposed to accurately recover the 3D projective structure of scenes from point correspondences across multiple images.First,the network divides the input point correspondences into several different subsets,and each subset acts independently for a projective reconstruction;then,the weight of each projective reconstruction is estimated through a weight-learning layer;finally,a merging layer is activated to perform weighted summation of these different projective reconstructions to get the final robust projective reconstruction.Experimental results demonstrate both the reconstruction accuracy and strong robustness of our network.