Cross-Modal Transformer for Point Cloud Completion

The point cloud obtained by 3D sensors (such as LiDAR and depth camera) is mostly incomplete and needs to be completed. Aiming at the problems of insufficient details and incomplete structure of single-modal point cloud completion methods, a cross-modal Transformer for point cloud completion is proposed. Point cloud features and image features are extracted by point cloud branch and image branch respectively. Point cloud branch adopts PoinTr as backbone, and image branch adopts 7 convolution layers. The feature fusion module fuses point cloud features and image features together to generate a full resolution point cloud in a coarse-to-fine manner. Experimental results indicate that the visualization of this method is better than the single-modal point cloud completion methods and the cross-modal point cloud completion method ViPC. Moreover, the CD-L₂ of this method is better than ViPC on most categories, and the average CD-L₂ is 2.74, which is 17% lower than ViPC. Our code is available at: https://github.com/Starak-x/ImPoinTr.