An Unsupervised Detail-Preserving Point Cloud Completion Network Guided by Projection Views

Traditional supervised learning based method for repairing point cloud shapes typically requires  the complete point cloud data as a prior information, which often leads to its poor generalization and low robustness. Meanwhile, the point completion results generated by the existing unsupervised learning approaches often deviate from the input shape itself, making it difficult to recover the fine details of the original shape. Owing to the network framework of generative adversarial network (GAN), an unsupervised detail preserving point cloud completion network is proposed which is guided by the feature information of three projection views obtained from the underlying shape. The network branch of point cloud shape repairing adopts a point cloud generator to initially generate the complete shape data, which can recover its overall structure of the input shape but always lose its shape details. To compensate for the detail information of the input shape, the network branch of projection view completion firstly obtains the three-view projection images of the missing point cloud shape. Then, the point cloud generator with a tree-shaped convolution module is adopted to repair three projection images of the underlying shape, obtaining the completed views with detail preservation. Next, the feature extraction network Resnet-18 is applied to extract image features from the generated projection views, and the feature distances between the three views are calculated and also aligned. The feature distance between these aligned image features and the shape features extracted from the generated point cloud can thus be calculated, and also be added to the loss function of the discriminator which is employed to judge the truth or falsehood of the generated point cloud. Finally, network parameters of the point cloud generator can be optimized to generate a completed 3d shape with detail preservation. The proposed unsupervised learning based network is trained and tested on ShapeNet dataset for shape completion task and also validated on the KITTI and ModelNet40 datasets. Compared with the existing unsupervised completion networks, the average CD error of our network is reduced by 11%~41%, and the average F1-Scoret is improved by 0.8%~14%, which can demonstrate its effectiveness for repairing shape structure of the input point cloud data and also recovering its shape details. In addition, our point completion network is robust to different degrees of data incompleteness or model noise, and also shows its generalization performance on unseen objects.