高级检索
邢燕, 马俊, 檀结庆. 保特征的单幅图像三维网格重建[J]. 计算机辅助设计与图形学学报, 2023, 35(3): 354-361. DOI: 10.3724/SP.J.1089.2023.19375
引用本文: 邢燕, 马俊, 檀结庆. 保特征的单幅图像三维网格重建[J]. 计算机辅助设计与图形学学报, 2023, 35(3): 354-361. DOI: 10.3724/SP.J.1089.2023.19375
Xing Yan, Ma Jun, and Tan Jieqing. Feature Preserving Mesh Reconstruction from a Single Image[J]. Journal of Computer-Aided Design & Computer Graphics, 2023, 35(3): 354-361. DOI: 10.3724/SP.J.1089.2023.19375
Citation: Xing Yan, Ma Jun, and Tan Jieqing. Feature Preserving Mesh Reconstruction from a Single Image[J]. Journal of Computer-Aided Design & Computer Graphics, 2023, 35(3): 354-361. DOI: 10.3724/SP.J.1089.2023.19375

保特征的单幅图像三维网格重建

Feature Preserving Mesh Reconstruction from a Single Image

  • 摘要: 针对图像重建三维物体方法中存在无法保持物体尖锐特征的问题, 基于深度神经网络, 对输入单幅图像提出一种有效的保特征三维网格生成方法. 对单幅输入图像使用 VGG-16 提取图像特征, 并特别设计了图像边缘检测层获取物体的尖锐特征; 将三维网格(初始为椭球)的顶点投影到特征图和边缘检测图上, 以获得顶点局部特征, 并判断其是否为尖锐特征点; 然后, 将局部特征和顶点位置串联输入到改进的图卷积神经网络(graph convolutional neuralnetwork, GCNN), 对于非尖锐特征点采用普通 GCNN, 对于检测到的尖锐特征点采用 0 邻域图卷积神经网络(0-neighborhood GCNN, 0N-GCNN), 以期其尽量不被邻域顶点过度光滑; GCNN 的输出预测了顶点的新位置和三维特征; 最后, 对网格的顶点及特征用 Loop 细分上采样. 执行 3 次上述变形(二维特征投影、尖锐特征检测、GCNN 变形、上采样)后, 初始椭球最终变形为输入图像中物体模样. 实验使用 ShapeNet 数据集, 在 PyTorch 框架下实现, 从定性和定量两方面与现有方法进行了比较. 实验结果表明, 在 Chamfer 距离和 F-score 两类定量指标上均优于大部分现有方法, 而 Chamfer 距离和 F-score(2τ)的均值表现为最优. 视觉比较也表明, 文中方法可有效地提升特征保持性能.

     

    Abstract: The reconstruction of 3D objects from image with the problem of failing to maintain sharp features of objects. Based on deep neural network, an effective feature preserving 3D mesh generation method is proposed for a single input image in this paper. Firstly, image features are extracted using VGG-16 for the input images, and the image edge detection layer is specially designed to obtain the sharp features. Secondly, the vertices of the mesh (initially ellipsoid) are projected onto the feature map and edge detection map to obtain the local features of the vertices, and judge whether they are sharp feature points. Thirdly, the local features and positions of the vertices are concatenated and input into the improved graph convolution neural network (GCNN). For the non-sharp feature points, the ordinary GCNN is used, and for the detected sharp feature points, the 0-neighborhood graph convolution neural network (0N-GCNN) is used to avoid being over-smoothed by the neighboring vertices as much as possible. The output of GCNN predicts the new position and features of the vertices. Finally, the vertices and features of the mesh are up sampled by Loop subdivision. After going through above deformation process (2D feature projection, sharp feature detection, deformation by GCNN, upsampling) three times, the initial ellipsoid is finally transformed into the shape in the input image. The experiments are implemented on ShapeNet dataset based on PyTorch framework. The proposed method is compared with the existing methods quantitatively and qualitatively. The experimental results show that this method is superior to most existing methods in both Chamfer distance and F-score, and the mean values of Chamfer distance and F-score( 22τ ) are the best. Visual comparison also shows that this method effectively improves the feature preservation performance.

     

/

返回文章
返回