高级检索

基于多视图一致性分割的感兴趣目标三维重建方法

Multi-View Consistent Segmentation-Based Method for 3D Reconstruction of Objects of Interest

  • 摘要: 3D高斯溅射技术在三维重建领域具有重要研究价值, 针对现有高斯重建方法在复杂背景下直接重建目标对象的局限性, 提出了一种基于多视图一致性分割的感兴趣目标三维重建方法, 该方法采用“分割-重建”两阶段框架. 多视图分割时, 扩展多参考掩膜提示, 在特征提取之后, 加入跨视角注意力, 通过融合不同视角的前景特征, 提升特征表达一致性和准确性; 同时, 设计多视图一致性损失, 减少因视角变化导致的分割抖动. 目标重建时, 加入掩膜引导的重建分支, 利用前景掩膜优化3DGS训练, 并优化重建损失, 提高重建精度. 在MVSeg数据集上的实验结果表明, 所提方法在倒角距离评价指标上相较于SA3D、SPIn-NeRF、SAGA分别提升了43.5%、38.7%、7.1%, 并且在复杂背景和目标边界不清的场景中也表现出较高的重建精度.

     

    Abstract: 3D Gaussian Splatting has shown significant research potential in the field of 3D reconstruction. To address the limitations of existing Gaussian-based methods in directly reconstructing target objects under complex backgrounds, this paper proposes a novel two-stage framework for object reconstruction based on multi-view consistent segmentation. In the segmentation stage, multi-reference mask prompting is introduced and cross-view attention is incorporated after feature extraction to fuse foreground features from different viewpoints, thereby enhancing the consistency and accuracy of feature representations. Additionally, a multi-view consistency loss is designed to reduce segmentation jitter caused by viewpoint variations. In the reconstruction stage, a mask-guided branch is introduced to optimize the 3DGS training process using foreground masks, improving the reconstruction loss and overall accuracy. Experimental results on the MVSeg dataset demonstrate that the proposed method achieves superior performance in terms of Chamfer Distance, with improvements of 43.5%, 38.7%, and 7.1% over SA3D, SPIn-NeRF, and SAGA, respectively. The method also shows robust reconstruction accuracy in challenging scenes with complex backgrounds and ambiguous object boundaries.

     

/

返回文章
返回