高级检索

时序引导与法线感知的高质量三维高斯人体重建

Temporal Guidance and Normal Awareness for High-Quality 3D Gaussian Human Reconstruction

  • 摘要: 单目视频输入的三维高斯人体重建中, 由于骨骼关联的姿态时序不连续性、高斯优化的几何尺度依赖性和高斯泼溅的纹理细节缺失性, 难以实现高质量重建和实时渲染. 为此, 提出时序引导与法线感知的高质量三维高斯人体重建方法. 首先基于输入视频帧的全身姿态信息, 通过时序引导姿态变形, 将规范空间初始化姿态进行全身关联变形, 得到观测空间姿态; 然后通过法线感知高斯优化对观测空间姿态进行高斯密集化与剪枝, 并采用法线图的监督进行优化, 得到高斯表示下的人体; 最后对法线图中提取的光影特征与球谐函数计算的颜色信息进行光影增强渲染, 采用高斯光栅化最终输出高斯表示下人体的渲染效果. 在ZJU-Mocap数据集上的新视角合成实验结果表明, 与对比方法相比, 所提方法的平均PSNR和LPIPS指标均达到最优, 且总和分别为189.95dB和171.6, 提升了1.91 dB和10.9, 能够高质量重建和实时渲染三维高斯人体.

     

    Abstract: High-quality 3D Gaussian human reconstruction from monocular video faces several challenges, such as pose discontinuity caused by skeletal articulation, geometric scale dependency in Gaussian optimization, and texture detail loss in Gaussian splatting. To address these issues, this paper proposes a temporal guid-ance and normal awareness method for high-quality 3D Gaussian human reconstruction. Firstly, based on full-body pose information from input video frames, the temporal-guided pose deformation transformed the canonical space initialization pose into an observation space pose through full-body articulated defor-mation. Then, a normal-aware Gaussian optimization was applied to densify and prune the Gaussian repre-sentation in the observation space, and the Gaussian representation of the human body can be achieved by optimized with normal maps supervision. Finally, shadow enhancement rendering combined shadow fea-tures extracted from normal maps with color information calculated by spherical harmonic functions, using Gaussian rasterization to produce the final rendered output of the Gaussian-represented human body. Ex-perimental results on the ZJU-MoCap dataset for novel view synthesis demonstrate that the proposed method achieves optimal performance in both average PSNR and LPIPS metrics, with totals of 189.95 dB and 171.6, respectively, outperforming comparison methods by 1.91 dB and 10.9, enabling high-quality reconstruction and real-time rendering of 3D Gaussian human bodies.

     

/

返回文章
返回