具有网格转化和材质光照分离的文本到三维物体生成
Text-to-3D Object Generation with Grid Transformation and Material Illumination Separation
-
摘要: 针对3D内容生成仍然存在推理速度慢、多样性低以及Janus问题, 提出一种基于3D高斯抛雪球的3D内容端到端生成算法, 从高斯中使用泊松重建算法进行网格提取, 并采用基于点的渲染框架对多视图图像中的材质和照明进行了分解和重建, 使其能够适用于当今的3D建模软件, 并能进一步提升内容生成的效率和质量. 进一步的, 还搭建了一个触觉增强现实交互系统. 与采用T3Bench设计的文本到3D基准测试方法进行实验的结果表明, 所提算法具有较好的生成多样性, 且使用的法线扩散模型有效地缓解了Janus问题.Abstract: Aiming at the problems of slow reasoning speed, low diversity and Janus in 3D content generation, an end-to-end 3D content generation algorithm based on 3D Gaussian snowballing is proposed. Poisson reconstruction algorithm is used to extract meshes from Gaussian, and the materials and lighting in multi-view images are decomposed and reconstructed by point-based rendering framework, which makes it suitable for today’s 3D modeling software. The efficiency and quality of content generation can be further improved. Furthermore, a haptic augmented reality interactive system is built. The experimental results with the text-to-3D benchmark method designed by T3Bench show that the proposed algorithm has good generation diversity, and the normal diffusion model used effectively alleviates the Janus problem.