Text-to-3D Object Generation with Grid Transformation and Material Illumination Separation

Aiming at the problems of slow reasoning speed, low diversity and Janus in 3D content generation, an end-to-end 3D content generation algorithm based on 3D Gaussian snowballing is proposed. Poisson reconstruction algorithm is used to extract meshes from Gaussian, and the materials and lighting in multi-view images are decomposed and reconstructed by point-based rendering framework, which makes it suitable for today’s 3D modeling software. The efficiency and quality of content generation can be further improved. Furthermore, a haptic augmented reality interactive system is built. The experimental results with the text-to-3D benchmark method designed by T³Bench show that the proposed algorithm has good generation diversity, and the normal diffusion model used effectively alleviates the Janus problem.