高级检索

结合张量分解与高频感知的多视图高质量合成

Combining Tensor Decomposition and High-Frequency Perception for Multi-View High-Quality Synthesis

  • 摘要: 针对传统张量辐射场(TensoRF)在视图合成任务中高维数据表达能力有限、高频特征捕捉能力较弱及模型体积较大导致的高内存开销问题, 构建轻量、紧凑型张量辐射场(lightweight compact TensoRF, LC-TensoRF)模型以实现复杂三维场景新视图的快速、高质量合成. 首先, 提出了一种新的张量分解方法M2以实现高维数据的低秩张量表达, 从而以更紧凑、更灵活的方式分解三维场景. 其次, 构建了一种新的张量解码器增强模块(TDEM), 通过使用固定傅里叶基对MLP权重进行重新参数化来减轻网络训练过程中的低频偏置问题, 并结合门控机制和残差网络结构提高模型对场景颜色的敏感度, 从而增强网络捕捉高频信息(尤其是纹理细节)的场景外观特征的能力. 最后, 通过搜索最佳体素剪枝率来减少TensoRF在M2分解和TDEM下的存储开销, 从而在保持高渲染质量的同时减少模型体积. 在Synthetic、Replica和UrbanScene3D等公开数据集上的大量实验表明, 本文模型可显著提高渲染质量且体积减少约50%.

     

    Abstract: To address the limitations of traditional Tensor Radiation Fields (TensoRF) in view synthesis tasks—specifically, their constrained expressiveness for high-dimensional data, weak capability in capturing high-frequency features, and significant memory overhead due to large model size—we construct a Lightweight Compact TensoRF (LC-TensoRF) model for fast, high-quality synthesis of novel views in complex 3D scenes. First, a novel tensor decomposition method, M2, is proposed to achieve low-rank tensor representation of high-dimensional data, enabling decomposition of 3D scenes in a more compact and flexible manner. Second, a Tensor Decoder Enhancement Module (TDEM) is developed. This module mitigates low-frequency bias during network training by reparameterizing MLP weights using fixed Fourier bases. It further incorporates gating mechanisms and residual network structures to enhance the model’s sensitivity to scene colors, thereby strengthening its ability to capture high-frequency information (particularly textural details) in scene appearance. Finally, storage overhead under M2 decomposition and TDEM is reduced by searching for the optimal voxel pruning rate, effectively minimizing model size while preserving high rendering quality. Extensive experiments on public datasets (Synthetic, Replica, and UrbanScene3D) demonstrate that the proposed model significantly improves rendering quality while reducing model size by approximately 50%.

     

/

返回文章
返回