高级检索

实时阴影的时域可靠网络去噪

Temporal Reliable Neural Denoising for Real-time Shadow

  • 摘要: 基于蒙特卡洛光线追踪的实时、可交互渲染通常需要低采样率下的时域去噪技术满足渲染质量、速度的双重需求. 时间域帧间映射信息一般通过计算屏幕空间中的运动向量来得到, 但对于存在阴影的区域, 使用传统运动向量则会产生错误的映射关系, 从而导致阴影鬼影等问题. 为此, 设计了一个针对动态阴影的多尺度分层去噪网络. 首先通过主干网络提取输入噪声阴影和辅助信息中的多尺度特征; 然后经过核预测模块生成逐像素的滤波核; 接着在时域融合模块中引入时域可靠阴影运动向量在阴影区域构建准确的帧间映射关系, 以更加有效地利用连续帧信息; 最后, 对噪声图像使用滤波核, 实现实时动态阴影的高质量去噪. 文中采用Falcor渲染器光线追踪流程生成数据集, 其包括2000帧渲染图像. 实验结果表明, 文中方法适用于复杂场景动态阴影的实时去噪, 尤其是存在高速移动的大面积光场景中, 单帧去噪时间低于14毫秒. 与已有方法相比, 文中方法渲染质量PSNR提升约5.4%, 帧间连贯性(tPSNR)提升约1.7%, 能在有效地消除软阴影噪声的同时获取更准确的阴影细节, 且有效地解决了光源、视点、物体快速移动时的阴影鬼影问题.

     

    Abstract: In the Monte Carlo ray tracing based real-time and interactive rendering applications, the temporal denoising technology at low sampling rate is required to achieve high-quality and high-performance renderings. The temporal inter frame mapping information usually relies on the motion vector in the screen space. However, the traditional motion vector approach fails at the areas with shadows, leading to shadow ghosts and other issues. Therefore, a multi-scale hierarchical denoising network for dynamic shadows is designed. Firstly, the multi-scale features of input noise shadow and auxiliary information are extracted through the backbone network. Then the per pixel filter kernel is generated by the kernel prediction module. Next, the temporal reliable shadow motion vector is introduced into the temporal fusion module to construct accurate inter-frame mapping relationship in the shadow region. It can make more effective use of continuous frame information. Finally, the filter kernel is applied to the noisy image to achieve high quality denoising of real-time dynamic shadows. In this paper, Falcor renderer ray tracing process is used to generate dataset, which includes 2000 frames of rendered images. The experimental results show that the proposed method is suitable for real-time denoising of dynamic shadows in complex scenes, especially in large area light scenes with high-speed movement, and the time of single frame denoising is less than 14 ms. Compared with the existing method, the rendering quality (PSNR) of the proposed method is improved by about 5.4%, and the interframe coherence (tPSNR) is improved by about 1.7%, which can effectively eliminate soft shadow noise while obtaining more accurate shadow details, and effectively solve the shadow ghost problem when the light source, view point, and object are moving fast.

     

/

返回文章
返回