Temporal Reliable Neural Denoising for Real-time Shadow

In the Monte Carlo ray tracing based real-time and interactive rendering applications, the temporal denoising technology at low sampling rate is required to achieve high-quality and high-performance renderings. The temporal inter frame mapping information usually relies on the motion vector in the screen space. However, the traditional motion vector approach fails at the areas with shadows, leading to shadow ghosts and other issues. Therefore, a multi-scale hierarchical denoising network for dynamic shadows is designed. Firstly, the multi-scale features of input noise shadow and auxiliary information are extracted through the backbone network. Then the per pixel filter kernel is generated by the kernel prediction module. Next, the temporal reliable shadow motion vector is introduced into the temporal fusion module to construct accurate inter-frame mapping relationship in the shadow region. It can make more effective use of continuous frame information. Finally, the filter kernel is applied to the noisy image to achieve high quality denoising of real-time dynamic shadows. In this paper, Falcor renderer ray tracing process is used to generate dataset, which includes 2000 frames of rendered images. The experimental results show that the proposed method is suitable for real-time denoising of dynamic shadows in complex scenes, especially in large area light scenes with high-speed movement, and the time of single frame denoising is less than 14 ms. Compared with the existing method, the rendering quality (PSNR) of the proposed method is improved by about 5.4%, and the interframe coherence (tPSNR) is improved by about 1.7%, which can effectively eliminate soft shadow noise while obtaining more accurate shadow details, and effectively solve the shadow ghost problem when the light source, view point, and object are moving fast.