Joint Optimization for Deblurring and 3D Reconstruction of Blurry Face Image

In order to address the issue of low performance in deblurring and 3D reconstruction on blurred face images due to lack of information, a joint optimization method for deblurring and 3D reconstruction of blurry face images is proposed. Given a blurry face image, the following steps are iteratively executed: apply 3D reconstruction on the face image to generate a 3D facial model; deblur the face image using Res-UNet, with 3D reconstruction information as assist input and the original blurry image as correction input, to generate a deblurred image; repeat three times. The experiment utilized approximately 20 000 blurred facial images synthesized from clear facial images from AFLW dataset and blur kernels collected by Giacomo Boracchi as dataset. Peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) were calculated from re-rendered images from the 3D face model and deblurred images to blurry input images respectively, as metrics of the performance of the deblurring and 3D reconstruction. For deblurring, in comparison to 3DFaceDeblur method, average PSNR improved from 27.38 dB to 29.23 dB, average SSIM improved from 0.833 to 0.855. For 3D reconstruction task, compared to Deep3DFace method, average PSNR improved from 22.78 dB to 27.16 dB, average SSIM improved from 0.848 to 0.918.