高级检索

人脸图像去模糊和三维重建的联合优化方法

Joint Optimization for Deblurring and 3D Reconstruction of Blurry Face Image

  • 摘要: 为了解决在模糊人脸图像的去模糊任务和人脸三维重建任务存在的信息缺失导致的效果不佳的问题, 提出一种对模糊人脸图像去模糊和三维重建任务进行联合优化的方法. 对于输入的模糊人脸图像, 交替重复执行以下步骤 3次:首先, 对人脸图像进行三维重建, 生成三维人脸模型;然后, 使用 Res-UNet对人脸图像进行去模糊;并以三维重建所得信息作为辅助输入, 以原始模糊图像作为校正输入, 生成消除模糊的图像. 在 AFLW数据集的清晰人脸图像, 以及约 20 000幅 Giacomo Boracchi采集的模糊核合成模糊人脸图像组成的数据集上, 分别以去模糊图像和人脸三维模型再渲染的图像对输入图像计算峰值信噪比(peak signal-to-noise ratio, PSNR)和结构相似度(structural similarity, SSIM), 以衡量去模糊和三维重建任务效果. 对比实验结果表明, 在测试集上的去模糊任务中, 相比3DFaceDeblur方法, 该方法的平均 PSNR从 27. 38 dB提升到 29. 23 dB, 平均 SSIM从 0. 833提升到 0. 855;在三维重建任务中, 相比 Deep3DFace方法, 该方法的平均 PSNR从 22. 78 dB提升到 27. 16 dB, 平均 SSIM从 0. 848提升到 0. 918.

     

    Abstract: In order to address the issue of low performance in deblurring and 3D reconstruction on blurred face images due to lack of information, a joint optimization method for deblurring and 3D reconstruction of blurry face images is proposed. Given a blurry face image, the following steps are iteratively executed: apply 3D reconstruction on the face image to generate a 3D facial model; deblur the face image using Res-UNet, with 3D reconstruction information as assist input and the original blurry image as correction input, to generate a deblurred image; repeat three times. The experiment utilized approximately 20 000 blurred facial images synthesized from clear facial images from AFLW dataset and blur kernels collected by Giacomo Boracchi as dataset. Peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) were calculated from re-rendered images from the 3D face model and deblurred images to blurry input images respectively, as metrics of the performance of the deblurring and 3D reconstruction. For deblurring, in comparison to 3DFaceDeblur method, average PSNR improved from 27.38 dB to 29.23 dB, average SSIM improved from 0.833 to 0.855. For 3D reconstruction task, compared to Deep3DFace method, average PSNR improved from 22.78 dB to 27.16 dB, average SSIM improved from 0.848 to 0.918.

     

/

返回文章
返回