Single-View 3D Face Reconstruction via Cross-View Consistency Constraints

Deep neural network-based unsupervised single-view 3D face reconstruction has achieved remarkable success. Existing work relied on the photometric rendering constraint and the symmetric regularization to learn from 2D single-view facial images. However, the single view facial images lack reliable face geometric and texture constraints due to self-occlusion and illumination variations. In this paper, we propose a two-stage single-view 3D face reconstruction framework by virtue of cross-view consistent constraints. First, the part network (PartNet) with parallel branches is used to estimate the view-dependent pixel-wise UV positional and albedo maps. The missing geometries and textures due to self-occlusion are filled by the low-dimensional statistical facial 3DMM model. Second, the complete network (CompNet) is used to refine the UV positional and albedo maps with geometry and texture details. We design a cross-view consistency constraint in terms of photometric rendering, facial texture, and UV positional maps. The proposed end-to-end model is optimized from the multi-view facial image datasets in an unsupervised manner. Experiments show that the proposed method is effective in accurately aligning faces and inferring reliable facial geometries and textures in self-occlusion regions from a single-view image. Our method is feasible to reconstruct high-fidelity 3D faces with geometry and texture details. Specifically, the proposed method reduces the root mean square error by 6.8% compared with the state-of-the-art on MICC Florence dataset.