3D Human Body Reconstruction with Shadow

Body shadow cast by lights om the ground can provide constraint information for human body reconstruction, which helps to alleviate the depth ambiguity and self-occlusion issues in a single image. In this paper, a deep neural network model based on shadow input is proposed for reconstructing 3D body mesh. First, two independ-ent convolutional neural networks are constructed to respectively extract image feature and shadow feature, and then estimate corresponding body model parameters. Then, a differentiable projection operator (DPO) is formu-lated to align the outputs of two networks. The loss function built on DPO is applied in two scenarios, namely fi-ne-tuning the neural network and directly refining reconstruction results. Experiments performed on Human3.6M with synthetic shadow showed that mean per joint position error of the fine-tuned network decreased from 1.4 mm to 4.9 mm compared with baseline methods. A small-scale dataset of 3 actors and 2 views with the real shadow is captured. Qualitative results on this dataset demonstrated that directly refined body pose better matches the original image.