Abstract: Inverse rendering of objects aims to recover the object geometry and surface materials from multi-view images. Recent methods have demonstrated impressive performance on this task by representing object materials with neural networks and optimizing the network parameters through physically based rendering, where they typically assume that the lighting is infinitely far away. However, this assumption seldomly holds in indoor scenarios that exhibit complex illumination. To resolve these problems, we introduce a novel illumination representation to model the spatially-varying and high-frequency lighting in indoor scenes, which is implemented as a set of emitters that are automatically localized based on input images. These emitters are the main cause of specular visual effects.  Explicitly incorporating emitters in the Monte Carlo sampling greatly improves the ability to capture the specular effects, thus effectively alleviating the ambiguity in the inverse rendering process.