A multi-modal visual analytic approach for attributing and authenticating ancient Chinese paintings

The traditional workflow of attributing and authenticating ancient Chinese paintings has heavily relied on experts’ reasoning and justification, who need to consider various information, such as inscriptions, seals, and painting content, as well as extensive literature materials. The recent advances deep learning-based methods have improved the efficiency of traditional attribution and authentication workflow, but the black-box nature of DNNs has brought new challenges to effective human-machine collaboration. To address this issue, we propose a visual analysis framework that extract multi-modal semantics to facilitate the easy analysis of ancient Chinese paintings. We also develop a mixed- initiative system to empower users with object extraction, content association matching, and literature retrieval, which deeply involve experts in the analytic process. Finally, we validate the effectiveness of our framework using case studies and a user study.