A Visual Framework for Diagnosing and Mitigating the Robustness of Transfer Learning Models

    Transfer learning, a popular model reuse technique in the deep learning community, enables developers to build custom models (student models) that fit the target task based on complex pre-trained models (teacher models). However, just like the defect inheritance in traditional software reuse, the student model in transfer learning may also inherit the defect of the teacher model, which is reflected in the adversarial robustness of the model. In the field of deep learning, the method of defect mitigation or joint training of a student model and a teacher model is usually used to reduce the defects of the inherited teacher model and improve the robustness of the model. These methods are generally inefficient and not general, so this paper proposes a visual analysis framework to explore changes in the adversarial robustness of model. In this framework, the method of model slicing is added to alleviate model defect inheritance, and the performance changes and underlying behavior mechanism of the teacher model and the student model are checked and diagnosed from multiple dimensions such as overall model performance, data instance, instance feature and local structure of model. Experiments show that this framework can help users analyze the robustness of the model in transfer learning, the inherited defect knowledge of the model and the performance change after the im-provement of the model defect.