A Visual Method for Diagnosing and Mitigating the Robustness of Transfer Learning Models

Although transfer learning enables developers to build a custom model (student model) that meets the target task based on a complex pre-trained model (teacher model), the student model in transfer learning may inherit the defects in the teacher model, and model robustness is one of the important indicators to measure the inheritance of model defects. In the field of transfer learning, the method of defect mitigation or joint training of student model and teacher model is usually used to reduce the defect knowledge inherited from teacher model. Therefore, this paper proposes a visual analysis method to explore the changes of model robustness in the process of transfer learning, and constructs the corresponding prototype system TLMRVis. Firstly, the robustness index of the student model is calculated. Secondly, the performance of each type of model is shown at the data instance level. Then, at the level of case features, the reuse knowledge inherited between teacher model and student model is revealed through model abstraction. Finally, the model slicing method is used to improve the defect inheritance of the model to improve the robustness of the model. At the same time, TLMRVis system not only displays the similarities and differences between various student models and teacher models with various visualization methods, but also introduces defect mitigation technology to view and diagnose the performance changes of teacher models and student models and the underlying predictive behavior mechanism. The experimental results of two cases show that, TLMRVis system can help users analyze the robustness of the model in transfer learning, the inherited defect knowledge of the model, and the performance changes after the improvement of the model defect.