高级检索

基于生成式特征图的深度学习模型局部性能可视分析方法

Visual Analysis Method for Local Performance of Deep Learning Models Based on Generative Feature Maps

  • 摘要: 现有的分类模型性能评估方法主要集中于整体性能, 而在准确性要求较高的应用中, 局部性能评估十分重要. 针对深度学习模型局部性能评估面临的样本数据多样性和灵活探索任务的挑战, 提出一种基于生成式特征图的深度学习模型局部性能可视化分析方法. 首先训练潜空间语义方向发现模型, 生成细微语义变化的测试样本, 为模型评估提供可控的测试样本; 然后设计一个集成方向选择器与性能探索器的可视化系统, 其中, 方向选择器支持用户生成指定语义变化的测试样本, 性能探索器由样本图和性能图组成, 样本图展示局部样本分布, 性能图通过颜色编码直观地展示模型在局部样本上的性能分布. 实验结果表明, 所提出的潜空间语义方向发现模型所发现的方向在准确性、可解释性和样本变化自然度方面, 均优于随机方向和坐标轴方向; 在MNIST和CIFAR-10数据集上, 生成式特征图能够有效地识别和解释分类模型在局部样本上的性能变化, 并找出有助于发现模型缺陷的局部样本.

     

    Abstract: Existing classification model performance evaluation methods mainly focus on the overall performance, but local performance evaluation is very important in applications with high accuracy requirements. Aiming at the challenges of sample data diversity and flexible exploration tasks faced by the local performance evaluation of deep learning models, we propose a visual analysis method based on generative feature maps for the local performance of deep learning models. Firstly, we train a latent space semantic direction discovery model to generate test samples with subtle semantic variations to provide controllable test samples for model evaluation; then, we design a visualization system integrating a direction selector and a performance explorer, in which the direction selector supports the user to generate test samples with specified semantic variations, and the performance explorer consists of a sample graph and a performance graph, where the sample graph displays the distribution of the local samples, and the performance graph visualizes the model’s performance in the local area through colour coding. The performance graph is colour coded to visually display the model’s performance distribution over local samples. Experimental results show that the directions discovered by the proposed latent space semantic direction discovery model outperform the random directions and axes directions in terms of accuracy, interpretability, and naturalness of sample variations; the generative feature maps can effectively identify and explain the performance variations of the classification model on local samples on both MNIST and CIFAR-10 datasets, and identify the local samples that can help to discover the defects of the model samples.

     

/

返回文章
返回