Visual Analysis Method for Compositional Understanding in Vision-Language Models

Vision-language pre-training models have shown strong cross-modal understanding, yet their “compositional understanding” remains underexplored. Existing research often focuses on quantitative metrics and model architectures, lacking dynamic tools to probe cross-modal alignment. We propose an interactive visual analysis approach that reveals how these models attend to individual vision entities. By optimizing traditional grid layouts, we enhance the visual perception of alignment patterns on large-scale datasets and interpret the role of multi-head attention in cross-modal understanding. User evaluations show that 90% of participants found this method more insightful and innovative than metric-based approaches, especially for studying modality isolation in vision-language models.