Hide and Seek: HIDE Concepts to SEEK explanations for black box models

TL;DR

Hide and Seek (HnS) enables concept-level attribution for black-box vision models without needing model weights or labeled concept data. It uses a vision-language model to find concept-aligned images, localize concepts, mask them, and measure output changes to derive attributions. The method supports diverse tasks and achieves higher alignment with ground-truth explanations than prior approaches.

Why It Matters

HnS makes model interpretation accessible for a broad range of vision systems, including those without open weights or concept annotations. It enables local, faithful explanations in complex tasks like image captioning and satellite segmentation, where previous methods fail. This advances trust and transparency in real-world AI applications.

Key Findings

• HnS operates without model weights or concept-labeled datasets, relying instead on vision-language models for concept retrieval and localization.
• The method produces image-level attributions by measuring output changes after masking concept regions, leading to more accurate and faithful explanations.
• HnS outperforms existing methods on CelebA and COCO-Stuff, and extends to tasks like captioning and segmentation where prior concept attribution methods are inapplicable.