Catch Your Concepts: A Flexible Concept Locator for Interpretable Visual Recognition


Qiyang Wan (Institute of Computing Technology, Chinese Academy of Sciences (CAS)), Ruiping Wang (Institute of Computing Technology, Chinese Academy of Sciences (CAS)), Chengzhi Gao (Institute of Computing Technology, Chinese Academy of Sciences (CAS)), Xilin CHEN (Institute of Computing Technology, Chinese Academy of Sciences (CAS))
The 35th British Machine Vision Conference
PDFPosterVideo (Right click to download)Supplementary

Abstract

The interpretability of visual recognition models has attracted significant attention, particularly with the advancement of concept-based methods exemplified by Concept Bottleneck Models (CBMs), which decompose categories as a set of representations to human perceivable concepts. While much of the prior research has contributed to extending concept sets to enhance the semantic interpretability of the models, the spatially activated region of the concept, or localization interpretability, has been routinely neglected. Based on insights from cognitive psychology, both types of interpretability are crucial for constructing explanations that enhance human understanding and trust. This paper introduces a flexible concept locator, which serves as a simple yet effective plugin for localizing concepts in CBMs during training, and providing both semantic and localization explanations during inference. A set of dynamic masks, applicable to various backbones and gradually shrinking during the training steps, is employed to constrain the activation region of concept neurons. This allows the concepts used for category recognition to be concentratedly expressed in specific spatial areas. Experiments conducted on multiple datasets demonstrate that the proposed locator effectively locates concepts in CBMs, offering additional localization explanations at a relatively low cost.

Citation

@inproceedings{Wan_2025_BMVC,
author    = {Qiyang Wan and Ruiping Wang and Chengzhi Gao and Xilin CHEN},
title     = {Catch Your Concepts: A Flexible Concept Locator for Interpretable Visual Recognition},
booktitle = {36th British Machine Vision Conference 2025, {BMVC} 2025, Sheffield, UK, November 24-27, 2025},
publisher = {BMVA},
year      = {2025},
url       = {https://bmva-archive.org.uk/bmvc/2025/assets/papers/Paper_707/paper.pdf}
}


Copyright © 2025 The British Machine Vision Association and Society for Pattern Recognition
The British Machine Vision Conference is organised by The British Machine Vision Association and Society for Pattern Recognition. The Association is a Company limited by guarantee, No.2543446, and a non-profit-making body, registered in England and Wales as Charity No.1002307 (Registered Office: Dept. of Computer Science, Durham University, South Road, Durham, DH1 3LE, UK).

Imprint | Data Protection