Pointly-Supervised Weak-Shot Semantic Segmentation via Dual Mapping Transfer


Wenhui Jiang (Jiangxi University of Finance and Economics), Ruikang Luo (Jiangxi University of Finance and Economics), Zeyu Luo (Jiangxi University of Finance and Economics), Xiaowei Zhao (Sany Heavy Industry Co.), Junjie Chen (Jiangxi University of Finance and Economics), Yuming Fang (Jiangxi University of Finance and Economics)
The 35th British Machine Vision Conference
PDFPosterVideo (Right click to download)

Abstract

Despite the existing large-scale datasets with full annotations, semantic segmentation still suffers from high annotation cost when extending to novel classes. One learning paradigm to relieve such annotation burden is weak-shot semantic segmentation, which jointly uses full annotations for base classes and weak annotations (image-level labels) for novel classes. In this work, we follow the weak-shot learning paradigm, but use more informative point-level annotations for novel classes. Our method is built upon query-based segmentation network, which learns a query-to-mask mapping. With annotated points (keypoints), we additionally learn a keypoint-to-mask mapping. Both mappings are transferable from base classes to novel classes. Moreover, the latter mapping could help the former mapping when learning to segment novel classes. Besides, we design a confident memory bank to alleviate the ambiguity of novel classes. Extensive experiments on COCO-Stuff-10K and ADE20K datasets show the superiority of our method.

Citation

@inproceedings{Jiang_2025_BMVC,
author    = {Wenhui Jiang and Ruikang Luo and Zeyu Luo and Xiaowei Zhao and Junjie Chen and Yuming Fang},
title     = {Pointly-Supervised Weak-Shot Semantic Segmentation via Dual Mapping Transfer},
booktitle = {36th British Machine Vision Conference 2025, {BMVC} 2025, Sheffield, UK, November 24-27, 2025},
publisher = {BMVA},
year      = {2025},
url       = {https://bmva-archive.org.uk/bmvc/2025/assets/papers/Paper_56/paper.pdf}
}


Copyright © 2025 The British Machine Vision Association and Society for Pattern Recognition
The British Machine Vision Conference is organised by The British Machine Vision Association and Society for Pattern Recognition. The Association is a Company limited by guarantee, No.2543446, and a non-profit-making body, registered in England and Wales as Charity No.1002307 (Registered Office: Dept. of Computer Science, Durham University, South Road, Durham, DH1 3LE, UK).

Imprint | Data Protection