CLIMB-3D: Class-Incremental Imbalanced 3D Instance Segmentation


Vishal Thengane (University of Surrey), Jean Lahoud (Mohamed bin Zayed University of Artificial Intelligence), Hisham Cholakkal (Mohamed bin Zayed University of Artificial Intelligence), Rao Muhammad Anwer (Mohamed bin Zayed University of Artificial Intelligence), Lu Yin (University of Surrey), Xiatian Zhu (University of Surrey), Salman Khan (Mohamed bin Zayed University of Artificial Intelligence)
The 35th British Machine Vision Conference
PDFPosterVideo (Right click to download)Supplementary

Abstract

While 3D instance segmentation (3DIS) has advanced significantly, most existing methods assume that all object classes are known in advance and uniformly distributed. However, this assumption is unrealistic in dynamic, real-world environments where new classes emerge gradually and exhibit natural imbalance. Although some approaches address the emergence of new classes, they often overlook class imbalance, which leads to suboptimal performance, particularly on rare categories. To tackle this, we propose CLIMB-3D, a unified framework for CLass-incremental IMBalance-aware 3DIS. Building upon established exemplar replay (ER) strategies, we show that ER alone is insufficient to achieve robust performance under memory constraints. To mitigate this, we introduce a novel pseudo-label generator (PLG) that extends supervision to previously learned categories by leveraging predictions from a frozen model trained on prior tasks. Despite its promise, PLG tends to be biased towards frequent classes. Therefore, we propose a class-balanced re-weighting (CBR) scheme that estimates object frequencies from pseudo-labels and dynamically adjusts training bias, without requiring access to past data. We design and evaluate three incremental scenarios for 3DIS on the challenging ScanNet200 dataset and additionally validate our method for semantic segmentation on ScanNetV2. Our approach achieves state-of-the-art results, surpassing prior work by up to 16.76\% mAP for instance segmentation and approximately 30\% mIoU for semantic segmentation, demonstrating strong generalisation across both frequent and rare classes. Code is available at: https://github.com/vgthengane/CLIMB3D.

Citation

@inproceedings{Thengane_2025_BMVC,
author    = {Vishal Thengane and Jean Lahoud and Hisham Cholakkal and Rao Muhammad Anwer and Lu Yin and Xiatian Zhu and Salman Khan},
title     = {CLIMB-3D: Class-Incremental Imbalanced 3D Instance Segmentation},
booktitle = {36th British Machine Vision Conference 2025, {BMVC} 2025, Sheffield, UK, November 24-27, 2025},
publisher = {BMVA},
year      = {2025},
url       = {https://bmva-archive.org.uk/bmvc/2025/assets/papers/Paper_1091/paper.pdf}
}


Copyright © 2025 The British Machine Vision Association and Society for Pattern Recognition
The British Machine Vision Conference is organised by The British Machine Vision Association and Society for Pattern Recognition. The Association is a Company limited by guarantee, No.2543446, and a non-profit-making body, registered in England and Wales as Charity No.1002307 (Registered Office: Dept. of Computer Science, Durham University, South Road, Durham, DH1 3LE, UK).

Imprint | Data Protection