MO-SHW: Hierarchy-Aware Multi-Objective Optimization for Open-World Segmentation


Erico M. Pereira (Universidade Federal de Minas Gerais), Frederico Gadelha Guimarães (Universidade Federal de Minas Gerais), Jefersson A Dos Santos (University of Sheffield)
The 35th British Machine Vision Conference
PDFPosterVideo (Right click to download)Supplementary

Abstract

The exploitation of hierarchical information by vision models has shown significant benefits in various segmentation tasks. However, this remains largely unexplored in open-world scenarios, where models must cope with unknown, evolving, and underrepresented labeled class spaces. Most existing hierarchy-aware segmentation approaches are not readily applicable to open-world settings. This is primarily because they rely on architectural modifications that are incompatible with the design constraints of open-world models. Moreover, hierarchy-aware losses are challenging to integrate into such pipelines, as they often conflict with task-specific objectives and exacerbate optimization complexity in already multi-objective training environments. In this work, we demonstrate that hierarchy-aware losses can be effectively leveraged in open-world models when optimized under a multi-objective learning framework. Specifically, we show that gradient-based multi-objective optimization methods, such as multi-objective gradient descent (MOGD), are well-suited for jointly optimizing hierarchical and task-specific objectives, leading to better overall performance. To support this, we propose SHW, a novel hierarchy-aware loss function based on the Wasserstein distance. SHW is lightweight, model-agnostic, and encourages intra-class compactness and inter-class separation across multiple semantic levels. The integration of SHW with MOGD yields a general, model-agnostic framework that enables the effective exploitation of semantic hierarchies in open-world segmentation tasks, improving the performance of several recent methods.

Citation

@inproceedings{Pereira_2025_BMVC,
author    = {Erico M. Pereira and Frederico Gadelha Guimarães and Jefersson A Dos Santos},
title     = {MO-SHW: Hierarchy-Aware Multi-Objective Optimization for Open-World Segmentation},
booktitle = {36th British Machine Vision Conference 2025, {BMVC} 2025, Sheffield, UK, November 24-27, 2025},
publisher = {BMVA},
year      = {2025},
url       = {https://bmva-archive.org.uk/bmvc/2025/assets/papers/Paper_1180/paper.pdf}
}


Copyright © 2025 The British Machine Vision Association and Society for Pattern Recognition
The British Machine Vision Conference is organised by The British Machine Vision Association and Society for Pattern Recognition. The Association is a Company limited by guarantee, No.2543446, and a non-profit-making body, registered in England and Wales as Charity No.1002307 (Registered Office: Dept. of Computer Science, Durham University, South Road, Durham, DH1 3LE, UK).

Imprint | Data Protection