MedOpenSeg: Open-World Medical Segmentation with Memory-Augmented Transformers


Luisa Vargas (Eurecom), Eleonora Poeta (Politecnico di Torino), Tania Cerquitelli (Politecnico di Torino), Elena Baralis (Politecnico di Torino), Maria A Zuluaga (Eurecom)
The 35th British Machine Vision Conference
PDFPosterVideo (Right click to download)Supplementary

Abstract

Open-world segmentation in medical imaging presents unique challenges, as models must generalize to seen and unseen classes while retaining knowledge of previously seen structures. We propose MedOpenSeg, a Memory-Augmented transformer framework that dynamically stores and updates class prototypes to enhance segmentation accuracy, improve adaptability to new anatomical structures, and detect novel regions during inference. MedOpenSeg integrates a Swin-Transformer 3D backbone with a memory bank module that retrieves class-specific feature embeddings and facilitates prototype-based novelty detection using cosine similarity and Euclidean Distance Sum (EDS). We benchmark MedOpenSeg on multiple datasets against state-of-the-art closed-set segmentation and foundation models, demonstrating its effectiveness in handling open-set medical segmentation. Code is publicly available at https://github.com/robustml-eurecom/MedOpenSeg.git

Citation

@inproceedings{Vargas_2025_BMVC,
author    = {Luisa Vargas and Eleonora Poeta and Tania Cerquitelli and Elena Baralis and Maria A Zuluaga},
title     = {MedOpenSeg: Open-World Medical Segmentation with Memory-Augmented Transformers},
booktitle = {36th British Machine Vision Conference 2025, {BMVC} 2025, Sheffield, UK, November 24-27, 2025},
publisher = {BMVA},
year      = {2025},
url       = {https://bmva-archive.org.uk/bmvc/2025/assets/papers/Paper_989/paper.pdf}
}


Copyright © 2025 The British Machine Vision Association and Society for Pattern Recognition
The British Machine Vision Conference is organised by The British Machine Vision Association and Society for Pattern Recognition. The Association is a Company limited by guarantee, No.2543446, and a non-profit-making body, registered in England and Wales as Charity No.1002307 (Registered Office: Dept. of Computer Science, Durham University, South Road, Durham, DH1 3LE, UK).

Imprint | Data Protection