OmniSegNet: Towards Scalable, Efficient & Universal Medical Image Segmentation


Soma Dasgupta (Tata Consultancy Services), Swarnava Dey (Tata Consultancy Services), Arijit Mukherjee (Tata Consultancy Services), Arpan Pal (Tata Consultancy Services)
The 35th British Machine Vision Conference
PDFPosterVideo (Right click to download)Supplementary

Abstract

Medical image segmentation spans diverse modalities, including MRI (Magnetic Resonance Imaging), CT (Computed Tomography), OCT (Optical Coherence Tomography), and USG (Ultrasound Sonography), each with unique spatial and contextual characteristics. This heterogeneity demands architectures that balance global context with fine-grained anatomical details, a challenge for standard models like U-Net and Transformer-based approaches, which either lack generalization or struggle with subtle features. Privacy constraints further restrict the use of large models like the Segment Anything Model (SAM) for cloud-based inference, complicating edge deployment and increasing development complexity. To address these challenges, we propose OmniSegNet, a unified, scalable encoder-decoder architecture that integrates SE-enhanced residual blocks for efficient local-global context capture and Atrous spatial pyramid pooling (ASPP) for multi-scale feature aggregation. Compound scaling of depth ($\alpha$) and width ($\beta$) supports flexible model variants ranging from 1.5M to 6M parameters, identified through latency-aware neural architecture search for real-time deployment on devices like Raspberry Pi and Arduino Nano. To generalize without task-specific tuning, we introduce OmniKD, a unified distillation framework that transfers knowledge from fine-tuned SAM models via logits, intermediate features, attention maps, relational structures, and contextual similarities, eliminating the need for handcrafted loss functions. OmniSegNet achieves up to $10\times$ parameter reduction while improving Dice scores by 10--12\% across diverse medical imaging benchmarks, including ISIC, CHAOS, and MRBrainS18, offering a scalable, efficient, and privacy-preserving solution for real-world medical segmentation.

Citation

@inproceedings{Dasgupta_2025_BMVC,
author    = {Soma Dasgupta and Swarnava Dey and Arijit Mukherjee and Arpan Pal},
title     = {OmniSegNet: Towards Scalable, Efficient & Universal Medical Image Segmentation},
booktitle = {36th British Machine Vision Conference 2025, {BMVC} 2025, Sheffield, UK, November 24-27, 2025},
publisher = {BMVA},
year      = {2025},
url       = {https://bmva-archive.org.uk/bmvc/2025/assets/papers/Paper_528/paper.pdf}
}


Copyright © 2025 The British Machine Vision Association and Society for Pattern Recognition
The British Machine Vision Conference is organised by The British Machine Vision Association and Society for Pattern Recognition. The Association is a Company limited by guarantee, No.2543446, and a non-profit-making body, registered in England and Wales as Charity No.1002307 (Registered Office: Dept. of Computer Science, Durham University, South Road, Durham, DH1 3LE, UK).

Imprint | Data Protection