RP-SAM2: Refining Point Prompts for Stable Surgical Instrument Segmentation


Nuren Zhaksylyk (Mohamed bin Zayed University of Artificial Intelligence), Ibrahim Almakky (Mohamed bin Zayed University of Artificial Intelligence), Jay Nitin Paranjape (Johns Hopkins University), S. Swaroop Vedula (Johns Hopkins University), Shameema Sikder (Johns Hopkins University), Vishal M. Patel (Johns Hopkins University), Mohammad Yaqub (Mohamed bin Zayed University of Artificial Intelligence)
The 35th British Machine Vision Conference
PDFPosterVideo (Right click to download)

Abstract

Accurate surgical instrument segmentation is essential in cataract surgery for tasks such as skill assessment and workflow optimization. However, limited annotated data makes it difficult to develop fully automatic models. Prompt-based methods like SAM2 offer flexibility yet remain highly sensitive to the point prompt placement, often leading to inconsistent segmentations. We address this issue by introducing RP-SAM2, which incorporates a novel shift block and a compound loss function to stabilize point prompts. Our approach reduces annotator reliance on precise point positioning while maintaining robust segmentation capabilities. Experiments on the Cataract1k dataset demonstrate that RP-SAM2 improves segmentation accuracy, with a 2% mDSC gain, a 21.36% reduction in mHD95, and decreased variance across random single-point prompt results compared to SAM2. Additionally, on the CaDIS dataset, pseudo masks generated by RP-SAM2 for fine-tuning SAM2’s mask decoder outperformed those generated by SAM2. These results highlight RP-SAM2 as a practical, stable and reliable solution for semi-automatic instrument segmentation in data-constrained medical settings. The code for RP-SAM2 is available at: https://github.com/BioMedIA-MBZUAI/RP-SAM2.

Citation

@inproceedings{Zhaksylyk_2025_BMVC,
author    = {Nuren Zhaksylyk and Ibrahim Almakky and Jay Nitin Paranjape and S. Swaroop Vedula and Shameema Sikder and Vishal M. Patel and Mohammad Yaqub},
title     = {RP-SAM2: Refining Point Prompts for Stable Surgical Instrument Segmentation},
booktitle = {36th British Machine Vision Conference 2025, {BMVC} 2025, Sheffield, UK, November 24-27, 2025},
publisher = {BMVA},
year      = {2025},
url       = {https://bmva-archive.org.uk/bmvc/2025/assets/papers/Paper_1004/paper.pdf}
}


Copyright © 2025 The British Machine Vision Association and Society for Pattern Recognition
The British Machine Vision Conference is organised by The British Machine Vision Association and Society for Pattern Recognition. The Association is a Company limited by guarantee, No.2543446, and a non-profit-making body, registered in England and Wales as Charity No.1002307 (Registered Office: Dept. of Computer Science, Durham University, South Road, Durham, DH1 3LE, UK).

Imprint | Data Protection