SAMWave: Adapting Segment Anything Model to difficult tasks


Saurabh Yadav (IIIT Delhi), Avi Gupta (IIIT Delhi), Koteswar Rao Jerripothula (IIT Kanpur)
The 35th British Machine Vision Conference
PDFPosterVideo (Right click to download)Supplementary

Abstract

The emergence of large vision models has propelled significant advances in various domains. The Segment Anything Model (SAM), a leading model for image segmentation, exemplifies these advances, outperforming traditional methods. However, such foundation models often suffer from performance degradation when applied to complex tasks for which they are not trained. Existing methods typically employ adapter-based fine-tuning strategies to adapt SAM for difficult tasks and leverage high-frequency features extracted from the Fourier domain. However, our analysis reveals that these approaches offer limited benefits due to constraints in their feature extraction techniques. To overcome this, we propose **SAMWave**, a novel and interpretable approach that utilizes the wavelet transform to extract richer, multi-scale high-frequency features from input data. Extending this, we introduce complex-valued adapters capable of capturing complex-valued information via complex wavelet transforms. By adaptively integrating these wavelet coefficients, SAMWave enables SAM's encoder to capture more relevant information for dense prediction. Empirical evaluations on four difficult vision tasks demonstrate that SAMWave significantly outperforms existing adaptation methods. This superior performance is consistent across both the SAM and SAM2 backbones and holds for both real and complex-valued adapter variants, highlighting the efficiency, flexibility, and interpretability of our proposed method for adapting segment anything models.

Citation

@inproceedings{Yadav_2025_BMVC,
author    = {Saurabh Yadav and Avi Gupta and Koteswar Rao Jerripothula},
title     = {SAMWave: Adapting Segment Anything Model to difficult tasks},
booktitle = {36th British Machine Vision Conference 2025, {BMVC} 2025, Sheffield, UK, November 24-27, 2025},
publisher = {BMVA},
year      = {2025},
url       = {https://bmva-archive.org.uk/bmvc/2025/assets/papers/Paper_698/paper.pdf}
}


Copyright © 2025 The British Machine Vision Association and Society for Pattern Recognition
The British Machine Vision Conference is organised by The British Machine Vision Association and Society for Pattern Recognition. The Association is a Company limited by guarantee, No.2543446, and a non-profit-making body, registered in England and Wales as Charity No.1002307 (Registered Office: Dept. of Computer Science, Durham University, South Road, Durham, DH1 3LE, UK).

Imprint | Data Protection