ALSA: Anchors in Logit Space for Out-of-Distribution Accuracy Estimation


Chenzhi Liu (The University of Queensland), Mahsa Baktashmotlagh (The University of Queensland), Yanran Tang (The University of Queensland), Zi Huang (University of Queensland), Ruihong Qiu (The University of Queensland)
The 35th British Machine Vision Conference
PDFPosterVideo (Right click to download)Supplementary

Abstract

Estimating model accuracy on unseen, unlabeled datasets is crucial for real-world machine learning applications, especially under distribution shifts that can degrade performance. Existing methods often rely on predicted class probabilities (softmax scores) or data similarity metrics. While softmax-based approaches benefit from representing predictions on the standard simplex, compressing logits into probabilities leads to information loss. Meanwhile, similarity-based methods can be computationally expensive and domain-specific, limiting their broader applicability. In this paper, we introduce ALSA (Anchors in Logit Space for Accuracy estimation), a novel framework that preserves richer information by operating directly in the logit space. Building on theoretical insights and empirical observations, we demonstrate that the aggregation and distribution of logits exhibit a strong correlation with the predictive performance of the model. To exploit this property, ALSA employs an anchor-based modeling strategy: multiple learnable anchors are initialized in logit space, each assigned an influence function that captures subtle variations in the logits. This allows ALSA to provide robust and accurate performance estimates across a wide range of distribution shifts. Extensive experiments on vision, language, and graph benchmarks demonstrate ALSA’s superiority over both softmax- and similarity-based baselines. Notably, ALSA’s robustness under significant distribution shifts highlights its potential as a practical tool for reliable model evaluation. Code has been released at \href{https://github.com/chenzhi-liu/ALSA}{https://github.com/chenzhi-liu/ALSA}.

Citation

@inproceedings{Liu_2025_BMVC,
author    = {Chenzhi Liu and Mahsa Baktashmotlagh and Yanran Tang and Zi Huang and Ruihong Qiu},
title     = {ALSA: Anchors in Logit Space for Out-of-Distribution Accuracy Estimation},
booktitle = {36th British Machine Vision Conference 2025, {BMVC} 2025, Sheffield, UK, November 24-27, 2025},
publisher = {BMVA},
year      = {2025},
url       = {https://bmva-archive.org.uk/bmvc/2025/assets/papers/Paper_705/paper.pdf}
}


Copyright © 2025 The British Machine Vision Association and Society for Pattern Recognition
The British Machine Vision Conference is organised by The British Machine Vision Association and Society for Pattern Recognition. The Association is a Company limited by guarantee, No.2543446, and a non-profit-making body, registered in England and Wales as Charity No.1002307 (Registered Office: Dept. of Computer Science, Durham University, South Road, Durham, DH1 3LE, UK).

Imprint | Data Protection