Knowledge Distillation via Cross Supervising with Attention for Remote Sensing Object Detection


KefanZhan (Xiangtan University), An Luo (Xiangtan University), Yunpeng Zeng (Xiangtan University), Jiaxin Li (Xiangtan University), Yuan Zhang (Xiangtan University), Kai Hu (Xiangtan University)
The 35th British Machine Vision Conference
PDFPosterVideo (Right click to download)Supplementary

Abstract

Knowledge distillation, which serves as a model compression approach that centers on transferring knowledge from a teacher model to a more compact student model, has been extensively employed to derive lightweight models. However, when applied to remote sensing object detection (RSOD), the unique characteristics of remote sensing images, such as small objects and complex background, substantially impede the student model in effective assimilation of knowledge from its superior teacher model. In this paper, we present a novel self-distillation framework designated as Knowledge Distillation via Cross Supervising with Attention (CSAKD), in which the teacher will offer module-level mentoring and adaptive guidance to eliminate the performance discrepancy between the teacher and the student. Specifically, the teacher will have access to the intermediate features of the student, generate a new set of features, and provide additional Cross Supervising for the student. Meanwhile, we propose an Adaptive Attention Weight Modulation (AAWM) to dynamically adjust the intensity of Cross Supervising, which ensures that the student can receive targeted guidance precisely where it is most needed. In addition, we introduce an Attention-Guided Knowledge Alignment (AGKA), enabling the student to concentrate on the area that the teacher deems crucial instead of learning from the teacher’s features blindly. To verify the effectiveness of the proposed method, we perform extensive experiments on two publicly available datasets, i.e., DIOR and NWPU VHR-10. The experimental results show that our CSAKD outperforms existing state-of-the-art knowledge distillation approaches in the realm of RSOD.

Citation

@inproceedings{KefanZhan_2025_BMVC,
author    = {KefanZhan and An Luo and Yunpeng Zeng and Jiaxin Li and Yuan Zhang and Kai Hu},
title     = {Knowledge Distillation via Cross Supervising with Attention for Remote Sensing Object Detection},
booktitle = {36th British Machine Vision Conference 2025, {BMVC} 2025, Sheffield, UK, November 24-27, 2025},
publisher = {BMVA},
year      = {2025},
url       = {https://bmva-archive.org.uk/bmvc/2025/assets/papers/Paper_398/paper.pdf}
}


Copyright © 2025 The British Machine Vision Association and Society for Pattern Recognition
The British Machine Vision Conference is organised by The British Machine Vision Association and Society for Pattern Recognition. The Association is a Company limited by guarantee, No.2543446, and a non-profit-making body, registered in England and Wales as Charity No.1002307 (Registered Office: Dept. of Computer Science, Durham University, South Road, Durham, DH1 3LE, UK).

Imprint | Data Protection