Multimodal Feature Collaboration and Fusion for Fine-Grained Action Recognition


Xinyu Bian (Beijing University of Posts and Telecommunications), Dongliang Chang (Beijing University of Posts and Telecommunications), Yuqi Yang (Beijing University of Posts and Telecommunications), Lei Chen (Tsinghua University), Zhanyu Ma (Beijing University of Post and Telecommunication)
The 35th British Machine Vision Conference
PDFPosterVideo (Right click to download)

Abstract

Fine-grained action recognition faces challenges of subtle inter-class differences and significant intra-class variations in complex real-world scenarios. Existing methods are often limited by insufficient feature representation capabilities of single modalities or redundant computational costs. In this paper, we propose a Multimodal Feature Collaboration and Fusion framework to achieve efficient recognition through cross-modal guided spatial and temporal modeling. For the RGB modality, skeleton joint coordinates collaboratively localize and crop action regions to suppress background noise, and design a lightweight Spatio-Temporal Attention and Aggregation (STAA) that combines short-term motion excitation and long-term temporal modeling to enhance feature extraction efficiency and accuracy. For the skeleton modality, we propose a Class-aware contrastive learning that dynamically selects high-similarity samples to optimize feature discriminative boundaries. Finally, a score‑weighted fusion strategy integrates the optimized representations from both modalities. Experiments on the FineGym and NTU RGB+D benchmarks demonstrate impressive performance improvements over state‑of‑the‑art methods.

Citation

@inproceedings{Bian_2025_BMVC,
author    = {Xinyu Bian and Dongliang Chang and Yuqi Yang and Lei Chen and Zhanyu Ma},
title     = {Multimodal Feature Collaboration and Fusion for Fine-Grained Action Recognition},
booktitle = {36th British Machine Vision Conference 2025, {BMVC} 2025, Sheffield, UK, November 24-27, 2025},
publisher = {BMVA},
year      = {2025},
url       = {https://bmva-archive.org.uk/bmvc/2025/assets/papers/Paper_284/paper.pdf}
}


Copyright © 2025 The British Machine Vision Association and Society for Pattern Recognition
The British Machine Vision Conference is organised by The British Machine Vision Association and Society for Pattern Recognition. The Association is a Company limited by guarantee, No.2543446, and a non-profit-making body, registered in England and Wales as Charity No.1002307 (Registered Office: Dept. of Computer Science, Durham University, South Road, Durham, DH1 3LE, UK).

Imprint | Data Protection