TACTFL: Temporal Contrastive Training for Multi-modal Federated Learning with Similarity-guided Model Aggregation


Guanxiong Sun (University of Bristol), Majid Mirmehdi (University of Bristol), Zahraa S. Abdallah (University of Bristol), Raúl Santos-Rodriguez (University of Bristol), Ian James Craddock (University of Bristol), Telmo M Silva Filho (University of Bristol)
The 35th British Machine Vision Conference
PDFPosterSupplementary

Abstract

Real-world federated learning faces two key challenges: limited access to labelled data and the presence of heterogeneous multi-modal inputs. This paper proposes TACTFL, a unified framework for semi-supervised multi-modal federated learning. TACTFL introduces a modality-agnostic temporal contrastive training scheme that conducts representation learning from unlabelled client data by leveraging temporal alignment across modalities. However, as clients perform self-supervised training on heterogeneous data, local models may diverge semantically. To mitigate this, TACTFL incorporates a similarity-guided model aggregation strategy that dynamically weights client models based on their representational consistency, promoting global alignment. Extensive experiments across diverse benchmarks and modalities, including video, audio, and wearable sensors, demonstrate that TACTFL achieves state-of-the-art performance. For instance, on the UCF101 dataset with only 10\% labelled data, TACTFL attains 68.48\% top-1 accuracy, significantly outperforming the FedOpt baseline of 35.35\%. Code will be released upon publication.

Citation

@inproceedings{Sun_2025_BMVC,
author    = {Guanxiong Sun and Majid Mirmehdi and Zahraa S. Abdallah and Raúl Santos-Rodriguez and Ian James Craddock and Telmo M Silva Filho},
title     = {TACTFL: Temporal Contrastive Training for Multi-modal Federated Learning with Similarity-guided Model Aggregation},
booktitle = {36th British Machine Vision Conference 2025, {BMVC} 2025, Sheffield, UK, November 24-27, 2025},
publisher = {BMVA},
year      = {2025},
url       = {https://bmva-archive.org.uk/bmvc/2025/assets/papers/Paper_69/paper.pdf}
}


Copyright © 2025 The British Machine Vision Association and Society for Pattern Recognition
The British Machine Vision Conference is organised by The British Machine Vision Association and Society for Pattern Recognition. The Association is a Company limited by guarantee, No.2543446, and a non-profit-making body, registered in England and Wales as Charity No.1002307 (Registered Office: Dept. of Computer Science, Durham University, South Road, Durham, DH1 3LE, UK).

Imprint | Data Protection