Video Dataset Condensation with Diffusion Models


Zhe Li (FAU Erlangen-Nürnberg), Hadrien Reynaud (Imperial College London), Mischa Dombrowski (FAU Erlangen-Nürnberg), Sarah Cechnicka (Imperial College London), Franciskus Xaverius Erick (FAU Erlangen-Nürnberg), Bernhard Kainz (FAU Erlangen-Nürnberg, Imperial College London)
The 35th British Machine Vision Conference
PDFPosterVideo (Right click to download)

Abstract

In recent years, the rapid expansion of dataset sizes and the increasing complexity of deep learning models have significantly escalated the demand for computational resources, both for data storage and model training. Dataset distillation has emerged as a promising solution to address this challenge by generating a compact synthetic dataset that retains the essential information from a large real dataset. However, existing methods often suffer from limited performance, particularly in the video domain. In this paper, we focus on video dataset distillation. We begin by employing a video diffusion model to generate synthetic videos. Since the videos are generated only once, this significantly reduces computational costs. Next, we introduce the Video Spatio-Temporal U-Net (VST-UNet), a model designed to select a diverse and informative subset of videos that effectively captures the characteristics of the original dataset. To further optimize computational efficiency, we explore a training-free clustering algorithm, Temporal-Aware Cluster-based Distillation (TAC-DT), to select representative videos without requiring additional training overhead. We validate the effectiveness of our approach through extensive experiments on four benchmark datasets, demonstrating performance improvements of up to \(10.61\%\) over the state-of-the-art. Our method consistently outperforms existing approaches across all datasets, establishing a new benchmark for video dataset distillation.

Citation

@inproceedings{Li_2025_BMVC,
author    = {Zhe Li and Hadrien Reynaud and Mischa Dombrowski and Sarah Cechnicka and Franciskus Xaverius Erick and Bernhard Kainz},
title     = {Video Dataset Condensation with Diffusion Models},
booktitle = {36th British Machine Vision Conference 2025, {BMVC} 2025, Sheffield, UK, November 24-27, 2025},
publisher = {BMVA},
year      = {2025},
url       = {https://bmva-archive.org.uk/bmvc/2025/assets/papers/Paper_594/paper.pdf}
}


Copyright © 2025 The British Machine Vision Association and Society for Pattern Recognition
The British Machine Vision Conference is organised by The British Machine Vision Association and Society for Pattern Recognition. The Association is a Company limited by guarantee, No.2543446, and a non-profit-making body, registered in England and Wales as Charity No.1002307 (Registered Office: Dept. of Computer Science, Durham University, South Road, Durham, DH1 3LE, UK).

Imprint | Data Protection