Events Meet Dynamic Mode Decomposition: Capturing the Spatiotemporal Dynamics of Moving Objects


Zhouning Du (AISIN CORPORATION), Israr Ulhaq (AISIN CORPORATION), Thanh Thi Huyen Phan (AISIN CORPORATION), Yuichiro Yoshimura (AISIN CORPORATION), Jigyasa Chand (AISIN CORPORATION), Truong Vinh Truong Duy (AISIN CORPORATION)
The 35th British Machine Vision Conference
PDFPosterVideo (Right click to download)Supplementary

Abstract

This paper introduces a novel mechanism for Moving Object Segmentation (MOS) that utilizes Dynamic Mode Decomposition (DMD) applied to event streams from event cameras to reconstruct missing dynamic information between RGB frames. First, we develop an Event-Driven Dynamic Mode Decomposition (ED-DMD) framework capable of capturing intrinsic motion dynamics between consecutive frames. By transforming high-resolution event streams into structured event slices, we convert event data into a compact Event-Driven Descriptor (ED-Dr), facilitating precise recovery of high-speed object movements. We then introduce a Motion-Aware Segmentation Network (MAS-Net) to seamlessly integrate these dynamics with spatial information, comprising a Dynamic Predict Block (DPB) and a Spatial Latent Block (SLB). During training, these blocks collaboratively process the RGB frames and ED-Dr to learn static appearance features and motion dynamics. During inference, the model operates in an event-free mode, requiring only RGB inputs to achieve event-aware performance by leveraging the learned motion priors. We also conduct a comprehensive analysis of the impact of different ED-Dr features, block configurations, and backbone architectures on segmentation accuracy using the DSEC-MOS dataset. Comparative evaluations with state-of-the-art methods demonstrate that our approach consistently surpasses existing baselines in these evaluation metrics, validating the robustness and effectiveness of our framework in capturing spatiotemporal dynamics in complex real-world scenarios.

Citation

@inproceedings{Du_2025_BMVC,
author    = {Zhouning Du and Israr Ulhaq and Thanh Thi Huyen Phan and Yuichiro Yoshimura and Jigyasa Chand and Truong Vinh Truong Duy},
title     = {Events Meet Dynamic Mode Decomposition: Capturing the Spatiotemporal Dynamics of Moving Objects},
booktitle = {36th British Machine Vision Conference 2025, {BMVC} 2025, Sheffield, UK, November 24-27, 2025},
publisher = {BMVA},
year      = {2025},
url       = {https://bmva-archive.org.uk/bmvc/2025/assets/papers/Paper_339/paper.pdf}
}


Copyright © 2025 The British Machine Vision Association and Society for Pattern Recognition
The British Machine Vision Conference is organised by The British Machine Vision Association and Society for Pattern Recognition. The Association is a Company limited by guarantee, No.2543446, and a non-profit-making body, registered in England and Wales as Charity No.1002307 (Registered Office: Dept. of Computer Science, Durham University, South Road, Durham, DH1 3LE, UK).

Imprint | Data Protection