Fourier-BasedGlobal-LocalJointPerceptionSpatiotemporalPredictionModel

Abstract: The spatio-temporal sequence prediction learning aims to generate future image data by learning from the context of historical data. The challenge of spatio-temporal prediction lies in capturing complex spatial correlations and temporal evolutions in the physical world. Current research is often geared towards specific tasks and tends to focus on the overall trend of change while neglecting local details. Additionally, there are inefficiencies related to non-parallel recursive unit models. To address these issues, this paper presents a versatile framework for spatio-temporal data prediction. The framework integrates local and global spatial features, utilizing Fourier-based transformations to capture global dependencies, which are then fused with local relationships extracted by the Swin-Transformer to achieve joint global-local spatial awareness. Temporal features of various scales are extracted through a multi-scale fully convolutional module, and a Fourier transformation is applied again to convert the time domain into the frequency domain, capturing features within the continuous evolutionary time stack comprehensively. This not only preserves the long-term dependencies of the data but also enhances the computational efficiency of the model. Experimental results demonstrate the model's superior generality, effectiveness, and scalability on the SEVIR, KTH, and MovingMNIST datasets