Progressive Memory Transformer: Memory-Aware Attention for Time-Series

TL;DR

The Progressive Memory Transformer (PMT) is a learning framework that exploits the structural hierarchy of time-series data at multiple scales. PMT augments a transformer with writable, window-aligned memory to expose mid-range scale representations. This enables strong performance in low-label classification and competitive forecasting.

Why It Matters

The PMT framework has practical significance in time-series analysis, enabling better understanding and modeling of complex data. This can lead to improved forecasting and classification performance in various applications. Effective time-series analysis can inform decision-making in fields like finance, healthcare, and climate science.

Key Findings

• PMT learns representations that probe well at global, mid-range, and local scales
• strong low-label classification performance
• competitive forecasting performance across multiple horizons