Online Neural Space Time Memory for Dynamic Novel View Synthesis

TL;DR

This paper proposes an online neural space-time memory approach for dynamic novel view synthesis, enabling real-time performance and state-of-the-art results. The method decouples memory updates from application, using cross-view attention and auxiliary mechanisms. This allows for efficient and stable long-horizon memory reconstruction.

Why It Matters

The proposed approach has significant implications for real-time video processing and novel view synthesis applications, such as virtual reality, robotics, and video production. It enables the efficient reconstruction of temporarily occluded regions in dynamic scenes, enhancing the overall visual experience. This technology can be applied to various fields, including entertainment, education, and healthcare.

Key Findings

• decoupling memory updates from application
• cross-view attention for deformation management
• auxiliary memory loss and caching strategy