FL4N: Feed-forward Latent Model for 4D Novel View Synthesis

TL;DR

FL4N introduces the first feed-forward latent 4D novel view synthesis model that encodes a unified spatiotemporal scene prior in a latent space, avoiding explicit 4D Gaussian representations. By modeling temporal dynamics as residuals on a pretrained static latent prior, it achieves high-fidelity 4D rendering with real-time inference and strong generalization to real-world videos.

Why It Matters

This approach enables efficient, artifact-free 4D scene reconstruction from monocular videos, crucial for applications like virtual reality, autonomous driving, and digital twins where real-time, temporally consistent rendering is essential.

Key Findings

• Learning temporal dynamics as residuals over a static latent prior improves temporal consistency and reduces artifacts.
• The lightweight dynamic encoder effectively disentangles object motion from camera parallax using static scene features.
• FL4N achieves state-of-the-art 4D NVS quality with real-time rendering and generalizes well to real-world video data.