TrackEverything: Long Horizon Dense Tracking via De-Duplicating 3D Scene Representations

TL;DR

TrackEverything enables dense 3D point tracking across long video sequences by representing scenes in persistent world coordinates and using voxel-based de-duplication to avoid redundant tracking. It combines a two-stage refinement process with efficient 3D feature sampling to scale beyond memory limits, achieving state-of-the-art performance on both short and long sequences.

Why It Matters

This approach enables comprehensive, long-term tracking of all visible scene points without sacrificing memory efficiency, which is crucial for robotics, AR/VR, and autonomous systems requiring persistent scene understanding over time.

Key Findings

• TrackEverything achieves dense 3D tracking across videos exceeding 1000 frames using only 40 GB of GPU memory, breaking prior scalability limits.
• The voxel-based de-duplication mechanism prevents repeated tracking of the same physical surface across frames, reducing redundancy.
• 3D WAFT enables efficient feature matching without 4D correlation volumes, allowing fast and memory-efficient trajectory decoding.