RefVideo-6M: A Reliable Reference-Based Dataset for Instructional Video Editing

TL;DR

RefVideo-6M is a large-scale dataset with 5 million video and 1 million image editing samples, using real, artifact-free videos as targets and incorporating over 6 million visual references. It improves supervision reliability through a multi-expert filtering pipeline and supports reference-guided editing with diverse visual cues. A model trained on this dataset, Ref-MoT, achieves better visual quality, controllability, and reference consistency.

Why It Matters

The dataset addresses key shortcomings in current video editing benchmarks by replacing synthetic targets with real videos and adding visual references, enabling more accurate and controllable editing models. This advancement supports the development of practical tools for content creation, video personalization, and identity-preserving edits.

Key Findings

• Using real, artifact-free videos as editing targets leads to more reliable supervision compared to synthetic or model-generated videos.
• Incorporating diverse visual references significantly improves a model's ability to maintain identity and control over edits.
• Ref-MoT, trained on RefVideo-6M, demonstrates superior performance in visual quality, controllability, and reference consistency over baseline models.