OmniVBench: A Benchmark and Large-Scale Dataset for Omni Reference-to-Video Generation

TL;DR

The paper introduces OmniVBench, a benchmark for omni reference-to-video generation, and the Omni-R2V Dataset for training omni R2V models. OmniVBench evaluates R2V models across broader reference types and tasks. It highlights performance gaps in current R2V models.

Why It Matters

OmniVBench and the Omni-R2V Dataset address the need for more comprehensive evaluation and training resources in omni reference-to-video generation. This contributes to the development of more versatile and general R2V models. Improved R2V models can enhance various applications, such as video editing and content creation.

Key Findings

• OmniVBench covers 7 task families and 18 fine-grained tasks
• The Omni-R2V Dataset comprises 340K processed training samples
• Extensive evaluation reveals performance gaps in current R2V models