GRACE: Generation-aware latent compression for efficient video generation

TL;DR

The paper proposes GRACE, a two-stage framework for compressing video autoencoders while maintaining compatibility with pretrained diffusion models. GRACE achieves 8x token count reduction and 11.1x latency reduction without sacrificing generation quality. This enables efficient video generation with diffusion models.

Why It Matters

Efficient video generation is crucial for various applications, including video editing, synthesis, and analysis. GRACE's compression technique can significantly accelerate video diffusion models, making them more practical for real-world use cases. This can lead to improved performance and reduced computational costs in video processing tasks.

Key Findings

• GRACE reduces token count by 8x
• GRACE reduces latency by 11.1x
• GRACE maintains generation quality