UniCache: Task- and Type-Aware KV Cache Compression for Unified Multimodal Models

TL;DR

UniCache introduces a training-free method for compressing key-value (KV) caches in unified multimodal models by adapting compression policies to specific tasks and cache types. It uses offline calibration to assign appropriate compression strategies and manages resource allocation via attention-guided scheduling and temporal coordination under a shared storage limit.

Why It Matters

Long-context multimodal applications are hindered by high KV cache memory demands. UniCache enables efficient scaling of unified models without quality loss, boosting throughput and making real-world deployment of complex multimodal systems more feasible.

Key Findings

• Existing KV cache compression methods fail in unified multimodal models due to their task- and type-agnostic nature.
• UniCache dynamically assigns task- and type-specific compression policies through offline calibration, preserving critical information across diverse tasks.
• The framework achieves up to 5× compression for understanding and editing, 2.5× for generation, with minimal quality degradation and up to 1.78× higher throughput in long-context scenarios.