Shared-Prefix KV Reuse Across Standard LoRA Adapters: Quality and Serving Tradeoffs

TL;DR

This paper studies the reuse of shared-prefix KV cache across standard LoRA adapters, finding that full-prefix reuse has a small quality difference and the lowest prefill cost. The serving benefit is a reduction in warm-cache time-to-first-token. Partial recomputation provided no advantage.

Why It Matters

The research has practical significance as it can reduce serving costs for small-model deployments with multiple LoRA specialists. This can lead to more efficient use of resources and improved performance. The findings can inform the design of more efficient AI systems.

Key Findings

• full-prefix reuse has a small quality difference
• partial recomputation provides no advantage
• reusing KV values does not achieve shared-cache memory savings