Flash-dLLM: IO-Aware KV Caching and Parallel Decoding for Fast, Memory-Efficient Diffusion LLMs

TL;DR

Flash-dLLM is a training-free inference acceleration framework for diffusion Large Language Models (dLLMs) that improves inference speed and memory efficiency. It achieves this through an I/O-aware fused KV-cache kernel and a unified draft-and-verify decoding strategy. Flash-dLLM outperforms existing state-of-the-art dLLM acceleration methods.

Why It Matters

The development of Flash-dLLM has practical significance as it enables faster and more efficient deployment of dLLMs in real-world applications. This can lead to improved performance in tasks such as text generation and code generation. The scalability of Flash-dLLM also makes it suitable for longer sequences and larger batch sizes.

Key Findings

• I/O-aware KV caching reduces redundant memory movement
• Unified draft-and-verify decoding strategy improves decoding speed
• Flash-dLLM achieves significant speedups over prior baselines