vllm v0.32.0
GitHub: vllm-project/vllm1 min read
This release introduces significant improvements to the VLLM project, including structured decision-making and generic parsers, low-precision KV cache enhancements, and a new Model Runner V2. These changes aim to increase throughput, reduce latency, and expand model support.
AI summary