Back to the feed
infra

TensorRT-LLM v1.4.0rc1

GitHub: NVIDIA/TensorRT-LLM1 min read

The NVIDIA/TensorRT-LLM v1.4.0rc1 release introduces significant updates, including added support for various models such as DeepSeek-V4.1-Flash and Qwen3.8-Flash-Next, as well as improvements to the KV cache manager and API changes. This release also includes numerous feature enhancements, such as integrated Mooncake connector, RoPE precision options, and optimized performance for specific models. The release addresses known issues and provides a more robust and efficient framework for AI-focused applications.

AI summary
Read at GitHub: NVIDIA/TensorRT-LLM