TensorRT-LLM v1.4.0rc0
GitHub: NVIDIA/TensorRT-LLM1 min read
The NVIDIA/TensorRT-LLM v1.4.0rc0 release introduces significant updates, including new model support, API changes, and feature enhancements, such as added support for GLM-5.3-Flash and K-EXAONE-2 models, removal of the Python backend of KVCacheManagerV2, and improvements to sparse KV cache configuration. This release also addresses several known issues, including crashes and failures with specific models and GPUs. The updates and new features make this release notable for AI-focused technical audiences.
AI summary