llama.cpp v0.7.0
GitHub: ggerganov/llama.cpp1 min read
The v0.7.0 release of llama.cpp introduces significant performance enhancements, including a new GPU cache for MoE experts and LiliCorr speculative decoding, as well as support for several new models such as K2 Horizon and Prism Bonsai 2 27B. This release also includes a refactored chat session API and a new default port. The update brings substantial improvements to the overall functionality and efficiency of the software.
AI summary