Bellman Language Models: Solving the Recurring Decisions of LLM Agents

TL;DR

This paper introduces Bellman Language Models, which solve recurring decisions in LLM agents as an optimization problem, making them faster, cheaper, and more reliable. The approach, called Discretize, Then Decide, discretizes conversations into states and estimates probabilities and costs. This leads to significant improvements in task completion rates and reduced model calls.

Why It Matters

The proposed method has practical significance as it can improve the efficiency and reliability of LLM agents in various applications, such as customer service and task automation. By reducing the number of model calls, it can also lower costs and improve response times. This can lead to better user experiences and increased adoption of LLM agents in real-world scenarios.

Key Findings

• 79% of tasks solved by the controller with no model on τ^2-bench telecom
• Qwen3.8-27B achieves 96% task completion rate with 93% fewer model calls as a BLM
• BLM approach beats every published method on InventoryBench