Scaling Laws for Tool-Use Agents via Controllable Environment Synthesis

TL;DR

This paper introduces SynthAPI, a controllable environment synthesis pipeline that generates diverse tool-use tasks by varying tools, tasks, and databases. It establishes a scaling law showing that increasing tool and database diversity yields greater gains than more tasks, especially at scale, leading to superior agent performance across benchmarks.

Why It Matters

The findings provide actionable guidelines for efficiently scaling tool-use agent training, enabling better performance with limited data budgets. This advances the development of autonomous AI agents capable of complex, real-world tasks.

Key Findings

• Scaling tool and database diversity is more effective than increasing task count for improving agent generalization.
• Increasing data complexity yields larger performance gains than simply increasing data volume.
• SynthAPI-generated training data enables agents to outperform prior models across multiple evaluation benchmarks.