Co-Evolving Harnesses and Models: On-Policy Correction Helps Weaker Models Catch Up Where Imitation Fails

TL;DR

A method is introduced to co-evolve agent harnesses and lightweight models, where a meta-level MLE agent enables on-policy correction by having experts fix only failing steps in weaker models' rollouts. This preserves the model's native planning style while integrating expert knowledge, avoiding performance degradation seen in full-trajectory imitation. The approach enables cost-effective, high-performing agents on enterprise tasks without relying on large models.

Why It Matters

It enables smaller, cheaper models to achieve strong performance on specialized tasks by combining evolved harnesses with targeted expert corrections, reducing dependency on expensive frontier models. This approach is scalable and practical for enterprise applications where cost and efficiency are critical.

Key Findings

• Evolved harnesses significantly improve weaker models' performance on domain-specific tasks, but full-trajectory imitation from stronger models causes performance regression.
• Imitation disrupts model-harness alignment by transferring expert planning strategies without the execution capability, leading to misfit in the evolved system.
• On-policy correction—where only failing steps are rewritten by an expert—preserves the model's planning style and enables effective co-evolution without breaking compatibility.