CompactionRL: Reinforcement Learning with Context Compaction for Long-Horizon Agents

TL;DR

CompactionRL introduces a reinforcement learning framework that enables long-horizon language model agents to manage extended interaction sequences by compressing context during training. It jointly optimizes action selection and summary generation using token-level loss normalization and generalized advantage estimation, allowing agents to learn effectively from compacted trajectories. The method improves performance on coding benchmarks when applied to open-source models.

Why It Matters

Long-horizon tasks in agentic AI are constrained by fixed context windows, limiting model effectiveness. CompactionRL addresses this bottleneck by enabling efficient, scalable training of large models without context overflow, improving real-world task completion. This advancement supports the development of more capable open-source agents for complex, multi-step reasoning.

Key Findings

• CompactionRL enables reinforcement learning agents to train on compacted long-horizon trajectories by jointly optimizing actions and summary generation.
• Applied to GLM-4.5-Air and GLM-4.7-Flash, CompactionRL achieves significant gains on SWE-bench Verified and Terminal-Bench 2.0, leading to deployment in the GLM-5.2 training pipeline.