A Balanced Data Diet: Addressing the Exploration Bottleneck in Mega-Scale RL for Robot Control

TL;DR

The paper introduces Success Guided Sampling (SGS), an adaptive method that improves large-scale reinforcement learning by focusing training on task configurations near a policy's current performance frontier. This reduces wasted experience in parallel simulations and enables effective scaling to over one million environments. SGS allows solving complex locomotion and assembly tasks that prior methods could not handle.

Why It Matters

By making large-scale simulation more efficient, SGS reduces the need for handcrafted rewards and demonstrations, accelerating the development of general-purpose robots. It enables zero-shot transfer of learned policies to real-world hardware, advancing practical deployment of AI-driven robotic control.

Key Findings

• Success Guided Sampling dynamically prioritizes task configurations near the policy's performance frontier, improving sample efficiency in large-scale RL.
• Policies trained with SGS generalize to real-world assembly tasks without further training, demonstrating zero-shot transfer using only RGB observations.