RobustRLlib: A Unified Library and Benchmark for Robust Reinforcement Learning Algorithms

TL;DR

RobustRLlib is a unified library and benchmark that integrates 17 robust reinforcement learning algorithms under a common interface, enabling improved reusability and systematic evaluation across 273 perturbation configurations. It provides a comprehensive capability map of current robust RL methods, identifying strengths and persistent robustness gaps in real-world scenarios involving shifts and disruptions.

Why It Matters

By standardizing evaluation and implementation across diverse robust RL algorithms, RobustRLlib lowers the barrier to adopting and advancing robust policies in safety-critical applications like robotics and autonomous systems. Its open-source, extensible design accelerates research and deployment of reliable AI agents in dynamic, unpredictable environments.

Key Findings

• Robust RL algorithms exhibit inconsistent performance across different types of environmental shifts, revealing significant robustness gaps.
• Offline and safe RL methods show improved resilience under certain perturbations but struggle with distributional shifts in vision-language-action tasks.
• The unified benchmark enables reproducible comparisons, highlighting the need for more generalizable robustness mechanisms.