Near Minimax Rate for Quantile Distributional Reinforcement Learning

TL;DR

This paper studies quantile distributional policy evaluation in reinforcement learning, establishing non-asymptotic error bounds under Wasserstein distances. The authors obtain a root-n high-probability bound for estimating the quantile Bellman fixed point. They also characterize the exact local asymptotic minimax risk.

Why It Matters

The paper's findings have practical significance for reinforcement learning applications, particularly in situations where the return distribution is important. The results provide insights into the statistical accuracy of estimating the return distribution, which can inform decision-making in complex systems. This can lead to more robust and reliable reinforcement learning algorithms.

Key Findings

• non-asymptotic error bounds under Wasserstein distances
• root-n high-probability bound for estimating the quantile Bellman fixed point
• characterization of the exact local asymptotic minimax risk