D&A: Decoupling Description and Assessment Enables VLMs as Zero-Shot Robotic Reward Models

TL;DR

D&A decouples vision-language models' progress estimation into description and assessment phases, using explicit rubrics to prevent confirmation bias and enable zero-shot, fine-grained task-progress evaluation. By grounding success in external criteria rather than internal video dynamics, it achieves accurate, non-linear progress scoring without post-training.

Why It Matters

This approach enables reliable, generalizable reward modeling for robotics without costly real-world data or model retraining, significantly lowering barriers to deploying zero-shot vision-language models in reinforcement learning.

Key Findings

• D&A mitigates confirmation bias by decoupling description and assessment, preventing VLMs from hallucinating success in failed executions.
• The method produces fine-grained, partial progress metrics that accurately penalize failures and interpolate successful runs.
• As a purely prompt-based framework, D&A generalizes across tasks, robots, and benchmarks, matching or exceeding post-trained reward models.