Self-Evolving Visual Agent via Margin Reward

TL;DR

Marvo introduces a self-evolving framework for visual agents that learns tool use from unlabeled images without human annotations. It uses a challenger-solver architecture with a margin reward based on performance difference with and without tools to generate progressively challenging questions. The method achieves up to 7.1% higher accuracy across multiple reasoning benchmarks.

Why It Matters

This approach reduces reliance on expensive labeled data or expert demonstrations, enabling scalable training of visual agents for real-world tasks. It advances autonomous learning in multimodal systems by dynamically curating training content through agent interaction.

Key Findings

• The margin reward mechanism effectively guides the generation of questions that are challenging enough to require tool use but still solvable.
• Marvo's self-evolution process leads to improved tool invocation accuracy across diverse reasoning tasks without human supervision.
• The adaptive curriculum generated by the challenger enables the solver to progressively learn more complex tool-based reasoning strategies.