From Gameplay to Policy: Towards Scalable Robot Data Collection via Gamified Robot-Free Interaction

TL;DR

The paper introduces Project Kitchen, a VR-based platform that turns robot manipulation data collection into a gamified experience, enabling scalable, hardware-agnostic data generation. It proposes Game2Policy, a method that extracts embodiment-invariant affordance cues from gameplay to train real robot policies with minimal real-world demonstrations.

Why It Matters

This approach enables large-scale, diverse manipulation data collection without requiring specific robot hardware, making it feasible for crowdsourced data gathering and improving policy transfer efficiency. It reduces reliance on costly real-robot interactions while maintaining high performance in few-shot settings.

Key Findings

• Project Kitchen successfully collects diverse, goal-directed manipulation behaviors through an engaging VR game interface.
• User studies confirm the platform promotes varied manipulation strategies and maintains high user engagement during data collection.