Robot Learning with Visual Predicted Force

TL;DR

The paper presents a method for force-aware robotic manipulation using visual prediction of forces from a compliant gripper's deformation, eliminating the need for physical force sensors. It trains a visual force estimator and a policy that jointly proposes actions and their expected forces, enabling effective contact-rich tasks through inference-time action selection based on predicted force similarity to demonstrated targets.

Why It Matters

This approach enables robust manipulation in complex environments without relying on expensive or fragile force/tactile sensors, making it more accessible and practical for real-world robotic systems. It demonstrates that visual cues alone can support precise force control, expanding the applicability of vision-based learning in robotics.

Key Findings

• Visual force prediction from gripper deformation enables accurate force estimation without physical sensors.
• A policy trained on force-augmented demonstrations can jointly propose actions and their expected forces.
• Inference-time action selection based on predicted force similarity achieves effective performance across diverse contact-rich tasks.