DSDyn-VLA: A Dual-Stream Dynamic Manipulation Framework with Motion Perception, Future Awareness, and Realtime Correction

TL;DR

DSDyn-VLA introduces a dual-stream architecture to improve vision-language-action models in dynamic manipulation tasks. It combines a slow flow-planner with motion perception and future state prediction for high-level planning, and a fast residual refiner using reinforcement learning for real-time closed-loop corrections. The system significantly outperforms state-of-the-art methods in both simulated and real-world dynamic settings.

Why It Matters

Dynamic manipulation is critical for real-world robotics applications like automated assembly and logistics, where objects move continuously. Current models fail under motion and latency, limiting deployment. DSDyn-VLA addresses core limitations in perception, timing, and control, enabling more robust and reliable robotic behavior in real-time environments.

Key Findings

• DSDyn-VLA reduces failure rates by over 76% compared to the best existing method on the Kinetix dynamic benchmark under high-latency conditions.
• The framework achieves approximately 6 times higher success rate than PI0.5 in real-world dynamic manipulation tasks and about 5 times higher on the newly introduced DynBench benchmark.
• The dual-stream design—combining motion-aware macro-planning with high-frequency closed-loop correction—effectively bridges the perception, latency, and control gaps in dynamic manipulation.