Time-Indexed Skill Success Models for Latency-Aware Skill Selection and Timing in Dynamic Manipulation

TL;DR

The paper introduces time-indexed skill success models that predict task success based on when a skill is launched after observation, accounting for latency. These models enable a receding-horizon planner to jointly select the best skill and optimal launch time under variable delays. Experiments in simulation and on a real UR5e robot show significant performance gains over static models, especially under delayed execution.

Why It Matters

Robots performing dynamic manipulation face challenges due to sensor delays and rapidly changing environments. This approach improves decision-making under latency, enabling more reliable and accurate skill execution in real-world robotic systems, which is critical for applications like automated manufacturing and autonomous navigation.

Key Findings

• Time-indexed models outperform static competence models by 2.8 to 6.2 percentage points under 100–300 ms latency in simulation.
• The time-indexed model is well-calibrated (ECE 0.009), while static models overestimate success by up to 23 percentage points when latency is ignored.
• On a real UR5e with 0.2 s latency, the static model predicted 99% success for high-rated launches, but only 63% succeeded, while the time-indexed model predicted 56%—a more accurate estimate.