FlowHMR: Physically Plausible Motion Capture from Video

TL;DR

FlowHMR is a framework for recovering physically plausible 3D human motion from monocular video. It uses a flow matching model and Group Relative Policy Optimization (GRPO) with fidelity and tracking rewards. This approach outperforms state-of-the-art methods in motion fidelity and physical tracking success rate.

Why It Matters

Accurate human motion capture has numerous applications in fields like computer vision, robotics, and healthcare. FlowHMR's ability to produce physically plausible motion can improve the realism of virtual characters and enhance human-computer interaction. This technology can also be used in motion analysis and rehabilitation.

Key Findings

• FlowHMR achieves 82.47% physical tracking success rate
• Outperforms state-of-the-art methods in overall motion fidelity
• Introduces Wild-4K, a large dataset for evaluating human motion recovery