Joints, Camera, Action! Co-Generating 3D Motion and Video for Arbitrary Rigs

Noam Rotstein, Yoad Tewel, Chen Tessler, Haggai Maron, Gal Chechik, Or Litany, Yoni Kasten, Yuval Atzmon

arXiv: 2610.28632Published: October 20, 202622 upvotes

View on arXiv Read PDF

AI summary

TL;DR

This paper presents a model that co-generates 3D motion and video for arbitrary rigs, allowing for more realistic and flexible animation. The model can animate diverse rigs from text and respond to the surrounding scene. It also enables keyframe-based control of the 3D motion.

Why It Matters

This research has significant implications for the fields of computer vision, animation, and robotics, as it enables more realistic and interactive 3D animations. The model's ability to co-generate 3D motion and video can be used in various applications, such as video games, film production, and virtual reality. This technology can also be used to improve robotics and human-computer interaction.

Key Findings

co-generation of 3D motion and video
support for arbitrary skeletons
keyframe-based control of 3D motion

Topics

multimodal

3d reconstruction

computer vision

animation

generative models

Thank you to arXiv for use of its open access interoperability. arxiv.org. aibrief.ai is not affiliated with or endorsed by arXiv.