Perceptually Grounded and Semantics-Aware Evaluation for Holistic Co-Speech Gesture Generation

TL;DR

This paper presents a perceptually grounded and semantics-aware benchmark for holistic co-speech gesture generation. The proposed benchmark combines objective metrics with human-centered evaluation and introduces a new metric, Semantic Gesture Preservation. This approach improves the alignment between objective metrics and subjective human judgments.

Why It Matters

The development of a more accurate evaluation framework for co-speech gesture generation can improve the quality of human-computer interaction and enhance the realism of virtual agents. This has practical significance in fields like robotics, animation, and human-computer interaction. More accurate evaluation can also accelerate progress in co-speech gesture generation research.

Key Findings

• Introduction of Semantic Gesture Preservation metric
• Correlation between objective metrics and subjective scores
• Improved perceptual alignment using composite metrics