Do LLMs Act on What They Know? From Partner Representations to Cooperative Actions

TL;DR

The study examines how large language models interpret and act on communication conventions when cooperating with unfamiliar partners in a Hanabi-inspired setting. Despite models decoding sender intent more accurately than target conventions, their receiving decisions do not consistently align with the intended rules, and performance improves more with action-level recommendations than with general rule statements.

Why It Matters

Understanding how LLMs translate observed communication patterns into cooperative actions is crucial for deploying them in real-world multi-agent systems where partners may use unknown conventions. The findings reveal critical gaps in decision-making fidelity, guiding future work on improving alignment between perception and action in collaborative AI.

Key Findings

• Linear probes detect sender intent conventions more accurately than target conventions across eight LLMs.
• Receiving decisions fail to consistently reflect the sender's intended convention, even when intent is correctly decoded.
• Action recommendations lead to larger cooperation gains than rule statements, and Qwen3-8B shows greater sensitivity to action-level guidance than to rule-based descriptions.