Language Models that Play Chess and Explain Their Moves

TL;DR

Queen is a 4B-parameter language model that combines a chess engine's strategic insight with natural language explanation capabilities. It achieves Grandmaster-level play while generating coherent, human-like move justifications through iterative distillation and cross-attention with a silent expert encoder.

Why It Matters

This approach bridges the gap between high-performance domain-specific models and interpretable AI, enabling transparent decision-making in complex tasks like strategy games and robotics. It offers a scalable blueprint for integrating expert systems with language models across diverse fields.

Key Findings

• Queen's iterative distillation process improves both playing strength and explanation quality, gaining over 900 Elo points across seven training rounds.
• The model outperforms state-of-the-art systems in puzzle accuracy and generates explanations rated near GPT-5.6-Sol in coherence despite having significantly fewer parameters.
• The architecture's use of cross-attention between a chess encoder and instruction-tuned language model enables effective extraction and refinement of domain concepts.