Preserve to Expand: Source-Aware Adaptation and Model Merging for Multilingual LLM-Based ASR

TL;DR

This paper addresses the challenge of expanding multilingual ASR systems to new languages without retraining on source languages, focusing on connector-based Speech LLMs. It evaluates update construction and merging strategies, finding that SSU-Wanda+TIES achieves the best balance in single-target expansion, while update merging ensures stable source retention in multi-target settings. The work highlights the trade-offs between target acquisition, source preservation, and update order sensitivity.

Why It Matters

Enabling efficient, source-preserving expansion of multilingual ASR systems is critical for real-world deployment where data privacy and computational constraints limit retraining. This work provides actionable strategies for adapting large speech models without source data, advancing scalable and sustainable multilingual speech technology.

Key Findings

• SSU-Wanda+TIES provides the best source-preserving trade-off in single-target expansion.
• Update merging offers stable source retention in multi-target expansion, unlike recursive methods.
• Recursive expansion is effective only under favorable update orders, highlighting sensitivity to adaptation sequence.