Adaptive Fisher-Whitened Cross-Covariance for Low-Resource Speech Recognition

TL;DR

This paper proposes Adaptive Fisher-Whitened Cross-Covariance for low-resource speech recognition, outperforming traditional methods. It introduces two extensions: Asymmetric-Coupled FCCA and Adaptive-Rank FCCA. These approaches improve parameter efficiency without increasing model capacity.

Why It Matters

The proposed method enables more efficient adaptation of multilingual speech models to low-resource languages, which is crucial for expanding speech recognition capabilities. This has significant practical implications for languages with limited training data. It can improve speech recognition accuracy in these languages.

Key Findings

• Standard FCCA outperforms LoRA in some settings
• AR-FCCA provides consistent improvement over standard FCCA
• Adaptive rank allocation improves parameter efficiency