Compression Through a Context Lens: Evaluating Adaptation, Quantization, and Distillation in Arabic Legal LLMs

TL;DR

The study evaluates model compression techniques on Arabic legal language models using a new benchmark, ArabLegalEval, under both context-aware and context-free settings. It finds that removing context significantly reduces accuracy, and that quantization and distillation have mixed effects, with distillation showing greater sensitivity to student model size.

Why It Matters

Legal AI systems in resource-limited environments need efficient models without sacrificing accuracy. This work highlights the importance of evaluation context when assessing compression methods, guiding better deployment strategies for Arabic legal NLP.

Key Findings

• No-context evaluation reveals accuracy drops of up to 27 percentage points compared to context-aware testing, indicating context dependency in model performance.
• Four-bit quantization reduces model memory by 64-71% with minimal impact on no-context accuracy, varying between -3.2 and +1.6 percentage points.
• Knowledge distillation results are inconsistent, with accuracy generally decreasing as student model size shrinks, though a Qwen2.5-7B student matched its teacher's performance closely.