LoRAForge: Co-Designing Post-Hoc Compression and Execution for Efficient Multi-LoRA Serving

TL;DR

LoRAForge co-designs post-hoc compression and execution for efficient multi-LoRA serving by introducing an explicit retained-rank representation and FlashTSQR factor construction, enabling up to 2.28× faster factorization and 1.22× higher throughput. It supports rank-specialized execution via uniform or spectral-energy allocation policies, achieving 7.7% parameter reduction with minimal quality loss and substantial token-throughput gains on both AMD and NVIDIA GPUs.

Why It Matters

Efficient multi-LoRA serving is critical for deploying numerous task-specific adapters in real-world applications. LoRAForge reduces storage, loading, and execution overheads without sacrificing performance, enabling scalable and cost-effective deployment of large adapter fleets across diverse hardware.

Key Findings

• FlashTSQR accelerates factor construction by up to 2.28× over library QR by exploiting parallelism across modules and row tiles.
• LoRAForge-S with spectral-energy budget allocation retains only 7.7% of adapter parameters while incurring ≤1.2-point score drop on a 0–100 scale.
• The shared runtime achieves 1.17–1.22× higher throughput across compressed policies, with significant token-throughput gains on both AMD and NVIDIA GPUs.