LoRA-generating hypernetworks for efficient on-device LLM generative personalization

TL;DR

The paper introduces a method that uses a hypernetwork to generate LoRA adapters on mobile devices based on user context, enabling efficient personalization of on-device LLMs. This approach combines the low computational cost of in-context learning with the persistent, weight-based adaptation of parameter-efficient fine-tuning. It requires minimal storage and operates entirely on-device, making it ideal for resource-constrained environments.

Why It Matters

Personalizing large language models on mobile devices is challenging due to limited compute and storage. This method enables high-quality, user-specific model adaptation without increasing latency or requiring external servers, enhancing privacy and responsiveness. It opens new possibilities for adaptive, real-time AI assistants on consumer devices.

Key Findings

• The hypernetwork generates task-specific LoRA adapters on-device using only forward passes, avoiding expensive fine-tuning or input sequence extension.
• The approach achieves strong performance on long-form text generation tasks, outperforming both in-context learning and standard PEFT methods in personalization accuracy.
• The architecture reuses base LLM weights, minimizing additional storage and making it suitable for deployment on mobile devices with tight resource constraints.