VLCQ: Post-training quantization for deep neural networks using variable length coding

TL;DR

VLCQ introduces a post-training quantization method that uses variable-length coding instead of fixed-point integers, enabling higher compression with minimal accuracy loss. It achieves superior or comparable performance to state-of-the-art methods across image classification, segmentation, and detection tasks without requiring training data or fine-tuning.

Why It Matters

This approach enables more efficient deployment of deep learning models on edge devices by reducing model size and memory bandwidth needs while maintaining high accuracy. The flexibility of variable-length encoding opens new pathways for optimizing quantization without retraining.

Key Findings

• VLCQ achieves up to 6x compression over FP32 with per-channel quantization and surpasses FP32 accuracy on most models.
• With less than 4% accuracy drop, VLCQ enables up to 12.25x compression, outperforming existing post-training methods.
• For per-tensor quantization, VLCQ matches the performance of data-free methods while offering greater adaptability across models and tasks.