StableVQ: Practical Guidelines for Stable Vector-Quantized Tokenizer Training

TL;DR

StableVQ improves training stability and codebook utilization for vector-quantized tokenizers. It achieves this through dynamic STE, region VQ loss, and decoupled schedule. This leads to better reconstruction quality and robust system behavior.

Why It Matters

StableVQ has practical significance as it enables more efficient and effective training of autoregressive and masked image generation models. This can lead to improved performance in image generation tasks. StableVQ's methods can also be applied to other domains that utilize vector quantization.

Key Findings

• Dynamic STE corrects instability in the Encoder's learning objective
• Region VQ Loss guarantees full tracking of the encoder output distribution
• Decoupled Schedule ensures robust system-level behavior