Rethinking the design space of Vanilla Pixel Diffusion Transformers

TL;DR

The paper introduces MFlow, a dynamic parameterization for pixel-space diffusion transformers that adaptively blends clean image and velocity prediction based on data and model capacity. It shows that optimal parameterization shifts toward velocity prediction as model size increases, and proposes ConvRefiner to fix persistent high-frequency artifacts in generated images. The approach achieves strong performance across image, video, and text-to-image tasks.

Why It Matters

Pixel-space diffusion models avoid the quality loss from autoencoders, but face challenges in high-frequency detail and parameterization choice. This work provides a flexible, trainable framework that improves image fidelity and adaptability across scales and modalities, enabling more effective direct pixel generation.

Key Findings

• MFlow learns to interpolate between x0- and u-prediction along data-driven principal directions during training.
• Model capacity influences the optimal parameterization, with larger models favoring velocity prediction.
• ConvRefiner, a lightweight post-hoc module, restores high-frequency details without retraining the diffusion backbone.