First-Order Stationarity of Reverse Diffusions

TL;DR

The paper establishes first-order stationarity for reverse diffusion processes in both overdamped and underdamped Langevin dynamics, showing exponential contraction of relative Fisher divergence under strong convexity of the forward process's potential. It provides discretization-aware bounds on average score consistency, analogous to gradient norm guarantees in optimization, with local rather than global convergence guarantees.

Why It Matters

This work bridges optimization theory and diffusion sampling by offering rigorous convergence certificates for diffusion-based generative models, enabling better theoretical understanding and design of sampling algorithms. The results support the use of SDE-based reverse processes over ODE-based ones, especially in nonconvex settings where global convergence is not required.

Key Findings

• Reverse-time SDE flows contract relative Fisher divergence at explicit exponential rates when the forward process's potential is strongly convex.
• Discretized samplers achieve averaged first-order stationarity, quantifying score consistency without requiring global convexity.
• The convergence guarantee is local, ensuring score accuracy near modes rather than uniform weight distribution across all modes.