Dynamics to decision: A mathematical theory of Lyapunov spectra and decision boundaries in deep classifiers

TL;DR

This paper proposes a mathematical theory of Lyapunov spectra and decision boundaries in deep classifiers, providing a framework for understanding decision geometry emergence. The theory models a feed-forward classifier as a dynamical system and studies the Finite-Time Maximum Lyapunov Exponent. This approach offers insights into the evolution of data samples through classifier layers.

Why It Matters

Understanding the dynamics of deep classifiers is crucial for developing more efficient and effective models. This research has practical significance in improving the interpretability and robustness of deep learning models. It can also inform the development of layer-aware regularization strategies and geometry-aware fine-tuning methods.

Key Findings

• The FTMLE carries a geometric signature of the decision boundary at the probability level
• The relationship between FTMLE and decision boundary depends on the loss function at the logits level
• Boundary-related FTMLE can persist at the hidden representation level under certain structural conditions