Distributional Split Criteria for Random Forests: Extensions, Shrinkage, and the Robustness of Mean Splitting

TL;DR

The paper evaluates multiple distributional split criteria for random forests, including MMD-based and Wasserstein-based methods, within a unified honest-forest framework. It finds that isotropic MMD performs nearly as well as more complex variants, while mean-based splitting remains robust for univariate regression. Distributional methods show clear advantages only in multivariate settings with non-location dependencies.

Why It Matters

The results guide practitioners on when to use complex distributional splits versus simpler mean-based ones, preventing unnecessary computational overhead. The open-source drforest library enables efficient exploration of these criteria, supporting robust and scalable model development.

Key Findings

• Isotropic MMD-based splitting performs competitively across all tested scenarios and is not consistently outperformed by more complex variants.
• Mean-based CART splitting remains the most reliable default for univariate tabular regression tasks.
• Distributional splitting provides measurable benefits in multivariate response settings, especially when dependencies go beyond marginal location shifts.