Random Feature Pruning for Multi-Class Classification

TL;DR

The paper introduces SEED-MAX, a sequential feature selection method for random kitchen sinks that identifies the most informative subset of features by minimizing empirical excess risk. It combines this with an F-Test to accelerate pruning, achieving up to 92% feature reduction with minimal accuracy loss on time-series data.

Why It Matters

Reducing redundant features in random feature models improves computational efficiency and model interpretability without sacrificing performance, which is crucial for large-scale or real-time classification tasks.

Key Findings

• SEED-MAX recursively selects features by maximizing empirical excess risk difference, leading to a parsimonious yet effective feature set.
• The algorithm has a theoretical lower bound on correct feature selection probability that approaches one with larger training data.
• Applying the F-Test after SEED-MAX increases pruning speed, achieving 92% feature reduction while maintaining classification accuracy on UCR datasets.