Permutation-Equivariant Flow Matching for Alignment-Free Neural Weight Generation

TL;DR

The paper introduces a flow-matching approach using a permutation-equivariant graph meta network to generate neural network weights without requiring neuron alignment across independently trained models. It enables direct learning from diverse model collections, capturing joint distributions of accuracy, function, and weight similarity. The method generates task-specific models across varied architectures and unseen widths, matching ensemble performance on domain-shift tasks.

Why It Matters

This approach eliminates the need for costly alignment procedures when training generative models over neural networks, enabling scalable and flexible model generation across architectures. It supports conditional generation of high-performing models without memorizing specific checkpoints, advancing the feasibility of automated neural architecture design and transfer learning.

Key Findings

• A permutation-equivariant graph meta network enables flow matching directly on unaligned weight vectors from independently trained models.