Distilling Graph Geometry: Knowledge Gap from GNNs to MLPs

TL;DR

The paper proposes Graph Geometry-aware MLP (G^2MLP), a distillation framework that retains the predictive accuracy of a graph neural network (GNN) teacher while deploying a graph-free MLP at inference. G^2MLP uses Ollivier-Ricci curvature to guide the distillation process and improve the student's representation space. This approach consistently improves over graph-free distillation baselines.

Why It Matters

The proposed method enables the deployment of efficient and scalable MLP models for graph-based tasks, which is crucial for real-world applications where computational resources are limited. G^2MLP also reduces the teacher-student rank gap, making it a valuable tool for knowledge distillation. This has significant implications for node classification, link prediction, and other graph-based tasks.

Key Findings

• G^2MLP improves over graph-free distillation baselines
• The method reduces the teacher-student rank gap in both sparse and dense graph regimes
• G^2MLP transfers to Graph Transformer teachers and link prediction without architectural changes