Interpreting Supervised Pairwise Graph Representations: Local Evidence, Confounding, and Provable Post-hoc Correction

TL;DR

Supervised pairwise graph representations often appear to capture meaningful higher-order structure, but their apparent advantages over simpler baselines vanish when controlling for recoverable nuisance variables like sequence length. The study demonstrates that local evidence and confounding factors must be rigorously accounted for before attributing signal to graph topology, and proposes a Fisher post-hoc correction that maintains validity across diverse representations. A cross-attention analysis further reveals redundant circuitry and failure modes in Transformer models.

Why It Matters

This work provides a critical framework for evaluating graph-based representations in neural models, preventing spurious attribution of signal to topology. It underscores the importance of rigorous causal reasoning in interpretability, especially in high-stakes domains like fMRI analysis and error detection. The proposed correction method offers a robust, distribution-agnostic tool for validating graph interpretations under limited data.

Key Findings

• Sequence length is nearly perfectly recoverable from pooled hidden states (R² = 0.9995), explaining apparent gains in supervised pairwise graphs.
• Edge magnitude alone performs near chance; topology-aware augmentations yield minimal ROC-AUC improvements (≤0.004).
• A Fisher post-hoc corrector maintains statistical guarantees across representations and remains stable under reduced labeling budgets.