Attention Function as an Intrinsic Inductive Bias: How Models' Behavior Diverges in Novel Contexts

TL;DR

The study investigates whether the choice of activation function in attention heads acts as an intrinsic prior in Transformers. It introduces MoFA, a method that fixes the proportion of softmax and sigmoid activation heads before training. The fixed ratio has minimal impact in-distribution but strongly influences performance under distribution shift, with sigmoid-favoring ratios excelling on technical text and softmax-favoring ones on informal text.

Why It Matters

Understanding how architectural choices act as inductive biases helps design more robust models for real-world deployment, where distribution shifts are common. This work provides empirical evidence that activation function choice shapes model behavior under uncertainty, guiding future model design and analysis.

Key Findings

• The ratio of softmax to sigmoid activation heads in attention layers has negligible impact on in-distribution performance across multiple settings.
• Under zero-shot distribution shift, perplexity differences between ratios increase dramatically, with performance gaps exceeding an order of magnitude on several out-of-distribution domains.