Hallucination Neurons and Where to Find Them: An Investigation into the existence of Hallucination Neurons

TL;DR

This paper investigates the existence of hallucination neurons in large language models and proposes a diagnostic protocol to validate sparse-neuron localization claims. The results demonstrate detection replicates across models and datasets, but indicate non-unique neuron selection. The study highlights the need for routine diagnostic validation in mechanistic interpretability.

Why It Matters

The findings have practical significance for model auditing and behavioral steering, as they can help identify and mitigate potential issues in language models. The proposed diagnostic protocol can be used to evaluate the reliability of sparse-neuron localization claims, which is crucial for trustworthy AI applications. This research contributes to the development of more transparent and explainable language models.

Key Findings

• detection replicates across models and datasets
• non-unique neuron selection
• sparse predictive structure coexists with non-unique neuron selection