The study examines whether finite experiments can reliably measure integrated information (φ) in small stochastic networks, revealing that identical input-output behaviors can arise from vastly different internal mechanisms. Even with perfect data, partial observations and symmetry in hidden units impose fundamental limits on precision, and small changes in mechanism can cause large jumps in φ. However, lower bounds on φ can be certified with high confidence using controlled interventions.
Why It Matters
This work establishes practical limits on measuring mechanistic properties like integrated information from real-world experiments, which is critical for AI safety, interpretability, and assessing machine consciousness. It shows that certifying even minimal levels of φ requires careful experimental design and sufficient data, guiding future research on reliable mechanistic auditing.
Key Findings
Identical input-output behaviors in recurrent and feed-forward networks can mask large differences in internal computation, requiring only 4–53 trials to distinguish via internal recordings.
Partial observations of network units impose irreducible precision limits (0.30–1.02 bits) that no amount of data can overcome due to hidden unit symmetries.
Integrated information φ is highly sensitive to mechanism changes near partition boundaries, with 99% of edges in random networks showing 25–100% jumps from infinitesimal perturbations, and 1% of well-fitting surrogate models misreporting φ by over 20%.
Topics
integrated_information
mechanism_identification
causal_inference
ai_interpretability
experimental_design
Thank you to arXiv for use of its open access interoperability. arxiv.org. aibrief.ai is not affiliated with or endorsed by arXiv.
Despite sensitivity, lower bounds on φ can be rigorously certified across high-dimensional mechanism spaces using around 10^7 controlled transitions with a false-certification probability ≤ α.
Measurability of φ depends on experimental design, and certifiable bounds are computable, making it a design problem rather than a purely theoretical one.