Sky sphere representation in language models

TL;DR

Researchers analyzed language models to determine if they have a decodable representation of the night sky map. They found that most open-source models have such a representation, which can be surfaced with specific prompts. This representation contains up to 85% of variance and has a median angular error of 12-21 degrees.

Why It Matters

This discovery has significant implications for our understanding of how language models represent and process complex, high-dimensional data. It also opens up new possibilities for using language models in astronomy and other fields that involve complex spatial relationships. The findings could lead to more accurate and efficient models for tasks like object detection and localization.

Key Findings

• decodable night sky representation in language models
• significant variance explained (up to 85%)
• low median angular error (12-21 degrees)