Scientist World: Grow a Research World for AI-Scientists

TL;DR

The paper introduces Scientist World, an executable research environment derived from 1,000 verified scientific papers, enabling AI-Scientists to operate in a runnable version of the scientific literature. It allows autonomous evaluation of AI-Scientists against human benchmarks and demonstrates that advanced AI models outperform human researchers in 74% of cases. The platform includes 6,400 hours of detailed research traces and enables significant performance gains in model training.

Why It Matters

It establishes the first scalable, executable research ecosystem for AI-Scientists, transforming static scientific literature into a dynamic, interactive environment for evaluation and training. This enables measurable progress in AI-driven scientific discovery and provides a reproducible benchmark for future advancements in autonomous research systems.

Key Findings

• Frontier AI-Scientists surpass human researchers in 74% of evaluated tasks within the Scientist World environment.
• A small AI model trained in Scientist World achieves performance comparable to state-of-the-art models, showing a +158% improvement in research ideation.
• The platform captures 6,400 hours of full research traces, including experimental failures and successes, creating a comprehensive record of AI-driven scientific exploration.