Tier 5 Fully open / reproducible
What the publisher claims: open, open research, fully documented
The one-liner: The cleanest true reproducibility case: same data, same order, every checkpoint published.
The six questions
all sizes and intermediate checkpoints downloadable
Apache 2.0
standard Transformers and serving stacks
training data, datasheet, hyperparameters, checkpoint cadence and evals all published
a skilled group can rebuild a substantially equivalent model from the published Pile and code
EleutherAI is a volunteer collective that ran the Pythia scaling suite through public GitHub and Discord discussion, took community-contributed Pile datasets, and let outside contributors influence the checkpoint cadence and the model naming.
License
Apache 2.0 OSI-approved — license source ↗
What is actually released
- 16 models across 8 sizes trained on the same data in the same order
- The Pile dataset with a datasheet
- 154 intermediate checkpoints per model hosted as branches
- training code
- evaluation scripts and paper
Why it matters
The benchmark case for what fully open means, and the reason the book can show that reproducibility is a design choice rather than an accident.
Sources
Derived from the openness dataset behind But Is It Open? — snapshot 2026-11-07 ↗. Scores are qualitative readings of the cited evidence, not verdicts; tier measures openness, not intelligence.