Tier 4 Deeply open
What the publisher claims: open source, open data
The one-liner: A small model trained entirely in public.
The six questions
Weights, data, and code all downloadable.
Apache 2.0.
Runs on consumer hardware.
Trained entirely on public datasets (SlimPajama + Starcoderdata) with the mixture and schedule published.
A determined group can genuinely approximate the run at 1.1B scale, at real cost.
Small academic project with 7 non-bot external merges and a public training repo, now effectively dormant.
License
Apache 2.0 OSI-approved — license source ↗
What is actually released
- weights (ungated)
- training code (FlashAttention, llama2.c)
- public training datasets: SlimPajama-627B and Starcoderdata
- checkpoints at intermediate steps
Why it matters
Proof that the Chinchilla-optimal small-model recipe can be executed end-to-end on open data — the reproducibility ceiling most frontier releases never approach.
Sources
Derived from the openness dataset behind But Is It Open? — snapshot 2026-11-07 ↗. Scores are qualitative readings of the cited evidence, not verdicts; tier measures openness, not intelligence.