#pretraining

Wiki 6

• 2 OLMo 2 Furious AI2's OLMo 2 report, fully open 7B/13B/32B models on up to 6.6T tokens with a stability fix list, Dolmino mid-training and the Tülu 3 RLVR recipe
• Olmo The Allen Institute for AI's family of fully open language models, OLMo (2024) to OLMo 2 to Olmo 3, released with data, code, checkpoints and logs
• Olmo 3 AI2's Olmo 3 report (Dec 2025), fully open 7B/32B Base, Think, Instruct and RL-Zero models with every stage's data, code and checkpoints released
• OLMo: Accelerating the Science of Language Models AI2's first OLMo release (Feb 2024), 1B and 7B models on 2T+ Dolma tokens with weights, data, code, logs and 500+ checkpoints under Apache 2.0
• Pulpie: Pareto-Optimal Models for Cleaning the Web Encoder models that label HTML blocks as content or boilerplate at a twentieth of Dripper's cost
• Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling EleutherAI's 2023 Pythia suite, 16 models from 70M to 12B trained on the Pile in one fixed order, with 154 checkpoints each for training-dynamics research