Frontiers in synthetic data

Nathan Lambert wrote this in June 2024 on interconnects, while starting a project to take a strong open-weight base model and push it as far as possible with synthetic data. That project's target was an open fine-tune that closes most of the gap to the frontier with every part of the pipeline public, and the post is his inventory of what the big labs were probably doing that the academic literature was missing. Two years later he put it at the end of his open-source-ai-reading-list as the "more history" entry for llm-distillation. It is useful as a baseline: it records what "distillation" meant before reasoning models, RL at scale, and the political fight over Chinese labs.

Distillation as SFT on a stronger model's text

In 2024 the working meaning of distillation was plain supervised fine-tuning on completions from a stronger model. Lambert dates the phase to Zephyr-Beta, whose paper put "distillation" in the title. He is quick to point out how uneven it is. Open-ended requests such as "write me an email" or "summarize this text" are easy to copy from GPT-4, and most evaluations people start with (MT-Bench, AlpacaEval) reward exactly that. Code that calls a semi-popular library correctly, or hard math, is much harder, because the teacher fails at it too.

He also notes how stale many "created with GPT-4" datasets already were. They used a GPT-4 snapshot about 100 points behind the current GPT-4o on the LMSYS Chatbot Arena, the same distance that separates the original GPT-4 from the widely disliked Llama-2-chat-70B. Regenerating an old dataset with a newer teacher can therefore help a lot, and nobody knew which teacher or sampling settings were best for which task.

The logit-based kind at the big labs

The more technical claim is about small, cheap endpoints. Lambert read Gemini Flash and Claude Haiku as probably distilled rather than trained from scratch, and pointed to Google's on-policy distillation paper (Agarwal et al.), which its lead author called his most impactful product work. An edit to the post confirmed half of it: the updated Gemini 1.5 report says Flash is a dense model "online distilled" from Gemini 1.5 Pro, an MoE. In that setup the small model is updated alongside the large one's training. Lambert guessed Anthropic and Google had this capability and OpenAI might not. A second edit links an explanation of on-policy distillation from Lewis Tunstall at Hugging Face. The rlhf-book-synthetic-data-distillation chapter picks up this same line of work in 2026, when on-policy distillation had become a standard post-training method.

Filtering and accumulation, not closed loops

Iterative DPO papers were the fashionable alignment research at the time, most of them regenerating or relabeling one dataset and reporting AlpacaEval scores Lambert calls a bit absurd. Industry reports looked different. Llama 2 and Nemotron 340B both ran several rounds of alignment and kept adding data between rounds, and both filtered hard: Meta used Llama 2 to build the quality classifiers for Llama 3's pretraining data, and Nvidia's Nemotron paper is mostly about a reward model used to throw out low-scoring dialogues. Lambert connects this to work showing that accumulating data across iterations avoids the mode collapse seen when a model is simply retrained on its own outputs. His open question was why nobody had written up how that filtering should be done per task and per model.

Terms of service and the teacher gap

Big companies could not use the same synthetic data as academics and hobbyists. Nous Research's Hermes fine-tunes leaned heavily on GPT-4, while Nvidia was limited to permissively licensed models such as Mistral or its own outputs. For the open community, the right to use GPT-4, Gemini Pro and Claude 3 as teachers was a large advantage over a lab like Meta. Lambert observed that the worry about doing this had faded since 2023: small players trained on API outputs and nobody seemed to care. He adds that many "no training on our outputs" clauses came from the model owners' data vendors, who needed them to protect their own business. The how-much-does-distillation-matter-for-chinese-llms post in 2026 returns to the same clause, by then turned against Chinese labs.

Datasets built from many teachers

UltraFeedback and Nectar were the standard preference datasets, and both mix completions from nearly every model generation. Lambert suspects that caps their ceiling: the student is asked to learn sequences that are unlikely under its own distribution, such as GPT-4's pet phrases in a Llama 3 base, and nobody knows what that loss does to the model. He expected on-policy generation plus filtering to do better. He also concedes in an edit that the UltraFeedback authors had struggled to beat their own dataset with UltraInteract, so the mixed approach was still winning in academia.

Where he saw the opening

The post closes on three directions. Verifiable synthetic data, first for instruction formats like "answer in JSON" (aimed at IFEval) and then, he expected, for code that is known to run and math that is known to be right. Weak-to-strong generalization, where he quotes the Nemotron report's finding that the teacher did not cap the student. And prompts, which he argues are the real bottleneck: synthetic data started with self-instruct, a method for making prompts, and Nemotron wrote its own synthetic prompts even though LMSYS-Chat-1M existed.

Several of these predictions held up in his own later writing. Verifiable rewards became the basis of RL for reasoning models, and his July 2026 note how-distillation-is-used-today says getting good prompts is often the hardest part of building an SFT set. The student-beats-teacher point reappears in 2026 as his argument that Chinese models are not capped by the American models they learn from.