A Safe Path to Open Weights

Thinking Machines Lab published this post in July 2026, shortly after releasing Inkling and Inkling-Small, its two open-weight language models. The lab believes open weights are good: they spread training and deployment expertise beyond a few labs and make a model's built-in assumptions inspectable and revisable. The post is its account of how to square that with misuse risk. Nathan Lambert's open-source-ai-reading-list calls it "a clear articulation on how to balance releasing powerful open-weight models while taking safety seriously", and puts it at the head of the safety sub-list. It is the only piece in that group written by a lab that ships open weights itself.

The central claim is that a release is safe or unsafe depending on two things, the model and the ecosystem it lands in. Most safety writing looks only at the first.

The risk it takes seriously

The post does not open with the usual reassurance. Its example is Anthropic's April 2026 report that Claude Mythos Preview found thousands of previously unknown vulnerabilities across every major operating system and browser and wrote working exploits without human guidance. If open weights give everyone that, the lab asks, is society safer? It says the net effect on the offense-defense balance is genuinely uncertain. The same capability patches systems in defenders' hands and exploits unpatched ones in attackers' hands, and chemistry and biology have the same shape. Its conclusion is blunt: "releasing weights indiscriminately is not a safe path forward."

That puts it at odds with two other pieces on Lambert's list. myth-of-unsafe-open-source-ai argues from incident reports that real misuse runs almost entirely through closed models, and that Mythos-class releases will not change that. openai-huggingface-incident-autonomous-hacking argues that attackers already have frontier open weights, so restraint mostly slows defenders. Thinking Machines never names either argument, but its staged-release programme is a direct bet against the second one: that a window of defender-only access is worth having.

Is the model safe?

Safety testing is described as "an imperfect but essential proxy". The questions are what harmful tasks the model can complete, how accessible that makes them, and what happens once guardrails are gone. The last question matters most for open weights, because refusal training can be fine-tuned away and "cannot be treated as a durable safeguard".

Inkling is not a frontier model, so the lab's question was incremental: does releasing it add material risk beyond what downloadable models already pose? Being below the dangerous-capability frontier was judged necessary but not sufficient. The lab also checked whether multimodality, ease of customization or accessibility would make existing harmful capability easier to use.

Testing ran on three tracks. Internal evaluations covered CBRN and offensive cyber (both knowledge and whether the model could put it into practice), a broad misuse set including agentic tool-use settings, and a multimodal harmful-content suite in 17 languages across text, image and audio. Four external organizations red-teamed pre-release: Scale AI on general misuse against the model spec, Handshake AI on vulnerable-user interactions (suicide, self-harm, eating disorders, child safety), FAR.AI on CBRN and cyber, and Apollo Research on loss-of-control behaviour such as scheming, evaluation awareness and sabotage. The third track was adversarial fine-tuning: the lab trained "helpful-only" variants that comply instead of refusing and ran them against the dual-use evaluations. On all three, Inkling came out in line with existing open-weight models, and the stripped variants gave no new CBRN or cyber uplift. The lab says plainly that this balance will shift as its models approach the frontier.

The research bet is whether dangerous capability can be separated from general intelligence. The common assumption is that it emerges untrained and tracks general capability, which is why cyber-range scores get treated as one more benchmark to climb. The lab hypothesizes that some dangerous knowledge, especially in biology, rests on specific empirical findings (protocols, reagents, conditions) learned from particular documents rather than derived by reasoning. If so, it can be filtered out of pretraining without hurting ordinary use. It cites early evidence that document-level CBRN filtering lowers harmful-capability scores while leaving other skills intact (O'Brien et al., "Deep Ignorance", 2025). It also names the ways this could fail: a strong reasoner might re-derive what was removed, and the line between dangerous and ordinary technical knowledge may be too blurry to cut.

Is the ecosystem ready?

Readiness means layered defense: organizations able to patch early or catch attacks in progress, plus research that lets models strengthen defenses at scale. The awkward part is that defenders need access to capable models to build those layers, and every widening of access also widens misuse. The answer the post gives is staging, and each stage has a specific job.

Early inference access for defenders comes first, modelled on Anthropic's Project Glasswing, which gave a small group of trusted organizations Mythos Preview before public release. The post compares it to responsible disclosure, where vendors get a private window to patch. Next is fine-tuning access for vetted defenders. The examples are the winning DARPA AI Cyber Challenge team, which fine-tuned a model to find and fix vulnerabilities, and OpenAI's gpt-oss-safeguard classifiers, fine-tuned from the open gpt-oss weights. Then white-box access for vetted safety researchers. Here the post makes a point open-weights advocates often make: what we know about how shallow refusal is came from studying open weights. Refusal is bypassable by token-level adversarial optimization (Zou et al., 2023), removable along a single activation direction (Arditi et al., 2024), and concentrated in the first few output tokens (Qi et al., 2024). The last stage is monitored public access before full release, to see misuse patterns while controls still exist.

Fine-tuning APIs like the lab's own Tinker sit between inference access and open weights. The provider keeps the weights and can monitor, keep guardrails and revoke access, while users bring their own data, loss functions and training loops. The post claims this captures most of the benefit of openness at a lower misuse ceiling. It is also the lab's product, which the post does not dwell on.

The stages are not a conveyor that always ends in released weights. Each widening should happen only when evidence supports it, and a model held back today may be released later once defenses catch up. The stages also have to move quickly, or the defensive learning goes stale before the next capability jump.

What the post leaves open

The post calls itself "a high-level framework, not a complete release standard" and lists what it has not answered: what evidence justifies moving to the next stage, how uncertainty should weigh on that call, what should pause progression, and how ecosystem readiness is measured. It promises a detailed framework with evaluations, access criteria and stop conditions later, and announces Tinker safety grants.

The staged-release idea is older than this post. OpenAI released GPT-2 in stages in 2019, as the history in societal-impact-of-open-foundation-models recounts, and gradient-of-generative-ai-release treats release as a gradient rather than a binary. What Thinking Machines adds is a defender-first sequence tied to named 2026 programmes, plus pretraining filtering as a way to make the final open-weights stage less dangerous. Its framing matches Helen Toner's in nonproliferation-is-the-wrong-approach-to-ai-misuse: a delay before open release is worth something only if defenders use it. Joshua Saxe's national-ai-cybersecurity-policy attacks exactly the capability-threshold gating that this post's testing regime feeds into. The safety argument that arguments-against-open-source-ai declined to engage is argued here from the pro-open side, and in more mechanical detail than the frontier labs' critics usually get.