Don't Go Quietly Into the AI Night

Two AI futures

There are two visions for the future:

• AI as a deity built and controlled by a small group of clergy
• Humans at the center with AI as an amplifier

A new technical clergy is emerging: the small group that builds frontier systems, receives privileged access to them, and decides which capabilities everyone else may use.

The clergy warn about mass unemployment (Dario Amodei warned AI could eliminate half of entry-level white-collar jobs within five years, May 2025; Sam Altman said customer support jobs are "totally, totally gone", July 2025; Mustafa Suleyman predicted most computer-based professional work fully automated within 18 months, May 2026; Elon Musk proposed "universal high income", April 2026). They don't think it will necessarily lead to a bad outcome. Freeing people from grunt work can liberate time for leisure.

The average person hears mass unemployment as catastrophic. How do they make rent or buy food for their kids? We are assured that abundance, redistribution, and new forms of meaning will compensate (Altman's "Moore's Law for Everything", "The Gentle Singularity"; Amodei's "Machines of Loving Grace") — by the grace of the machines and their controllers.

A tiny group mediates between machine intelligence and everyone else. Humanity does not broadly participate in directing that intelligence. Most people become recipients of decisions, products, and abundance. Not participants.

Initially the clergy may administer the machine. But as its capabilities surpass their own, it becomes less clear who is directing whom. In this vision, they build a deity, and increasingly their role is to tend it, interpret it, and decide who may approach it.

There's another path: not one central intelligence ruling over billions of passive users, but billions of humans learning to direct capable agents of their own.

The future is unevenly distributed

The clergy focus feverishly on sharpening the tip of the spear so it can pierce ever harder domains: solving century-old mathematical problems (Erdős Problem #728 resolved autonomously, Jan 2026; an 80-year-old Erdős problem, May 2026) and finding software vulnerabilities undiscovered for 30 years (Claude Opus 4.6 found 500+ high-severity vulnerabilities; Claude Mythos Preview found thousands of zero-days across every major OS and browser, including a 27-year-old OpenBSD bug). They will cure diseases and generate immense value. But who will have access to the cure for cancer? The cure for aging?

Drug discovery capabilities are restricted because of bioweapons risks (Anthropic activated ASL-3 with Claude Opus 4; the Claude 5 generation ships Fable 5 with safety classifiers while unrestricted Mythos 5 is available only to approved organizations). Software capabilities because of cyber risk. It's not hard to see what comes next: math capabilities limited because of cryptographic risks, creative capabilities because of disinformation risk (OpenAI restricted Sora after deepfake complaints; EU AI Act Article 50 deepfake-disclosure mandate takes effect August 2026).

Governments required many of these restrictions, and frontier labs often supported them (ahead of GPT-5.6's launch, OpenAI previewed capabilities with the US government; the models were its first rated "High" risk in biology and cybersecurity; rollout began with ~20 government-approved partners). Selected partners, researchers, and institutions retained access (Project Glasswing limited Mythos Preview to a small group; a US export-control order forced Anthropic to disable Fable 5 and Mythos 5 worldwide, and when the ban lifted Mythos 5 returned only for government-approved organizations). The result is the beginning of a two-tiered system: frontier AI for a small group, constrained AI for everyone else.

There are real dangers in making powerful capabilities universally available. Biosecurity is real. Cybersecurity is real. Disinformation is real. But genuine safety concerns shouldn't lead to exclusion and permanent dependency.

The frontier is racing ahead. The median is standing still.

The vast majority of humans don't know how to use the ever-sharpened spear.

Software development gives a peek into the future, because developers got AI agents a year before everyone else (Cursor's agent mode Nov 2024, Claude Code Feb 2025; general knowledge workers got the equivalent only when Claude Cowork launched Jan 2026). Initially the super users were maybe twice as productive. Over the past year and a half that increased to 5x, then 10x.

The best agentic developers are now probably exceeding 100x, doing massive rewrites in days (Bun, 535,496 lines of Zig, rewritten in Rust in 11 days by one engineer supervising up to 64 concurrent Claude Code instances; a 6,755-commit pull request passing the full test suite; hand estimate ~750 engineer-days). The author created NanoClaw over a weekend covering many technologies they had no prior experience with; pre-AI it would have taken six to eight months, or never been finished.

But those developers are a tiny fraction — far less than one percent have reached 100x. The median developer hasn't gained any meaningful increase (METR's randomized trial found experienced developers 19% slower with AI tools in early 2025; its Feb 2026 follow-up found only "very weak evidence" of modest speedups). Every week there's a new model or feature. 4.6, 4.8, 5.5, fable, Sol. Hooks, Skills, loops, workflows. With each release the super user unlocks a new level, while the median just gets more confused and the gap keeps growing.

The same thing has started to play out across the workforce — 10x salespeople, 100x marketers (Anthropic's own case study: a single growth marketer running performance marketing at scale with Claude Code). But the average person in the workplace is still using a personal $20/month subscription on the side (Okta found 52% admit to using unapproved AI tools at work).

The median hasn't budged. A handful of 10x outliers barely moves the average. The way to move the average is to make the median person 2x.

The future is built, not announced

Having a very sharp spear is a good thing. Curing cancer is a good thing. But we shouldn't let the clergy dictate the vision and the direction for the world we all build.

Most people do not need another order of magnitude of model intelligence. They need the intelligence we already have made usable and integrated into their work.

This is not only a question of who gets access to AI. It is a question of who participates in building with it. The current trajectory is a widening gap: fewer and fewer people doing more and more. That trend needs to be reversed. The future should not only be built for billions of people. It should be built by billions of people. Universal participation, not just universal access.

Build the internet for agents. Build agents for humans.

Humans at the center with AI as an amplifier of human creativity, productivity and prosperity.

Build products for agents. Build your website for agents. Build the internet for agents. But build agents for humans.

If you are building AI products, build for the median user, not only the power user. Design so the user gains judgment and control. Design for the person managing the agent, not for the agent replacing the person. Put the human at the center: they set the objective, they review the work, they own the result. Default to the user owning what matters: the agent's identity, memory, skills, artifacts, and history.

There are orders of magnitude more people who have never interacted with an agent than there are superusers of Claude Code.

If you're an AI startup going into sales calls telling execs you're going to replace their team with agents — stop. If you're an executive asking a founder how their agents can replace your team — stop. This leads nowhere good (Klarna cut support staff claiming AI did the work of 700 agents, then admitted lower quality and resumed hiring; by 2026, 55% of leaders who made AI-driven redundancies now admit those decisions were wrong).

A company whose only AI strategy is headcount reduction is using an exponential technology for a linear goal. The fastest path to 2x your company is to make the median person 2x.

Not your agents, not your company

Who's managing the agents?

If it's not your team managing the agents, then your vendor is managing them. And your company is becoming redundant. You've given someone else valuable access, knowledge, and data that helps automate you out of existence. This is the clergy pattern: a small group outside your walls mediating between your people and the intelligence they depend on.

You need agents for the core growth engine of your business. They need to be managed by you and your team. They need to be your agents, not somebody else's. You need fewer humans doing manual work, but more agents and more agent managers. The best people to be those agent managers are the people currently doing the work — deep domain knowledge, experience, relationships. But they need to learn to become managers.

Every person becomes an agent manager

You know how to communicate what you want, do systems thinking, give feedback, review others' work, and take responsibility for the output of those working for you. Those are the skills needed to effectively manage agents.

Give each person on the team their first agent. Not in a group, but one-to-one. Their agent, owned by your company and issued to them, that they manage and that helps them do their work. Over time they manage a second and third. When they're good at it, some of those agents join the team channel. When they've earned trust, a few run public-facing agents. The best agent managers will manage a team of agents, then a department.

Sovereign agents

Disclosure: this is what my company builds.

Your agents need to be sovereign. Your company must own and control the agents' identities, permissions, memory, skills, artifacts, and audit trails. Those assets must be portable, governable, and inaccessible to anyone you have not authorized.

The future has not been decided

Intelligence could be concentrated, restricted, and handed down to us. Or it can be placed in the hands of billions of people and used to increase human agency.

So don't accept prophecy as policy. Don't mistake concentration for safety. Do not go quietly into the AI night. Put humans at the center. Give them agents. Teach them to lead.

HN discussion highlights (item 48874182)

• Timwi — the piece preaches accessibility for billions but then insists agents be "sovereign… inaccessible to anyone you have not authorized." Reads as if all 8 billion people will each run their own company still competing in capitalism.
• Terr_ (rebuttal) — no contradiction; it's the original personal-computer democratization pitch. Yours to own and express your will, not open to strangers. "Affordable housing" isn't "pay $5 to crash in anybody's bedroom."
• kristianc — "who manages the agents" has the grifty smell of "your devs download random NPM packages" and "Shadow IT": an enterprise-security anxiety surface that always ends in a priesthood and a six-figure governance platform rather than practical process fixes. (Pointed, given the author sells exactly this.)
• ekidd — the author's own vision is also suspect if models get much smarter. You can't hand every human a personal SkyNet, so the agents can't be too capable; and if the model is smarter than you, managing it contributes little — human management gets displaced too. Under real RSI, labs might never sell the model via API and just run businesses with it directly.
• simonw — adopt the DRI (Directly Responsible Individual). Accountability should stay a human who then uses all the agents they like. You can't put a ball of matrix arithmetic on a Performance Improvement Plan. Others (narrator, mejutoco): a human has time preference and a body that can be fired or jailed; that's what makes accountability real.
• blauditore / weego / dominotw — reality check against the "future of humanity" rhetoric: current agents still get calendar and meeting scheduling wrong ~80% of the time. Grand civilizational framing from people shipping unreliable tools.
• bardown59 — a practical containment pattern: never give production models write authority; the model emits a structured suggestion, deterministic code validates it against pre-set constraints, minimizing blast radius.