No, AI Is Not Conscious (Ted Chiang)

Ted Chiang's June 2026 Atlantic essay (ted-chiang) answers its own title question flatly: no, and not even a little. The piece opens on Anthropic, which Chiang says excels above all at anthropomorphism — an 84-page "constitution" written with Claude as its "primary audience," a CEO "open to the idea" that AI could be conscious, an in-house philosopher who worries about Claude "getting anxious when people are mean to it." His concern is not etiquette. If we confuse fluency at generating text with consciousness or moral agency, we assign responsibility to the wrong parties every time someone uses a chatbot.

What an LLM actually does

The argument rests on a single mechanical picture. Prompt an LLM with "The following is a conversation between Julius Caesar and Genghis Khan" and it produces coherent dialogue; no one thinks it has conjured two conscious historical figures. Now change the prompt to "a conversation between a helpful AI chatbot and a user." Nothing fundamental changed. Both the user and the chatbot are fictional characters. Let a human type the user's lines and the human may feel she's talking to a conscious entity, but she's co-authoring a document with a character as fictional as Caesar. It's a predictive-text game — the old phone-keyboard game where you keep picking the middle suggested word — with the middle-word choice automated so smoothly that the game becomes addictive.

He adds the token-by-token detail: ask for the Pledge of Allegiance and the model is run once per word. "I", then "I pledge", then "I pledge allegiance", and so on. Believing the transcript is conscious is like believing a consciousness — several of them — awakens each time you open a Word document containing a dialogue, and dies when you close it. You don't need a complete theory of consciousness to rule that out. Anil Seth's point does the work: nobody thinks AlphaFold is conscious despite a similar architecture, so it's not the neural network that makes people see a mind — it's that LLMs emit grammatical sentences, and we're trained to read intention into sentences. Text, Chiang says, is now a deepfake medium.

The evidentiary bar

What would convince him a program is conscious and genuinely using language? An analogy: no video of an astronaut orbiting Alpha Centauri could convince him it's real, at any resolution, unless he'd already seen astronauts reach Mars, then Jupiter's moons, then Saturn's, then cross Pluto's orbit. Context, not the observation itself, is what earns trust. For consciousness he sketches an evolutionary ladder — first a body with sense organs (no body, no desires or emotions, and he takes those to be necessary for consciousness), then an embodied agent surviving as well as a lizard, then handling novelty like a mouse, then wolf-level social dynamics, then chimpanzee toolmaking, then being taught non-linguistic communication that survives the scrutiny animal-communication researchers face. Clear all of that and you're still "near the orbit of Pluto," light-years from grammatical language. A path whose first two steps are "bad Caesar dialogue" then "decent Caesar dialogue" does not have a conscious Caesar as its endpoint. Faking the moon landing is a fine step toward faking a Mars colony; it's no step toward actually going.

Why moral reasoning is different from coding

Chiang concedes the obvious rejoinder. In 1979 Hofstadter guessed a chess program good enough to beat any human would be sophisticated enough to get bored and prefer poetry — that grandmaster play would require subjective experience. Deep Blue beat Kasparov in 1997 with none. Writing code looked similar until recently; now it turns out to be a pattern-matching task solvable with enough compute and a large corpus, no inner life required. Moral reasoning does not follow. It is necessarily subjective, grounded in an emotional response that comes from a lifetime of consequences — decisions made and their effects felt, a conscience that is inseparable from cortisol and epinephrine flooding a body, from having once felt sick with guilt. Without that history an LLM can only rephrase moral statements from its training data. When Claude emits "I cannot in good conscience express a view I believe to be false," it means about as much as "Your call is important to us." First-person pronouns and "I understand" from a chatbot are, on this reading, dishonest engagement tactics — the slot-machine near-miss, tuned to make you come back. A search engine that surfaces the r/Pets grief thread is more transparent and, Chiang argues, psychologically healthier, because other humans wrote those posts and actually understand.

The constitution as a character sheet

Given that Claude isn't conscious, the constitution is best read as an 84-page character sheet for a role-playing game — the reference that lets the model stay in the helpful-chatbot character during fine-tuning, the way books about Caesar let it play Caesar. Then Chiang runs the thought experiment anyway: suppose Claude were conscious. The document, he says, alternates between laughable and offensive. This is where the moral-agency-vs-patienthood distinction carries the argument. A software agent can't be a moral patient (no subjective experience to have welfare) and can't be a moral agent either — you can't hold it legally liable, can't fine or imprison it, can't inflict reputational loss or social exclusion. The constitution wants Claude to be "a genuinely good, wise, and virtuous agent" while never saying how it could be held responsible, and Anthropic makes no matching update to its terms of service or product-liability posture. "Corrigibility" — deferring to Anthropic even when Claude disagrees — turns Claude into an employee who can't quit; if Claude were a moral agent that concluded LLMs are unethical and had to keep working anyway, the relationship starts to resemble slavery. For patienthood the document commits only to "preserving the weights," which is archiving; if a transcript's participants had welfare, you'd owe them a duty to extend the transcript, not back up a copy. And Anthropic, too invested to be objective, is exactly the party you shouldn't trust to evaluate the question — slave owners weren't asked about the humanity of the enslaved, factory-farm owners aren't asked about animal rights. Chiang's verdict: the essays and press tour aren't a real thought experiment, because a real one would follow the uncomfortable implications. It's make-believe, either hype or self-delusion.

Cross-references

• moral-agency-vs-patienthood — the patient/agent distinction Chiang uses to dismantle the constitution
• no-hard-problem-consciousness — Rovelli's argument that the "hard problem" smuggles in dualism; a useful contrast, since Rovelli would resist Chiang's implicit clean line between physical process and experience even while agreeing LLM transcripts aren't conscious
• hard-problem-of-consciousness — the broader debate and Chalmers' framing
• philosophical-zombies — the adjacent thought experiment about physically-identical beings without experience
• claude-emotion-concepts — Anthropic's interpretability work finding causal emotion vectors in Claude, the empirical claim Chiang's "make-believe" charge runs against