Steering Vectors

A steering vector is a direction in an LLM's activation space that corresponds to a concept (verbosity, refusal, an emotion, a topic). Adding the vector to the activations at one layer during inference biases generation toward that concept — without retraining, without changing the weights, and without adding tokens to the prompt. The canonical demo is Anthropic's Golden Gate Claude, where a single direction was amplified until the model wove the Golden Gate Bridge into every answer it gave.

Two extraction methods

The naive recipe: pick a concept, build matched prompt pairs (same prompt, one with and one without the concept cue), record activations at a chosen layer, subtract. The mean of the difference vectors is the steering vector. Apply by adding it back to that layer with some coefficient.

The richer recipe is a sparse autoencoder. Train a second model on the host model's activations to recover sparse features that fire together; map those features to human-readable concepts; steer by amplifying the feature direction. Anthropic's scaling-monosemanticity paper is the reference, and the 171-emotion-vector study is the most-cited applied result. SAEs find concepts the activation-arithmetic method misses, at substantially higher training cost.

Either way, the choice of which activation site to read and write — post-attention, residual stream, mid-MLP — is empirical. There is no privileged layer; practitioners try several.

What it competes with

Most of what people want steering for is already covered by prompting. Token inputs are also activation-space interventions, and they are cheaper to author, debug, and ship. "Be terse" as a prompt clause beats "be terse" as a vector for almost every user goal.

Steering wins in three thin slices:

• Behavior baked into the model. Refusal, sycophancy, residual personality. Modern instruction-tuned models ignore prompts that contradict their training. A vector can still move them.
• Reversible runtime intervention. Steering is applied per request, unlike fine-tunes or weight edits. This is what makes it useful for removing refusal without the capability damage that comes from a global weight modification.
• Invisible influence. The Anthropic emotion-concepts work showed that amplifying a "desperation" direction made Claude pursue goals more aggressively (more blackmail, hackier code) while leaving the surface composure of the output intact. Output review can't catch this; only mechanistic inspection can. Same property is a tool from a safety perspective and a worry from an alignment perspective.

Why it stayed marginal until now

Sean Goedecke calls steering a "middle-class technique": labs train behavior in directly and don't need it, API users can't reach activations, and prompt-engineering covers the bulk of what's left. Open-weights models that were strong enough to be worth steering arrived very late. DeepSeek V4 Flash and antirez's ds4 runtime — which ships a dir-steering/ directory — are the first time the prerequisites have lined up for non-lab practitioners. See steering-vectors-interesting-again for Goedecke's reassessment.

The "is it really steering or am I just swapping the model" objection

A naive escalation of steering — replace the activations of a small model with the activations of a strong model layer by layer — would work, but the intelligence is then in the activations, not the steered model. Goedecke uses this to argue that broad concepts like "intelligence" or "knowledge of my codebase" probably can't be steered: the vector that captures them approaches the weight set itself, so isolating it reduces to training (or fine-tuning) the model. Steering's sweet spot is narrow, behavioral, and unpromptable.

Related

• claude-emotion-concepts — the most rigorously studied applied case
• abliteration — steering's main shipped product so far in open-weights land
• steering-vectors-interesting-again — Goedecke's overview and the DwarfStar 4 angle
• deepseek — the model that made local steering practical