# Everyone Should Know SIMD

Mitchell Hashimoto's claim is that the everyday case of SIMD has one shape, that the shape is about as hard to write as a for loop, and that engineers who dismiss SIMD as a specialist's tool are turning down a 4x-to-16x speedup on loops they already have. The examples are in [[zig]] because that's what Ghostty is written in, but nothing in the argument depends on the language beyond having generic vector types.

The trigger to look for is any loop of the form `for (byte in bytes)`, `for (character in string)`, `for (value in array)`. SIMD turns it into `for (8 byte chunk in bytes)`. The payoff scales with the lane count and requires the data to be large enough to amortize the setup — hundreds of thousands or millions of bytes pays, a few dozen doesn't. Hashimoto is explicit that simdutf and simdjson do something much harder than this and that you don't need to go there to benefit.

## The five steps

1. Broadcast the constants you need and initialize any vector accumulators.
2. Loop over the input one vector-width chunk at a time.
3. Do the comparison or arithmetic across all lanes at once.
4. Reduce or store the vector result.
5. Handle the leftovers with a scalar tail, which is the loop you started with.

## The Ghostty example

The loop consumes decoded codepoints until it hits a value at or below `0xF`, which is where a printable run ends. Terminals are mostly printable characters, so finding the end of the run fast is worth doing. The scalar version is one line:

```zig
while (end < cps.len and cps[end] > 0xF) end += 1;
```

The vector version adds twelve lines and no CPU-specific intrinsics:

```zig
if (simd.lanes(u32)) |lanes| {
    const V = @Vector(lanes, u32);
    const threshold: V = @splat(0xF);
    while (end + lanes <= cps.len) : (end += lanes) {
        const values: V = cps[end..][0..lanes].*;
        const greater_than_threshold = values > threshold;
        if (@reduce(.And, greater_than_threshold)) continue;
        const mask: std.meta.Int(.unsigned, lanes) = @bitCast(greater_than_threshold);
        end += @ctz(~mask);
        break;
    }
}

while (end < cps.len and cps[end] > 0xF) end += 1;
```

`simd.lanes(u32)` is a Ghostty helper returning how many `u32` values the target processes at once: 4 on ARM NEON, 8 on AVX2, 16 on AVX-512. It returns `null` when there's no vector width worth using, and then the whole block is skipped. `@splat(0xF)` copies the threshold into every lane, because a vector comparison needs a vector on both sides.

The loop only runs when a *complete* vector remains. Five values left with eight lanes means the vector loop can't load them, and rather than reach for a masked-load trick, Ghostty leaves them to the tail.

The comparison `values > threshold` is a single vector instruction producing one boolean per lane:

```
values:                 { 0x41, 0x42, 0x43, 0x0A, 0x44, 0x45, 0x46, 0x47 }
threshold:              {  0xF,  0xF,  0xF,  0xF,  0xF,  0xF,  0xF,  0xF }
greater_than_threshold: { true, true, true, false, true, true, true, true }
```

Step 4 is where algorithms diverge, and it's the part that looks alien. `@reduce(.And, ...)` collapses the booleans with `and`; when everything passed, `continue` moves to the next chunk, which is the common case for terminal text. Otherwise the boolean vector is `@bitCast` into an integer with one bit per lane, inverted so failures become `1`, and `@ctz` counts trailing zeros to give the index of the first failure:

```
greater_than_threshold: { true, true, true, false, true, true, true, true }
mask:                   {    1,    1,    1,     0,    1,    1,    1,    1 }
~mask:                  {    0,    0,    0,     1,    0,    0,    0,    0 }
```

`@ctz(~mask)` returns 3, which is the lane holding `0x0A`. The scalar tail then handles the zero-to-seven remaining values, and doubles as the fallback path when `simd.lanes` returned `null`. The original implementation stays in the file doing two jobs.

Hashimoto puts the ceiling at 4x on NEON including Apple Silicon, 8x on AVX2, 16x on AVX-512. Measured end to end on an AVX2 Intel desktop — terminal program output through to finalized terminal state — the improvement was about 5x. The gap between 8x and 5x is the surrounding work that didn't get vectorized.

Two caveats he attaches in footnotes are worth keeping. Generic vectors remove the CPU-specific *syntax*, not the CPU-specific code generation; Zig still lowers these builtins to whatever instruction set the target enables. And the single vector instruction is the comparison only — loading the vector, reducing the result, and locating the failed lane each cost their own instructions.

## Why not let the compiler do it

Sometimes it will. Auto-vectorization handles simple arithmetic loops without complicated control flow, and Hashimoto's advice is to compile the scalar version with optimizations and read the output before writing anything by hand. But he points at [recent work](https://arxiv.org/abs/2406.04693) that still opens from the observation that production compilers routinely miss vectorization opportunities, and treats that as a durable state of affairs rather than a bug about to be fixed.

The stronger objection is about predictability. When a loop matters enough to want 5x, he wants the vectorization written down, not inferred: "I don't want an unrelated code change or compiler update to quietly turn it back into a scalar loop." That is exactly the failure mode documented in [[compiler-codegen-luck]], where rewriting `*p = x; p++;` as `*p++ = x` flipped Clang between a branch and a branchless `csel` and moved a quicksort by more than 6x. The same reasoning drives [[go-bounds-checks-unsafe]]: when you can prove something the compiler won't, the choice is to state it explicitly or to keep re-checking that the optimizer still agrees with you.