# Spindle — QEMU microVM CI engine for Tangled

An engineering post from [[tangled]] (the atproto-based git collaboration platform) introducing a QEMU-**microVM** execution engine for **Spindle**, its self-hostable CI runners. Each workflow gets its own real virtual machine — a full environment you can do anything inside — with the security boundary of a VM and the startup profile of a microVM. It's an upgrade to the existing Nixery engine and stays fully compatible: change `nixery` to `microvm` in a workflow and it works.

## What a microVM buys here

Same architecture as [[qemu-microvm]] and [[toolbox/pve-microvm]]: a VM with the boring parts removed — no BIOS, no PCI probing, no emulated graphics, no legacy devices — just virtio, so it boots fast and uses little memory. QEMU is the only runner today, but the engine is written so other runners (Firecracker is named) can slot in later. This gives Spindle real kernel isolation for untrusted CI workloads, the CI-specific case of the argument in [[microvm-2026]] and [[sandboxing-ai-agents]] that a container is not a security boundary.

## The vsock guest agent

Spindle never SSHes into the guest or runs commands "from outside." Inside the VM runs a small agent; the moment it boots it dials back to Spindle over **vsock**, says hello, and from then on every workflow step is sent to it as a message. The agent runs each command as an unprivileged user, streams stdout/stderr back, and reports exit codes. The host side lives in `spindle`; the guest side is a small Rust binary called `shuttle` implementing a protocol called `agentproto` — anyone could implement it and supply their own agent. This host↔guest-over-vsock pattern is the same plumbing [[toolbox/pve-microvm]] uses for SSH-agent forwarding and that agent-sandbox orchestrators like [[toolbox/superhq]] use to control guests without exposing them on the network.

## NixOS images configured from the workflow

The distinctive part. Two image flavours:

- **NixOS images** — the whole guest is built with Nix, so you configure the machine *directly from the workflow YAML*. New keys the NixOS image understands:
  - `dependencies` — packages your steps get, placed in a `mkShellNoCC` devshell every step sources (so you get the full stdenv, `pkg-config` wiring, etc. — you can compile `openssl-sys` without pain). Bare names resolve from nixpkgs; `flakeref#attr` pulls from any flake.
  - `registry` — remap global refs (pin `nixpkgs` to `nixos-unstable`, alias your own flakes).
  - `caches` — map of binary-cache URL to trusted public key, wired into the read proxy so the guest substitutes prebuilt paths instead of building from scratch.
  - `services` / `virtualisation` — passed straight through to NixOS. `services.postgresql.enable` brings Postgres up before any step; `virtualisation.docker: true` gets a real Docker daemon inside the VM. By the time step one runs, Postgres is listening and the Docker socket exists — no sidecar dance. (`true` is shorthand for `.enable = true` anywhere an `enable` option exists.)
- **Non-NixOS images** — today just Alpine, but can be anything. No workflow-level NixOS config, but if Nix exists in the image it can still talk to the Spindle Nix cache.

## Two-way Nix caching

Boots are quick on the second run because Spindle caches aggressively in both directions: dependencies, services, and any Nix derivation built inside the microVM get pushed to Spindle's Nix cache, so the next workflow that needs them doesn't rebuild. If Spindle has built a given base-plus-config before, it hands the guest a store path to realize (fetching from a configured cache) instead of rebuilding — so identical second runs are fast. Like everything in Tangled, the whole thing is self-hostable.

## Where it sits

A concrete, CI-shaped instance of the microVM-as-isolation-substrate pattern, notable for pushing declarative machine config (NixOS) all the way into the CI manifest. Related: [[microvm]], [[qemu-microvm]], [[toolbox/pve-microvm]] (the same QEMU-microVM engine for Proxmox), [[matryoshka-isolation]], [[sandboxing-ai-agents]], and [[features-to-steal-from-npmx]] (the earlier Tangled mention in the vault).