Spindle — QEMU microVM CI engine for Tangled

An engineering post from tangled (the atproto-based git collaboration platform) introducing a QEMU-microVM execution engine for Spindle, its self-hostable CI runners. Each workflow gets its own real virtual machine — a full environment you can do anything inside — with the security boundary of a VM and the startup profile of a microVM. It's an upgrade to the existing Nixery engine and stays fully compatible: change nixery to microvm in a workflow and it works.

What a microVM buys here

Same architecture as qemu-microvm and pve-microvm: a VM with the boring parts removed — no BIOS, no PCI probing, no emulated graphics, no legacy devices — just virtio, so it boots fast and uses little memory. QEMU is the only runner today, but the engine is written so other runners (Firecracker is named) can slot in later. This gives Spindle real kernel isolation for untrusted CI workloads, the CI-specific case of the argument in microvm-2026 and sandboxing-ai-agents that a container is not a security boundary.

The vsock guest agent

Spindle never SSHes into the guest or runs commands "from outside." Inside the VM runs a small agent; the moment it boots it dials back to Spindle over vsock, says hello, and from then on every workflow step is sent to it as a message. The agent runs each command as an unprivileged user, streams stdout/stderr back, and reports exit codes. The host side lives in spindle; the guest side is a small Rust binary called shuttle implementing a protocol called agentproto — anyone could implement it and supply their own agent. This host↔guest-over-vsock pattern is the same plumbing pve-microvm uses for SSH-agent forwarding and that agent-sandbox orchestrators like superhq use to control guests without exposing them on the network.

NixOS images configured from the workflow

The distinctive part. Two image flavours:

• NixOS images — the whole guest is built with Nix, so you configure the machine directly from the workflow YAML. New keys the NixOS image understands:
  • dependencies — packages your steps get, placed in a mkShellNoCC devshell every step sources (so you get the full stdenv, pkg-config wiring, etc. — you can compile openssl-sys without pain). Bare names resolve from nixpkgs; flakeref#attr pulls from any flake.
  • registry — remap global refs (pin nixpkgs to nixos-unstable, alias your own flakes).
  • caches — map of binary-cache URL to trusted public key, wired into the read proxy so the guest substitutes prebuilt paths instead of building from scratch.
  • services / virtualisation — passed straight through to NixOS. services.postgresql.enable brings Postgres up before any step; virtualisation.docker: true gets a real Docker daemon inside the VM. By the time step one runs, Postgres is listening and the Docker socket exists — no sidecar dance. (true is shorthand for .enable = true anywhere an enable option exists.)
• Non-NixOS images — today just Alpine, but can be anything. No workflow-level NixOS config, but if Nix exists in the image it can still talk to the Spindle Nix cache.

Two-way Nix caching

Boots are quick on the second run because Spindle caches aggressively in both directions: dependencies, services, and any Nix derivation built inside the microVM get pushed to Spindle's Nix cache, so the next workflow that needs them doesn't rebuild. If Spindle has built a given base-plus-config before, it hands the guest a store path to realize (fetching from a configured cache) instead of rebuilding — so identical second runs are fast. Like everything in Tangled, the whole thing is self-hostable.

Where it sits

A concrete, CI-shaped instance of the microVM-as-isolation-substrate pattern, notable for pushing declarative machine config (NixOS) all the way into the CI manifest. Related: microvm, qemu-microvm, pve-microvm (the same QEMU-microVM engine for Proxmox), matryoshka-isolation, sandboxing-ai-agents, and features-to-steal-from-npmx (the earlier Tangled mention in the vault).