# pve-microvm

A Debian package that makes QEMU's [[qemu-microvm|`microvm` machine type]] a first-class managed guest inside **Proxmox VE**. It closes the gap between LXC containers (instant, shared kernel, not isolated) and full VMs (hardware-isolated, but 5-10s to a login prompt): a microVM boots in under 300ms into a virtio-only guest that still sits inside its own KVM hardware boundary. Written by Rui Carmo ([taoofmac.com](https://taoofmac.com/space/blog/2026/06/18/1845)) for his own homelab, where it now runs Gitea, Caddy reverse proxies, mini-firewalls, and the agent that manages his Proxmox cluster.

The pitch is "container-speed VMs with actual isolation" — the [[matryoshka-isolation|VM-style boundary]] of [[microvm]] platforms made routine on a hobbyist Proxmox node rather than at hyperscaler scale.

## How it works

When you set `machine: microvm` on a VM config, the package intercepts Proxmox's QEMU command builder and constructs a stripped machine: no BIOS, no GRUB, no PCI bridges, no VGA, no USB. The guest gets a single serial console (wired into PVE's `xterm.js`), a virtio-blk disk, and a virtio-net NIC, booted by direct kernel load (`-kernel`/`-initrd` on the host). This is the same architecture as [[qemu-microvm]] and Firecracker; the contribution here is the Proxmox integration, not the machine type.

Three design decisions carry the project:

**Host-provided kernel, kernel-agnostic rootfs.** A single 12MB `vmlinuz` (Linux 6.12.22, built from stock `x86_64_defconfig` plus a virtio/vsock/virtiofs/9p/Docker-modules overlay) lives on the Proxmox host at `/usr/share/pve-microvm/vmlinuz`. Every Linux microVM on the node boots that *same* kernel. The guest disk holds only a root filesystem — no `/boot`, no per-guest kernel package, no `initramfs`. You update or audit the kernel in exactly one place, and no guest can ever pull a broken kernel from an `apt` upgrade because no guest *has* a kernel to upgrade. You don't install an OS; you assemble a rootfs from an OCI base image (`pve-microvm-template` supports 12 of them — Debian, Alpine, Fedora, Rocky, Amazon Linux and friends) or import a prepared ext4/raw disk with `qm importdisk`. Carmo calls the summary "container-style kernel consistency, VM-style isolation."

**It patches Proxmox's Perl internals at install time.** The `.deb` modifies three files in the PVE Perl stack: `Machine.pm` (accept `microvm` as a valid machine type), `QemuServer.pm` (delegate to the custom builder when `machine: microvm`), and ships `MicroVM.pm` as the complete command builder. Patches are reversible (`pve-microvm-patch revert`, originals backed up), re-applied automatically after PVE upgrades via a `dpkg` trigger (`interest-noawait qemu-server`), and forced live before any VM auto-starts by a systemd early-boot service (`pve-microvm-early.service`, ordered `Before=pvedaemon.service pve-guests.service`).

**PCIe transport, not MMIO, for Linux guests.** This diverges from the classic [[qemu-microvm]] design. QEMU `microvm` can carry its virtio devices over bare-bones MMIO (lightest — how a SmolBSD/NetBSD guest boots in 31ms) or over PCIe. On QEMU 10.x the MMIO path has a device-probing bug for Linux guests — only `virtio-blk` binds, and net/serial/balloon drivers never claim their devices — so every Linux guest falls back to PCIe with non-transitional (modern-only) virtio devices, which binds all of them reliably. Cost is ~50ms of extra bring-up against a 300ms boot. Carmo suspects it's a bug in his own kernel config and explicitly asks for contributor patches.

## What a config looks like

A microVM is an ordinary `qm` guest with a particular `machine` type and a kernel cmdline — `/etc/pve/qemu-server/114.conf` for his Gitea guest:

```
agent: 1
args: -kernel /usr/share/pve-microvm/vmlinuz -append "console=ttyS0 root=/dev/vda rw quiet"
boot: order=scsi0
cores: 2
machine: microvm
memory: 2048
name: gitea
net0: virtio=BC:24:11:00:6E:01,bridge=vmbr0
onboot: 1
scsi0: local-lvm:vm-114-disk-0,size=32G
serial0: socket
vga: serial0
```

The only microVM-specific lines are `machine: microvm`, the `args` carrying the kernel + cmdline, and `serial0`/`vga: serial0` wiring the console to `xterm.js`. `MicroVM.pm` injects the `-initrd`, balloon, vsock and (when configured) virtiofs devices automatically. Everything else — cores, memory, the virtio NIC on `vmbr0`, the disk, `onboot`, the guest agent — is exactly what you'd write for any Proxmox VM.

## Getting started

```
# On any PVE 9.x node:
wget https://github.com/rcarmo/pve-microvm/releases/latest/download/pve-microvm_0.3.12-1_all.deb
dpkg -i pve-microvm_0.3.12-1_all.deb

# Build a Debian microVM template (~60s: pulls OCI image, installs packages, writes rootfs):
pve-microvm-template --vmid 9000 --storage local-lvm --profile standard

# Clone it into a real VM:
qm clone 9000 100 --name my-microvm --full
qm set 100 --machine microvm --memory 1024 --cores 2
qm start 100
```

Linked clones after the first template build are near-instant.

## Why it's "just a normal PVE guest"

Because the disk is a plain `virtio-blk-pci` device and the NIC lands on a normal Linux bridge, almost everything Proxmox does works unchanged: every storage backend (LVM/LVM-thin, ZFS, Ceph/RBD, NFS, CIFS, directory), snapshots, linked and full clones, `vzdump` backups, the per-VM `nftables` firewall, and network isolation via Proxmox [SDN](https://pve.proxmox.com/wiki/Software-Defined_Network) VLAN/VXLAN zones. Carmo deliberately does *not* re-solve network isolation inside the package — the policy lives in Proxmox SDN, not a one-off ruleset. A non-networked `vsock` CID (VMID + 1000) handles host→guest SSH-agent forwarding and virtiofs/9p directory sharing. Memory sizing is a ceiling, not an allocation: KVM backs only touched pages, same-page merging dedupes shared kernels/libc/base-image layers across every microVM on the node, and recently added `balloon` (with `free-page-reporting`/`deflate-on-oom`) plus `virtio-mem` give real auto-ballooning and fine-grained hot-add/remove.

## Limitations

- **No live migration** — QEMU's `microvm` machine type doesn't implement it. Offline migration works; because boot is sub-second, a stop→migrate→start HA relocate on shared storage takes ~2s.
- **Patching someone else's product is fragile.** Every `qemu-server` upgrade can break the setup. Partial/half-applied PVE upgrades have left `pvedaemon` unable to compile the patched module. Always do a full `dist-upgrade`, never a partial one, on nodes running this.
- **`/dev/vda` → `/dev/sda` trap.** If a guest ever starts under the standard chipset (half-applied patch, or an `onboot=1` VM starting before the early-boot service re-patches), the disk enumerates as `/dev/sda`, root isn't found, and the guest looks like it lost its filesystem. The `initrd` now falls back to `/dev/sda`, but expect the occasional papercut until (if ever) Proxmox supports microVMs natively.
- **Serial console only** — no VGA, no graphical console. Fine for servers; boot-time kernel panics are read on a terminal. Plan 9 guests dislike it.
- **No USB at all** — no controllers, no passthrough. Rules out USB Zigbee/Z-Wave dongles (Carmo's home automation stays in LXC).
- **GPU/PCI passthrough is disabled**, not impossible — `hostpci*` is stripped and there's no vIOMMU plumbing. An RTX 3060 worked in early testing; he turned it off for simplicity and lack of hardware to test properly.
- **The kernel is opinionated.** If your workload needs a module not in the custom 6.12.22 build, rebuild it or use the stock Debian kernel (works, but ~3× larger and slower to boot).

## Range of guests

Carmo runs a genuine menagerie on it, each booted and validated rather than theoretical: bare-metal Gitea with SQLite + Caddy, a `piclaw` agent that manages the whole cluster and runs Docker internally, an `exo` distributed-inference coordinator, and — because he likes odd operating systems — 21 guest OS types spanning Debian/Alpine/Fedora/Rocky/Amazon Linux, OpenWrt, OPNsense, OSv unikernels, gokrazy Go appliances, SmolBSD (NetBSD, 31ms via MMIO), and a dormant 9Front (Plan 9). His baseline torture platform was a fanless 2016 Atom x5-Z8350 with 2GB RAM, which ran six microVMs before slowing down — the "if it works here it works anywhere" test.

## Watchlist

On [[toolbox/watchlist]]: v0.3.x, single-author, Proxmox-only, and structurally coupled to patching upstream Perl internals it doesn't control. Worth re-checking whether the version matures, a second maintainer appears, or the MMIO/Linux and GPU-passthrough gaps get contributor patches — Carmo is explicitly asking for both but has no time to chase them himself.

## Related

- [[qemu-microvm]] — the QEMU machine type this packages for Proxmox (and the MMIO transport it can't reliably use for Linux guests)
- [[microvm]] — the architectural concept; [[microvm-2026]] — the 2026 microVM/agent-sandbox industry readout
- [[sandboxing-ai-agents]] — the broader agent-sandbox taxonomy this feeds; [[matryoshka-isolation]] — containers-inside-VMs defense in depth
- [[toolbox/proxmox-manager]] — a TUI for managing the same Proxmox guests

---

Repo: [github.com/rcarmo/pve-microvm](https://github.com/rcarmo/pve-microvm) — ~320 stars, Apache-2.0. Announced at [taoofmac.com](https://taoofmac.com/space/blog/2026/06/18/1845). Source: [[pve-microvm-taoofmac]].