Map
↑Linux preemption models

Linux 7.0 cuts PostgreSQL throughput in half

Wiki summarylinuxpostgresqlkernelperformance ↳ show in map Markdown
title
Linux 7.0 cuts PostgreSQL throughput in half
type
summary
summary
Removing PREEMPT_NONE in Linux 7.0 lets the scheduler preempt processes mid-page-fault while holding a PostgreSQL spinlock, melting CPU on contended workloads
parent
linux-preemption-models
tags
linux, postgresql, kernel, performance
sources
linux-broke-postgresql
created
2026-04-30
updated
2026-04-30

On 2026-04-03, AWS engineer Salvatore Dipietro posted to LKML that the same pgbench simple-update workload on a 96-vCPU Graviton4 went from 98,565 TPS on Linux 6.x to 50,751 TPS on Linux 7.0 — half the throughput from a kernel upgrade alone. perf showed the time was concentrated in one place: 56% of CPU was spinning inside s_lock, called from StrategyGetBuffer in postgresql's buffer manager.

What changed

Linux 7.0 removed PREEMPT_NONE from the available linux-preemption-models on modern CPU architectures. Distributions that defaulted to PREEMPT_NONE for server workloads now default to PREEMPT_LAZY, introduced in 6.12 as a "throughput-friendly" preemption model. For nearly all workloads it really is a drop-in replacement. PostgreSQL hits the one shape where it isn't.

Why PostgreSQL specifically

PostgreSQL caches data in a shared_buffers region — 120 GB in the benchmark. When a backend needs a page that isn't already cached, StrategyGetBuffer picks a victim slot. That selection is protected by a single global spinlock. The critical section is meant to be tens of nanoseconds. The whole point of a spinlock is that the holder is gone before anyone notices.

Two facts collide:

  1. The shared buffer pool is allocated lazily. The first time any byte of a 4 KB Linux page in that region is touched, a minor page fault occurs — the kernel allocates the physical page and updates the mapping. With 120 GB of shared_buffers at 4 KB pages, that's ~31 million potential first-touch faults sprinkled throughout a long run, not just at startup.
  2. StrategyGetBuffer reads and writes shared memory while holding the spinlock. Some of those accesses are the first touch.

So a backend can take a page fault while holding the spinlock. Under PREEMPT_NONE, the kernel handled the fault and the holder kept running until release — spinners waited microseconds. Under PREEMPT_LAZY, the scheduler is free to preempt the holder mid-fault. The lock now stays held for "fault duration + however long until the scheduler picks the holder again." Every other backend on the box is spinning that whole time, multiplying the wasted CPU by the number of waiters. On 96 vCPUs with 1,024 clients, that multiplier is enough to halve throughput.

The fix that already exists

PostgreSQL has a huge_pages setting. Switch from 4 KB pages to 2 MB or 1 GB huge-pages and the math changes:

  • 4 KB: ~31,000,000 potential page faults
  • 2 MB: ~61,440 potential page faults
  • 1 GB: ~120 potential page faults

Two things happen at once. First-touch faults become rare enough that the spinlock holder almost never takes one. And TLB pressure drops sharply — far fewer entries cover the same memory, so the working set sits in the TLB and StrategyGetBuffer runs at full speed. The regression disappears.

The article recommends huge_pages = on rather than the default try, so PostgreSQL refuses to start instead of silently falling back when huge pages aren't actually configured at the OS level.

The fix that was proposed

Peter Zijlstra (the Intel engineer who authored the preemption change) suggested PostgreSQL adopt Restartable Sequences (rseq) — a Linux facility that lets userspace mark a critical section so the kernel signals when it gets preempted, and the code restarts. PostgreSQL's response was cool. It puts the work on PostgreSQL to recover behavior it had for free, and it cuts against the kernel's "don't break userspace" rule. As the article frames it: the regression is real, but the fix being proposed shifts the cost the wrong way.

Why this matters beyond Postgres

The pattern — short critical section + spinlock + a hidden syscall or fault inside it — is a general hazard. It mostly didn't fire under PREEMPT_NONE because the scheduler refused to deschedule the holder. Removing that backstop exposes any program that quietly relied on it. PostgreSQL is the visible case because its workload concentrates contention on one global lock; the same shape exists in other long-running, highly-concurrent shared-memory systems.

Sources

Linked from