Message Passing Is Shared Mutable State

The author of causality-blog argues that the shared-memory vs. message-passing debate has been a false dichotomy from the start, using Go as the empirical test case. In 2006 Edward Lee published The Problem with Threads and predicted that switching the coordination mechanism from locks to messages would change the syntax of failure but not the underlying model. Three years later Go launched on the opposite bet — "do not communicate by sharing memory; share memory by communicating" — and got adopted at scale by Docker, Kubernetes, etcd, gRPC, and CockroachDB. That made Go the most prominent real-world test of the message-passing hypothesis the industry has ever run.

What the bug study found

Tu and colleagues studied 171 real concurrency bugs across the flagship Go projects and published Understanding Real-World Concurrency Bugs in Go at ASPLOS 2019. Around 58% of the blocking bugs (goroutines stuck and unable to make progress) were caused by message passing rather than shared memory. The thing supposed to be the cure was producing the same problems as the disease.

Message passing did eliminate one class of bug: unsynchronized memory access. If two goroutines only ever communicate through channels, they cannot simultaneously mutate the same variable. But eliminating data races does not eliminate coordination failures — deadlocks, leaks, protocol violations, and nondeterministic scheduling all remain. Go's built-in deadlock detector caught only 2 of the 21 blocking bugs the researchers tested. The race detector did better on non-blocking bugs, catching roughly half, which still leaves half of all production concurrency bugs invisible to the tools designed to find them.

Most of these bugs had long lifetimes. They survived testing and code review in some of the most heavily scrutinized Go codebases.

The Kubernetes leak

The simplified bug from the paper is a one-character mistake:

func finishReq(timeout time.Duration) ob {
    ch := make(chan ob)
    go func() {
        result := fn()
        ch <- result  // blocks forever if timeout wins
    }()
    select {
    case result = <-ch:
        return result
    case <-time.After(timeout):
        return nil
    }
}

If fn() exceeds the timeout, the parent returns nil and nobody reads from ch. The child blocks on the unbuffered send forever. Go garbage-collects objects but doesn't garbage-collect goroutines blocked on channels. In Kubernetes, every leaked goroutine pins references and never releases memory; under load they accumulate until the process degrades or gets OOM-killed. The fix is changing make(chan ob) to make(chan ob, 1).

The structural identity argument

Side-by-side, the same bug in Java uses BlockingQueue<Result> from java.util.concurrent — the package that lives next to Mutex and Semaphore. No Java developer would call it message passing; they'd say "I'm using a shared concurrent queue" and would know it carries all the risks of shared mutable state. But the Go channel code has the same structure: same shared mutable data structure, same blocking semantics, same leak when the consumer goes away. Only the vocabulary changed.

The original sin

Arthur O'Dwyer, writing about the same paper, identified what he called the original sin of Go channels: they aren't really channels. A real channel has two distinct endpoints (a producer end and a consumer end) with different types and capabilities. If the last consumer disappears, the runtime can detect it, unblock producers, and clean up. A Go channel has none of this — it's a single object, a concurrent queue, shared between however many goroutines hold a reference. Any goroutine can send, any can receive, no directional typing, no way for the runtime to detect that one side is gone.

Once you see this, the bug categories in the study become predictable:

• Deadlock — Goroutine A sends and waits for a response on another channel; B does the reverse. Both block. A circular dependency on shared state, expressed through queues instead of locks. Found in Docker, Kubernetes, gRPC.
• Leak — Nobody reads from a channel; the sender blocks forever. The shared queue retains a reference to the goroutine. The Kubernetes bug above.
• Race — Multiple goroutines read from the same channel; the runtime scheduler picks one nondeterministically. Found in etcd and CockroachDB.
• Protocol violation — Goroutine sends a message the receiver doesn't expect, sends on a closed channel (which panics), or closes an already-closed channel.

Every one is a classic shared-mutable-state bug wearing a message-passing costume. See go-channel-bug-patterns for the full taxonomy.

Erlang as the strong-form test

Erlang processes have separate heaps, no shared references, and copy messages between processes — the strongest form of the message-passing guarantee available anywhere. Christakis and Sagonas still found previously unknown race conditions in Erlang's heavily-tested standard library, clustered around ETS tables. ETS is Erlang's escape hatch from pure actor isolation: shared mutable storage that exists because the pure actor model couldn't meet performance requirements. The escape hatch reintroduced exactly the bugs the model was supposed to prevent.

The author's framing: message passing solves concurrency bugs the way moving the mess from one room to another solves clutter. The communication mechanism (channel, mailbox, message queue) is itself a shared mutable resource and inherits every problem shared mutable state has always had. See message-passing-vs-shared-memory.

What's next

The essay ends as a setup for a follow-up. If both sides of the dichotomy fail for the same structural reason, maybe the dichotomy itself is wrong, and the question to ask is what foundation both approaches share. The author teases that some languages have tried different foundations with real insight but none has reached the mainstream — that's where the next post will go.

References

• Lee, The Problem with Threads, IEEE Computer 39.5 (2006)
• Tu et al., Understanding Real-World Concurrency Bugs in Go, ASPLOS 2019
• O'Dwyer, Understanding Real-World Concurrency Bugs in Go (blog post, June 2019)
• Christakis & Sagonas, Static Detection of Race Conditions in Erlang, PADL 2010