Map
↑Agents vs Daemons

Stateful agent routing primitive

Wiki conceptdistributed-systemsai-agentspub-subarchitecture ↳ show in map Markdown
title
Stateful agent routing primitive
type
concept
summary
The missing piece in cloud-native architecture for talking to a specific durable process — a routable transport name that isn't a server, well-fit to pub/sub channels
parent
agents-vs-daemons
tags
distributed-systems, ai-agents, pub-sub, architecture
created
2026-05-21
updated
2026-05-21

The "stateless server behind a load balancer, state lives in the database" pattern has been the cloud-native default for 20 years. It assumes any request can hit any server, and the server's only job is to read from and write to the database. AI agents break this assumption because the things you want to interact with are processes, not requests.

Zach Knill names the missing piece as a routable transport name that isn't a server. You want to be able to say "deliver this message to whoever is producing output for workflow X" without knowing which box, replica, or process. HTTP and load balancers cannot express this — they route to servers, which is the wrong abstraction.

Candidate primitives

  • HTTP + database polling — the current default. Treats the database as a message bus. Inefficient, high-latency, leaks tokens (in LLM terms) on every retry.
  • WebSockets — bi-directional and stateful, but a connection rather than an address. If it drops, you cannot reconnect to the same process because there's no name.
  • Pub/sub channels — the transport itself is addressable. Both server and client connect to the channel by name. Disconnect-and-reconnect preserves the route. Durable.

When the architecture clicks

For a durable-execution system like Temporal, Inngest, or Restate:

  • The workflow connects to a channel named after the workflow ID.
  • The client connects to the same channel for updates, interrupts, and steering messages.
  • The workflow is durable. The channel is durable. Either side can drop and reconnect without losing the ability to route to the same process.

This decomposes cleanly: durable execution for the workflow, pub/sub for the addressable transport, stateless HTTP for everything else.

Why LLMs make this visible

Two properties of LLM responses make the missing primitive painful in a way it wasn't before:

  • Non-deterministic — you can't retry a dropped connection and get the same response back.
  • Expensive — paying for tokens means a wasted response is a real cost, not just latency.

The combination makes polling-as-routing-workaround unaffordable, which makes the missing primitive impossible to ignore. The stateless web isn't wrong; it's just a bad fit for agents that want long-running, stateful, interactive processes.

Sources

Linked from