Backpressure Is a Product Boundary
Backpressure is usually discussed as an engineering control: shed load, slow producers, cap queues, protect dependencies. That is all true, but it misses the part that users feel.
When a system accepts more work than it can responsibly finish, it has already made a product promise. The rest of the incident is the organization discovering that the promise was never designed.
The thesis
Backpressure is a product boundary, not just an infrastructure mechanism.
It decides what the product is willing to accept, what it will delay, what it will reject, and what it will ask the user to change. If those decisions are left to queue depth, retry storms, or default timeouts, the product experience becomes accidental.
Good backpressure is not about being harsh. It is about preserving truthful contracts when capacity, dependencies, or downstream recovery are constrained.
The production pattern
The common failure starts with a helpful product surface. Users can upload as much as they want, trigger as many imports as they need, request large reports, run broad searches, start bulk actions, or ask an agent to operate across a wide workspace.
The front door accepts the request quickly. Work moves into a queue. The queue grows. Workers fall behind. Retry policies add more pressure. Users see "submitted" or "processing" while the system silently drifts from minutes to hours. Support asks engineering for status. Engineering asks the queue. The queue says only that there is a lot of work.
The system did not fail at the first error. It failed when it accepted work without a credible plan for priority, delay, cancellation, or refusal.
The model
I think about backpressure through five product decisions:
- Admission: should this work be accepted right now?
- Shape: should the request be reduced, split, sampled, or scoped?
- Schedule: when should it run relative to other work?
- Degrade: what lower-cost result is still honest and useful?
- Refuse: what does the system say when the answer is no?
These are not only engineering knobs. Admission is about entitlement and promise. Shape is about product design. Schedule is about fairness and business priority. Degrade is about user trust. Refusal is about clarity.
The technical mechanisms still matter: quotas, concurrency limits, token buckets, bounded queues, priority lanes, cancellation, and circuit breakers. But those mechanisms should implement a product decision, not invent one during overload.
Where this goes wrong
The first mistake is using the queue as the product contract. A queue can absorb temporary mismatch between arrival and processing. It cannot decide whether a user's request still makes sense after waiting.
The second mistake is hiding delay. "Processing" is easy to ship and hard to operate. If the system knows work is delayed behind a large backlog, it should expose that state. Otherwise users retry, duplicate work, or make business decisions based on false freshness.
The third mistake is treating all work as equal until the workers are overloaded. By then, prioritization is an incident response activity. The system needs policy before overload: paid versus free, interactive versus batch, user-visible versus maintenance, retry versus first attempt, old work versus new work.
The fourth mistake is letting retries bypass admission. A retry is still demand. If the original request was too expensive or the downstream dependency is unhealthy, retrying can turn a local problem into a shared outage.
There is a counterpoint. Some internal batch systems can accept deep backlogs because users do not wait on them and the work remains valuable later. Even there, the product boundary exists. It is just an internal one: how much delay is acceptable, who gets paged, and when stale work should be dropped.
What I do now
I ask product questions during capacity design. What should users see when the system is busy? Which work should be refused rather than delayed? Which operations can be narrowed before acceptance? Which promises are tied to freshness?
I prefer admission checks at the front door. The system should know enough to say "not now," "smaller scope," "scheduled for later," or "accepted with delayed completion" before it creates a pile of work that someone else has to explain.
I make pending states precise. A request can be queued, waiting on dependency recovery, blocked by quota, paused by owner, or verifying after an uncertain outcome. Those are different states with different user actions and operator responses.
I also design cancellation as part of backpressure. If a user no longer needs a large report, import, or agent task, cancellation should stop future work and mark already completed effects honestly. Without cancellation, old demand keeps stealing capacity from current decisions.
For agents, backpressure is a safety feature. A planner that can ask for unbounded reading, writing, or tool execution needs the runtime to set limits. The model should not discover the boundary by exhausting the system.
Closing takeaway
Backpressure is the system telling the truth about capacity before that truth turns into delay, duplication, and confused users.