Rate limiting vs throttling vs backpressure

Published 18 October 2026

Every system has a speed it can work at. When work arrives faster than that, something has to give. The only question is what, and these four words are four different answers:

• Rate limiting: refuse anything over an agreed rate. "This API key gets 100 requests a second; the 101st gets 429 Too Many Requests." The bouncer at a full club.
• Throttling: slow the caller down instead of refusing. Requests over the rate wait and are served later. The toll plaza that lets cars through one at a time.
• Backpressure: the overloaded component signals upstream to send less, and the slowdown travels back towards the source. A full kitchen tells the waiters to stop taking orders.
• Load shedding: when overloaded anyway, deliberately drop some work, the least important first, so the rest still succeeds.

People use "rate limiting" and "throttling" loosely and sometimes interchangeably (AWS calls its 429 style rejections "throttling"). In a design discussion, be precise about whether the excess is rejected, delayed or pushed back.

Watch overload play out#

The unbounded queue is the instructive failure. Nothing errors at first. The queue just grows, latency grows with it, clients time out and retry (adding even more load), memory climbs, and then the process dies and loses everything it had accepted. An overloaded system without limits does not degrade gracefully, it falls off a cliff.

Rate limiting: protect yourself and be fair#

Rate limits sit at the edge, usually in an API gateway or a shared middleware, keyed by API key, user, IP or route. They protect your capacity from one noisy caller, enforce pricing tiers and blunt abuse such as credential stuffing. The algorithms (token bucket, sliding window) and the distributed counting problem are covered in design a rate limiter.

Good rate limit responses tell the caller what to do:

HTTP/1.1 429 Too Many Requests
Retry-After: 2
RateLimit-Policy: 100;w=1
RateLimit: limit=100, remaining=0, reset=2

Throttling: slow, not refused#

Throttling queues or delays the excess so it is served at the allowed rate. It fits callers that would rather be late than fail: a nightly sync job, a bulk email sender, a mobile app uploading photos in the background. It needs a bound too: a delay queue that grows without limit is just the unbounded queue again, wearing a different name. Client side throttling is common as well: an SDK that spaces out its own calls to stay under a provider's limit.

Backpressure: let the slowdown travel#

Backpressure works by making queues bounded and letting a full queue block or refuse its producer, which then slows down, which fills its own input, and so on up the chain until the original source sends less. Nothing is lost and memory stays capped, because each stage only accepts what it can hold.

It is everywhere once you look. TCP flow control: the receiver advertises how much buffer space it has, and the sender cannot exceed it. Node.js streams: write() returns false when the buffer passes highWaterMark, and pipe() pauses the source until drain. Kafka consumers pull at their own pace, so lag grows in Kafka's durable log instead of in a consumer's memory. Go channels with a buffer size block the sender when full.

// Reads a 20 GB file as fast as the disk allows and pushes it at a slow upload.
// Everything the network cannot take yet piles up in memory.
readStream.on('data', (chunk) => upload.write(chunk))

Load shedding: choose what fails#

Even with limits, a service can be pushed past capacity: a dependency slows down, a cache goes cold, a retry storm hits. Load shedding drops work early and cheaply, before it consumes resources: reject requests that have already waited longer than the client's timeout (nobody is listening for the answer), and drop low priority traffic first. Checkout before recommendations; paying users before free tier; real users before health checks from a partner's monitoring.

In the interview#

Checkpoint#