Push vs pull
Published
A newspaper delivered to your door every morning is push: the producer decides when it arrives. Walking to the stall when you feel like reading is pull: you decide. Push gets you the news sooner. Pull means you never come home to a pile of papers you could not get through.
The same choice appears all over system design, under different names: fan out on write or fan out on read, push or pull consumers, push or pull metrics, push or pull CDNs, CI push or GitOps pull deploys. It is always the same trade: who starts the transfer, and therefore who controls the rate.
A cricketer with 250 million followers posts on Instagram. If the post is pushed into every follower's feed the moment it is published, how many writes is that? What would you do instead for accounts like hers?
The trade in one table#
| Push | Pull | |
|---|---|---|
| Who starts | The producer, when data exists | The consumer, when it is ready |
| Latency | Lowest: data moves immediately | Up to one polling interval, unless long polling |
| Who controls the rate | The producer. A slow consumer can be overwhelmed | The consumer. It takes what it can handle |
| Wasted work | Work for consumers that never look | Empty polls when nothing changed |
| Failure handling | Producer must track who got what and retry | Consumer just asks again from where it left off |
| Producer knows consumers? | Usually yes | No, consumers find the producer |
Now the same trade in five places.
1. News feeds: fan out on write vs on read#
For a normal account, push is cheap and makes every follower's feed open instant. Drag the slider to the far right.
Real feeds are hybrid: push for the long tail of normal accounts, pull for the few accounts with huge followings, merged when the feed is read. That design is worked through in design a news feed.
2. Message consumers#
RabbitMQ pushes messages to consumers, limited by a prefetch count so a consumer is not flooded. Kafka consumers pull: each one asks for the next batch from its own offset when it is ready. Pull lets a consumer that fell behind catch up at its own speed in big efficient batches, and lets a new consumer start from any point in history. Push gives slightly lower latency for each message.
3. Metrics#
Prometheus pulls (scrapes) a /metrics endpoint on every target every 15 seconds or so.
The monitoring system controls the load, and a target that stops answering is immediately
visible as down. StatsD, Datadog agents and OpenTelemetry's OTLP push from the
application. Push suits short lived jobs (a batch job that ends before a scrape would find it,
which is what Prometheus's Pushgateway exists for) and apps behind NAT that cannot be scraped.
4. CDNs#
A pull CDN fetches a file from your origin the first time someone asks for it, then caches it. Zero setup, and the first request in each region is slower. A push CDN has you upload files ahead of time. Better for very large files (game patches, video) that you know will be requested, worse for constantly changing content. Most web traffic uses pull.
5. Deployments#
Classic CI/CD pushes: the pipeline holds production credentials and runs kubectl apply.
GitOps pulls: an agent inside the cluster (Argo CD, Flux) watches a Git repository and
pulls changes in. The cluster's credentials never leave the cluster, and drift from what Git
says is detected and corrected automatically.
| Choice | What you gain | What you pay | Pick it when |
|---|---|---|---|
| Push | Lowest latency. Producer can precompute work once for many consumers. | Producer must know consumers, track delivery and retry; slow consumers get flooded without backpressure. | Small fan outs, real time notifications, precomputed feeds for normal accounts, webhooks. |
| Pull | Consumer controls its own rate, batches efficiently, restarts from where it left off, producer stays simple. | Latency up to the poll interval, wasted polls, reader does the work at read time. | Stream consumers, metrics scraping, huge fan outs, GitOps, anything where consumers vary in speed. |
- PrometheusPull based metrics scraping
- DatadogAgent pushes metrics and traces
- Apache KafkaConsumers pull batches at their own pace
- RabbitMQBroker pushes with a prefetch limit
- ArgoArgo CD: GitOps, the cluster pulls deploys
- CloudflarePull CDN by default
In the interview#
Checkpoint#
1. What is the main thing pull gives the consumer?
2. Why do feeds switch to pull for celebrity accounts?
3. What does GitOps (pull based deployment) improve over CI pushing to the cluster?
- Push: producer starts, lowest latency, producer controls rate.
- Pull: consumer starts, consumer controls rate, simpler recovery.
- Feeds: push for normal accounts, pull for celebrities, merge at read.
- Prometheus pulls, StatsD and OTLP push. Kafka consumers pull.
- GitOps is pull: the cluster fetches its own deploys.
Push and pull answer who starts a transfer, and therefore who controls the rate. Push sends data the moment it exists, giving the lowest latency and letting the producer precompute work once, but the producer must track consumers and retries and can flood a slow consumer. Pull lets the consumer ask when ready, so it controls its pace, batches efficiently and recovers by asking again, at the cost of latency and empty polls. The same trade appears everywhere: feeds fan out on write for normal accounts and on read for celebrities, merged at read time; RabbitMQ pushes with a prefetch limit while Kafka consumers pull from offsets; Prometheus scrapes while StatsD and OTLP push; most CDNs pull from the origin on first request; and GitOps tools like Argo CD pull deployments into the cluster so credentials never leave it.