9 min read

Queue vs pub/sub vs event stream

Published 18 October 2026

All three let one part of a system hand messages to another without waiting for it. They differ in who gets each message and how long it is kept.

A queue is a stack of orders at a restaurant pass. Each ticket is picked up by one cook, and once cooked it is gone.
Pub/sub is a WhatsApp group. Each message goes to every member who is in the group right now.
An event stream is the restaurant's order book. Every order is written down in sequence and kept. The kitchen reads it, the accountant reads it at night, and a new manager can read last month's from page one.

Your answer

When an order is placed on Swiggy, the restaurant must be notified, an invoice emailed, loyalty points added, and the analytics team wants every order since 2023 to train a model. Would one queue do? What would you use for each?

Who gets the message

Competing consumers. Each message goes to exactly one worker, and is deleted once acked. Add workers to go faster. This is how you spread work: sending emails, resizing images, generating invoices.

Queue#

The point of a queue is distributing work. Producers drop jobs in; a pool of workers pulls them; each job is done once (at least once, strictly; see delivery guarantees) and removed. Queues absorb bursts (the queue grows, workers catch up), let you scale workers independently, and retry failures with a dead letter queue for messages that keep failing.

Amazon SQS, RabbitMQ queues, Celery and Sidekiq on Redis, and Postgres tables with SKIP LOCKED all play this role. Ordering is usually not guaranteed across workers (SQS FIFO queues and RabbitMQ single consumers are exceptions).

Pub/sub#

The point of pub/sub is decoupling one event from many reactions. The order service publishes "order placed" once and does not know or care that four services react to it. Delivery is push style to whoever is subscribed at that moment.

Amazon SNS, Redis pub/sub, Google Cloud Pub/Sub, RabbitMQ fanout exchanges and NATS core are pub/sub. Note the difference in durability: Redis pub/sub keeps nothing, so a subscriber that is down misses messages, while Google Pub/Sub keeps each subscription's backlog for days. A very common AWS pattern combines them: SNS topic fanned out to one SQS queue per subscriber, so each service gets every event and also gets its own durable work queue.

Event stream#

The point of a stream is a durable, ordered history that many readers consume at their own pace. Kafka, Amazon Kinesis, Redpanda and Apache Pulsar work this way. Three ideas carry it:

Retention, not deletion. Messages stay for a configured time (days, or forever with compaction) whether or not anyone read them.
Offsets. Each consumer group tracks its own position. A new analytics job can start at offset 0 and replay a year of orders; a bug fix can rewind a consumer and reprocess.
Partitions. A topic is split into partitions, each an ordered log. Ordering is only guaranteed within a partition, so you choose a key: every event for order 8812 goes to the same partition and stays in order. Partitions are also the unit of parallelism: a group can have at most one active consumer per partition.

Streams cost more to run, and you pay for retention in disk. They are the backbone of event driven architectures, change data capture, and the "log as the source of truth" idea.

Choice	What you gain	What you pay	Pick it when
Queue (SQS, RabbitMQ, Celery)	Simple work distribution, per message ack and retry, dead letter queues, easy to scale workers.	One consumer per message, no replay, usually no ordering.	Background jobs: emails, thumbnails, PDF invoices, webhooks to send.
Pub/sub (SNS, Redis, Google Pub/Sub)	One event, many independent reactions, publisher knows nothing about subscribers.	Often no history for late or offline subscribers (varies by product).	Notifications between services, cache invalidation, real time fan out.
Event stream (Kafka, Kinesis)	Retention and replay, per key ordering, very high throughput, many groups at their own pace.	Operational weight, partition planning, consumers must track offsets and handle rebalances.	Event driven backbones, analytics pipelines, change data capture, audit history.

Kafka is not a job queue

Because a partition is consumed by one consumer in a group, one slow message (a 30 second PDF render) blocks everything behind it in that partition, and you cannot ack messages individually or delay a retry of just one. For heterogeneous background jobs with retries, a real queue is the better tool. Kafka shines when messages are many, small, ordered and read by several consumers.

Names you will hear

Amazon SQSManaged queue, at least once, FIFO option
RabbitMQBroker with queues and exchanges, flexible routing
Apache KafkaPartitioned, replicated log with retention
RedisPub/sub (no history), Streams, and job queues
Google CloudGoogle Cloud Pub/Sub, durable per subscription
AWSSNS for fan out, Kinesis for streams

In the interview#

Interviewer

Kafka or RabbitMQ for your notification system?

They want the deciding factors, not a favourite.

You

It depends on what the messages are. If "order placed" events need to reach several services and we want to replay them, Kafka: a topic keyed by order id, one consumer group per service. The actual sending of notifications, where each job may be slow, needs per message retries and rate limits per provider, fits a queue better, RabbitMQ or SQS with a dead letter queue. A common design is both: Kafka for the event backbone, and the notification service consuming it and enqueuing send jobs into its own queue.

Splits the problem into fan out of events and execution of jobs.

Interviewer

How do you keep events for one order in order?

You

Use the order id as the partition key, so all events for that order land in one partition, which Kafka keeps in order, and make sure a consumer processes a partition sequentially. Across different orders there is no ordering, and there does not need to be.

Queue, pub/sub or stream?

Resize every uploaded photo into four sizes

Invalidate a product's cache on 40 app servers when its price changes

Rebuild a search index from every product change since launch

An order event that 5 services react to, each with its own retries

Checkpoint#

Checkpoint

1. What is the key difference between a queue and pub/sub?

2. In Kafka, what does a partition guarantee?

3. A new analytics team needs every order event from the last 30 days. Which design makes that easy?

Worth memorising

Queue: one consumer per message. Pub/sub: every subscriber. Stream: kept, replayable, offsets.
Streams: partitions give ordering per key and parallelism per partition.
SNS to SQS is the classic fan out with durable per service queues.
Kafka is not a job queue for slow, heterogeneous jobs.
Dead letter queues catch messages that keep failing.

Say this in 60 seconds

A queue distributes work: each message goes to one of many competing workers and is removed once acknowledged, which suits background jobs like emails and image processing, with retries and dead letter queues; SQS, RabbitMQ and Celery are examples. Pub/sub decouples one event from many reactions: every subscriber gets its own copy and the publisher does not know who is listening; SNS, Redis pub/sub and Google Pub/Sub, with durability varying by product. An event stream like Kafka or Kinesis is an append only, partitioned log with retention: consumer groups each track their own offset, so many systems read at their own pace and can replay history, ordering holds within a partition so I key related events together, and partitions bound parallelism. In practice I often combine them: Kafka as the event backbone, SNS fanned out to SQS for durable per service delivery, and a proper queue for slow jobs with retries.

Next: OLTP vs OLAP Related: Message queues (system design path)