LearnHLDMessage queues

Message queues and delivery semantics

A queue between two services buys you three things: the sender stops waiting, a burst gets absorbed instead of dropped, and the receiver can fail without losing work.

It also costs you three things that people discover later: messages arrive more than once, they arrive out of order, and a failure that used to be a 500 the caller could see is now a silent backlog nobody is looking at.

Your answer

Your consumer processes a message, then crashes before acknowledging it. What happens next, and what does that mean your handler must be?

Queue or log

These are different data structures that both get called a queue, and the difference decides what you can build.

Shape
One message goes to exactly one worker, and once acknowledged it is gone. Add workers and throughput rises, with no coordination. There is no history, so a second consumer that wants the same messages cannot have them, and a bug that ate a message leaves nothing to replay. SQS and RabbitMQ work this way.

Partitions decide your parallelism

In a log, ordering is guaranteed within a partition and nowhere else. That single fact drives most of the design decisions people get wrong.

Consumer group
part 0
lag 2consumer 0
part 1
lag 1consumer 1
part 2
lag 3consumer 2
part 3
lag 2consumer 0
part 4
lag 2consumer 1
part 5
lag 1consumer 2
Six partitions, three consumers, two partitions each. Lag is low and even. This is the state you design for, and adding a fourth consumer would help because there are still spare partitions.
Ordering costs you parallelism, always

Every requirement of the form “these events must be processed in order” is a requirement that they share a partition, and a partition is served by one consumer. Choose the narrowest key that satisfies the requirement. Per user is usually fine. Per account is often fine. Globally ordered means one partition and one consumer, which is a queue with extra steps.

Delivery semantics

Three phrases get thrown around, and only two of them are real.

At most once. Acknowledge before processing. If you crash in between, the message is gone. Almost never what you want, but correct for something like a metrics sample where a duplicate is worse than a gap.

At least once. Acknowledge after processing. If you crash in between, the broker redelivers and you process it twice. This is what essentially every system gives you, and it is the one to design for.

Exactly once is not a delivery guarantee, because the network cannot provide one. What exists is at least once delivery plus idempotent processing, which produces exactly once effects. Some systems automate this within their own boundary with transactional offsets, but the moment your handler calls an external API, you are back to making the handler idempotent yourself. Saying this plainly in an interview is a strong signal.

1/6 The broker hands over message 41. It stays on the broker, unacknowledged, until the consumer confirms it is done.

The failures worth naming

The poison message. One malformed message throws every time. At least once delivery means it is redelivered forever, and it blocks its partition. Count attempts, and after a few, move it to a dead letter queue and carry on. A dead letter queue nobody alerts on is a folder where data goes to be forgotten, so alert on it being non empty.

Backpressure. Producers are faster than consumers, the backlog grows, and nothing errors. The system looks healthy right up until retention expires and messages are dropped silently. Consumer lag is the metric that matters here, not queue depth and not error rate.

The retry storm. A downstream service is struggling, every message fails, every failure retries immediately, and the retries triple the load on the thing that was already struggling. Exponential backoff with jitter, and a circuit breaker so you stop calling a dependency you already know is down.

Rebalance thrash. A consumer takes too long between polls, the group decides it is dead, partitions move, the work is redone, and the extra load makes the next consumer slow too. Keep processing off the poll thread, or raise the timeout deliberately rather than by accident.

Where a queue belongs

Figure 1. The write the user waits for is small and synchronous. Everything that can happen a second later happens behind the log.

Figure 1 hides a real problem. Steps 1 and 2 are two separate systems, so a crash between them either commits a row nobody hears about or publishes an event for a row that does not exist. The usual fix is the outbox pattern: write the event into a table in the same transaction as the row, and have a separate process ship rows from that table to the log. One commit, no gap.

Break it

Consumer health
steady
steadydownstream slowretry stormbacklog past retentionprotected
Healthy. Consumers keep up, lag is a second or two, and the log is doing what it is for: absorbing jitter so nobody has to wait for anybody.

Trade-offs

ChoiceWhat you gainWhat you payPick it when
Event log over work queueMany independent consumers on one stream, and replay when a consumer had a bug.Ordering and parallelism are tied to partition count, which you choose up front.Several teams need the same events, or you want to reprocess history.
At least once with idempotent handlersNo message is ever lost, and duplicates are harmless.Every handler needs a natural key and a uniqueness check, which is real work per handler.The default. Assume duplicates, because you will get them.
Partition by user idOrdering per user, which is the only ordering most products actually need, with full parallelism across users.A single very active user becomes a hot partition that more consumers cannot fix.Most event streams. Narrow the key until ordering still holds and no further.
Outbox patternRemoves the gap where a row is committed but its event never published, or the reverse.A table, a relay process, and events arriving a moment later than the commit.Whenever a database write and a published event must both happen or neither.

Interview replay

Interviewer
Your consumer crashes after doing the work but before acknowledging. What happens?
The question that decides whether you understand delivery semantics or have memorised the phrases.
You
The broker redelivers, because from its side an unacknowledged message is unprocessed. So the handler runs twice, which means it has to be idempotent. I would derive a key from the message, usually an event id, and make the write conditional on it, so the second attempt collides and returns success without repeating the side effect.
Goes straight from the mechanism to the requirement it places on the handler.
Interviewer
Could you use exactly once delivery instead?
A trap. The phrase is on every broker’s marketing page.
You
Not really. Exactly once delivery is not achievable over a network, because the acknowledgement can always be the thing that gets lost. What you can have is at least once delivery with idempotent processing, which gives exactly once effects. Some brokers automate that within their own boundary using transactional offsets, but as soon as my handler calls an external service, the guarantee ends and I am back to doing it myself.
Corrects the premise without being smug and names exactly where the boundary is.
Interviewer
How many partitions?
Checking whether you know what the number controls.
You
More than my expected consumer count, because a partition only ever goes to one consumer in a group, so partitions are a hard ceiling on parallelism. I would overprovision, since raising it later changes which key lands where and breaks per key ordering across the change. And I would pick the partition key as the narrowest thing that still gives the ordering we need, usually per user rather than global.
Three sentences, three separate consequences, all of them practical.
Interviewer
How would you know the queue is unhealthy?
Open. A chance to name the failure that produces no errors.
You
Consumer lag, alerted well below the retention window. That is the one that matters, because the dangerous failure here is silent: consumers falling behind produce no errors at all, and if the backlog passes retention the messages are deleted unread. I would also alert on the dead letter queue being non empty, since a DLQ nobody looks at is just a place data goes to disappear.
Picks the metric that catches the silent failure and explains why error rate would not.

Checkpoint

Checkpoint

1. You have 6 partitions and 8 consumers in one group. What happens?

2. Why is "exactly once delivery" the wrong thing to promise?

3. Consumers are falling behind and lag has been growing for two hours. No errors are being reported. What is the risk?

Say this in 60 seconds

I would use an event log rather than a work queue when more than one consumer needs the same events or I might want to replay them, and a plain work queue when it really is just background jobs. Ordering only holds within a partition, so I would partition by the narrowest key that still gives the ordering we need, usually per user, and pick a partition count above my expected consumer count because partitions are a hard ceiling on parallelism. I would assume at least once delivery and make every handler idempotent with a key derived from the message, since exactly once delivery is not a thing you can have over a network, only exactly once effects. To publish an event and commit a row together I would use an outbox table so there is no gap between them. And the metric I would alert on is consumer lag, well below the retention window, because falling behind produces no errors at all until messages start being deleted unread.

IndGeek provides solutions in the software field, and is a hub for ultimate Tech Knowledge.