Library
7 min read

At most once vs at least once vs exactly once

Published

A queue hands a message to a consumer. The consumer does some work and tells the queue "done" (an ack), and the queue deletes the message. Everything interesting happens when the consumer crashes, or the network drops, somewhere in that sequence. The three delivery guarantees are just names for what you get when that happens:

  • At most once: a message is never processed twice, but may be lost.
  • At least once: a message is never lost, but may be processed more than once.
  • Exactly once: never lost, never repeated. Everyone wants this one, and over a network, as a delivery guarantee, nobody can have it.
Your answer

A consumer takes 'credit ₹100 cashback' off a queue, credits the wallet, and then crashes before it can ack. The queue redelivers the message to another consumer. What happens to the user's wallet, and how would you stop it?

Where the ack goes decides everything#

Credit ₹100 cashback from a queue message
  1. 1Deliver message m-81
  2. 2Credit ₹100 to user 42
  3. 3Consumer crashes
  4. 4No ack within the visibility timeout, so the queue hands it out again
  5. 5Deliver message m-81 again (delivery 2)
  6. 6Credit ₹100 to user 42 (again)
  7. 7Ack: queue deletes m-81
Paid 2 times
At least once. Nothing is ever lost, but a crash between the work and the ack means the work runs again. This is the default for most queues (SQS, RabbitMQ, Kafka consumers), so every consumer must expect duplicates.

Ack before processing gives at most once. Once acked, the queue forgets the message; a crash before the work is done loses it. Acceptable for data where losing a little is fine: metrics samples, presence pings, debug logs.

Ack after processing gives at least once. A crash before the ack means the queue never heard "done", so after a visibility timeout (SQS) or a consumer group rebalance (Kafka) it delivers the message again. Nothing is lost, and the work may run twice. This is the default behaviour of SQS, RabbitMQ with manual acks, Kafka consumers committing offsets after processing, Google Pub/Sub, and webhook senders. It is the one you will almost always use.

Why exactly once delivery cannot exist#

The consumer does the work, then sends the ack. The ack can be lost. The queue cannot tell "the consumer crashed before the work" from "the consumer did the work and the ack got lost": both look like silence. It must either redeliver (risking a repeat) or not (risking a loss). No protocol escapes this, which is the same reasoning as the two generals problem.

What you can have is an exactly once effect: at least once delivery, plus a consumer that recognises repeats and does nothing the second time. Two ways:

  • Idempotent operations. "Set order 8812 to shipped" is harmless to repeat. Design messages as states, not increments, where possible.
  • Deduplication. Store each processed message id in the same transaction as the effect. A repeat finds its id already there and is skipped. If the dedupe record and the effect are not atomic, a crash between them reopens the gap.

The other half: publishing#

Duplicates and losses also happen on the way into the queue. A classic bug:

ts
await db.query('INSERT INTO orders ...')   // 1. save the order
await kafka.send('order-placed', order)    // 2. tell everyone

Crash between the two lines and the order exists but nobody hears about it. Swap them and you can announce an order that was never saved. You cannot make a database and a broker commit atomically together. The standard fix is the transactional outbox:

1/4 One local transaction. Either both rows exist or neither does.

Kafka adds two features worth knowing by name. The idempotent producer (on by default since Kafka 3.0) attaches sequence numbers so a broker drops duplicates caused by producer retries. Transactions let a consume, transform and produce step commit its output and its input offsets atomically, which Kafka calls exactly once semantics. That guarantee covers data moving within Kafka. The moment your consumer charges a card, sends an email or writes to another database, you are back to idempotent effects.

ChoiceWhat you gainWhat you payPick it when
At most onceSimplest, never duplicates, lowest latency.Messages vanish when consumers crash.Telemetry, metrics, presence, anything where a gap is harmless.
At least onceNever loses a message. The default almost everywhere.Duplicates on crash or timeout, so consumers must be idempotent.Nearly all business events: orders, payments, notifications, emails.
Effectively once (at least once + dedupe)Exactly once effect on your own data.A dedupe store, transactions around effect plus record, retention policy.Money, inventory, anything where a repeat is visible to a customer.
Names you will hear
  • Amazon SQSAt least once, visibility timeout; FIFO queues dedupe for 5 minutes
  • Apache KafkaIdempotent producer, transactions, offsets
  • RabbitMQManual acks, redelivery on channel close
  • DeDebeziumChange data capture, a common outbox relay

In the interview#

Interviewer
We use Kafka with exactly once enabled. So our consumers don't need to handle duplicates, right?
A common misunderstanding offered as bait.
You
Not quite. Kafka's exactly once covers reading from a topic and writing to another topic atomically with the offsets. If the consumer's effect is outside Kafka, sending an SMS or updating Postgres, a crash after the effect but before the offset commit still causes a redelivery. So the consumer should be idempotent: I would store the event id in the same database transaction as the update and skip ids already seen.
Knows the scope of the guarantee, then gives the fix.
Interviewer
How does the order service publish events reliably?
You
Transactional outbox: write the order and an outbox row in one transaction, and a relay, either a poller or Debezium on the write ahead log, publishes outbox rows to Kafka and marks them sent. It is at least once, so events carry ids consumers dedupe on.
Which guarantee does this give?
Auto commit Kafka offsets every 5 seconds while processing in the background
SQS: delete the message after the handler succeeds
UDP telemetry packets with no acks
Consumer writes the effect and the message id in one transaction, then acks

Checkpoint#

Checkpoint

1. A consumer credits a wallet, then crashes before acking. With ack after processing, what happens?

2. Why is exactly once delivery impossible over a network?

3. What problem does the transactional outbox solve?

Worth memorising
  • Ack before work: at most once. Ack after work: at least once.
  • Exactly once delivery is impossible. Exactly once effect is not.
  • Effect + processed message id in one transaction.
  • Transactional outbox for publishing. Debezium or a poller as the relay.
  • Kafka exactly once covers Kafka to Kafka. External side effects still need idempotency.
Say this in 60 seconds

Delivery guarantees describe what a crash does to a message. If the consumer acks before processing, a crash loses the message: at most once, fine for telemetry. If it acks after processing, a crash before the ack causes redelivery: at least once, which is the default for SQS, RabbitMQ, Kafka consumers and webhooks, and means consumers will see duplicates. Exactly once delivery is impossible because a lost ack and a crash look the same to the sender. What you build instead is an exactly once effect: at least once delivery plus idempotent consumers, either by expressing messages as state changes or by storing each processed message id in the same transaction as the effect. On the publishing side, the transactional outbox writes the event alongside the data in one transaction, and a relay like Debezium publishes it. Kafka's exactly once semantics cover topic to topic processing, so consumers with external side effects still dedupe.