Design an ad click aggregator

Counting is the easiest thing a computer does, right up to the moment the count is money. Then a duplicate is fraud, a dropped event is a refund, and an advertiser who cannot reconcile your number against theirs stops spending.

Every time someone taps an ad, the advertiser owes money. An ad click aggregator is the system that counts those taps, adds them up per ad and per minute, and hands the totals to two readers: a live dashboard the advertiser watches, and the invoice they pay at the end of the month. Watch one click go through it.

A few words come up on every line of this page:

• An event is one thing that happened: a click, or an impression (the ad was shown).
• A window is a bucket of time, here one minute. Every click that happened between 14:59:00 and 14:59:59 belongs to the 14:59 window.
• To aggregate is to turn many events into one number: 1,204 clicks on ad 88231 in the 14:59 window, stored as one row.
• Stream processing counts events as they arrive, a little at a time. Batch processing waits, then counts a whole hour or day in one go.

This is a staff level question because it has no comfortable answer. Streaming gives you fast numbers that are occasionally wrong. Batch gives you correct numbers hours late. The design is how you offer both without keeping two versions of the truth.

Step 1: Understand the problem#

The word "count" hides four different requirements. Separate them before you draw, because each one pulls the design somewhere else.

What you are building#

Back of the envelope#

The trick that makes this system affordable is in the last two rows. (New to estimates like this? The back of the envelope chapter shows the method.)

A columnar store keeps each column's values together on disk, so summing clicks across millions of rows reads only the clicks column. That is the opposite of an app database, which keeps each row together. OLTP vs OLAP shows the difference live.

Step 2: Propose the high level design#

The API#

Three endpoints. The interesting part of each is what it refuses to do.

{
  "eventId": "c-8f21ab...",
  "type": "click",
  "adId": 88231,
  "eventTime": "2026-08-25T14:59:58.221Z"
}

{ "from": "2026-08-25T14:00Z", "to": "2026-08-25T16:00Z" }

The data model#

One row per ad per minute. Two columns need a word first. Micros are millionths of the currency unit, so 3,612,000 micros is 3.612. Money is stored as a whole number of micros because floating point numbers cannot hold most decimals exactly: in JavaScript, 0.1 + 0.2 is 0.30000000000000004, and those errors add up across a billion clicks. Finalised says whether a row can still change.

The whole system on one whiteboard#

Walking Figure 1:

1. The click is a redirect through your ad server, which is what makes the event yours to count rather than something you have to be told about.
2. The ad server appends to the log and returns immediately. It never waits for anything downstream, and it never aggregates. The log is an event stream: it keeps events for a set retention period, so many readers can each read it at their own pace and replay it later (event streams explains the idea).
3. Events are deduplicated by the click id minted at step 1, before anything counts them.
4. The stream processor groups them into one minute windows by event time.
5. Those windows land in the serving store within about a minute of the click.
6. In parallel, everything on the log is archived raw to object storage, a cheap store for huge files (design object storage). This archive, not the serving store, is the source of truth.
7. An hourly batch job reads the archive and computes the same windows exactly, with all the late data that has arrived since.
8. It overwrites the rows the stream already wrote. Same table, same rows, better number.
9. Reconciliation compares your totals against what the advertiser's own system saw, because the disagreement is the thing that actually costs money.
10. The dashboard and the invoice read the same table. A row starts out approximate and becomes final, and a flag says which.

Edge 8 is the design. The batch job does not write to a different table for a different audience. It overwrites the same rows the stream produced, so a number on the dashboard gets more accurate over the following hours and then stops changing. There is one table and one truth, with a finalised flag saying which rows are safe to bill against.

Step 3: Design deep dive#

Event time is not arrival time#

The click happened at 14:59 on a phone with a poor connection. It reached you at 15:04. Which minute does it belong to?

Two clocks are in play. Event time is when the click happened, stamped by the ad server. Arrival time is when it reached your pipeline. Think of a postmark: a letter stamped on Monday was sent on Monday, even if it is delivered on Wednesday.

Choosing event time creates the problem the rest of this section is about: you never know when a window is complete, because there might always be one more straggler. The widget below is the middle rung, so you can feel why it fails.

Each line joins when a click happened (top) to when it arrived (bottom). Drag the slider from zero to the end. At zero you publish the moment the window closes and lose three events. At thirty you lose none and every number is thirty seconds older. There is no setting that gives both, and finding one is not an engineering problem you have failed to solve.

So do not choose. That is the third rung: emit early with what you have, keep the window open, and emit again if stragglers arrive. The dashboard shows the early number and corrects itself. Billing waits for the batch job, which reads the archive long after every straggler has landed.

Counting each click exactly once#

Every layer here delivers at least once: when in doubt it sends again, so nothing is lost but things can repeat. The ad server retries on a timeout, the log redelivers on a consumer restart, and the stream processor reprocesses a window after a crash. Without care, a single click becomes three. (Why no layer can promise "exactly once" delivery is explained in delivery guarantees.)

Think of a cashier counting the till at closing. If they lose their place and start again, "add each note to the running total" counts some notes twice. "Count the drawer and write down the total" gives the same answer however many times they start over.

Try all three. Send a click, have the ad server retry it, then crash the processor.

Here is the same thing as a sequence of messages.

Three ideas are doing all the work there, and they are worth naming separately because each one solves a different failure:

Event ids created at the edge. Anything generated later cannot tell a retry from a second real click.

Deduplication scoped to the window. Remembering every id ever seen is not affordable. Remembering ids within a window is, and a duplicate arriving after its window has closed is rare enough to be handled by the batch job. How long a dedupe memory has to last is its own question, worked through in exactly once in practice.

Writes that set rather than increment. An operation is idempotent when doing it twice has the same effect as doing it once (idempotency keys has more examples). An increment is not idempotent, so reprocessing doubles it. Writing the computed total for a window keyed by that window makes reprocessing free, which in turn makes crash recovery boring.

Two pipelines, one number#

You now have a fast path that is nearly right and a slow path that is exactly right. How do they become one number? There are two well known names for the answer. Lambda runs a stream pipeline and a separate batch pipeline and merges them when someone reads. Kappa runs only the stream pipeline and fixes history by replaying the log through it. Switch between them, then look at the third option, which is what this page proposes.

Partitioning the stream#

A million events a second needs many processors. The log is split into partitions, and each partition is read by exactly one processor, so the partition key decides which machine sees which events (partitions decide your parallelism).

Here are the three situations side by side. Salting is also used to spread view counts on Design YouTube.

Break it#

Raise the traffic and watch where the pressure shows up first. It is not where most people expect.

Trade-offs#

The level bar#

One moment from this round, answered three ways. Pick the one you think is staff level before the labels appear. (What changes at senior and staff explains the gap.)

Then run the question cold, with a 45 minute timer and no notes, and score the run on the five things this question is really testing.

Interview replay#

Checkpoint#