Someone scans a QR code at a tea stall, taps a PIN, and twelve rupees moves between two
accounts at two different banks in about two seconds. Both banks are separate companies
with separate databases and separate outage schedules. Neither of them can roll back the
other one.
Pay. The payer app sends one pay request, carrying an idempotency key it made itself. Everything after this is keyed off that.
Three words come up on every line of this page, so here they are in plain terms:
The switch is the central router in the middle. It passes instructions between the
banks and keeps track of where each payment is. It never holds anyone's money.
A leg is one half of a payment: the debit (money out of the payer's account) or the
credit (money into the payee's account). Every payment here has two legs, at two banks.
An idempotency key is a unique tag the payer's app puts on a request, so that if the
same request arrives twice, it is only acted on once.
Idempotency keys has a live demo of the idea.
Think of it like handing a parcel between two courier companies at a border. Each company
keeps its own records. Neither can reach into the other's office and undo what was signed
for. If the second courier drops the parcel, the first one cannot pretend it was never
handed over. Someone has to send a new parcel back. That is the whole problem, and
everything else in this design follows from it.
bank Abank Bdroppeda new parcel backno rollback across the border, only a second movement
This page uses UPI as the reference because it is the largest system of its kind and
because most people reading this have used it three times today. The design generalises:
any rail that moves money between institutions you do not control ends up in roughly the
same shape.
Payments interviews reward paranoia. Every question below is really asking "what happens
when only half of this succeeds", and the answers stop you from designing a distributed
transaction that cannot exist. (A distributed transaction is one change that has to land
in several databases at once, all or nothing.
Distributed transactions covers it from the start.)
You ask
They say
What it settles
Who owns the accounts, us or someone else?
Banks own them. We move instructions, not money.
The single most important answer. You cannot roll back another company's database, so every failure is undone with a compensating payment rather than an abort.
What is the peak rate, and when does it happen?
Tens of thousands a second, spiking hard on salary day and festival evenings.
Capacity planned for the spike, not the average, and a queue in front of anything that talks to a bank, because banks do not scale on your schedule.
Is it acceptable to debit twice under retry?
Absolutely not. Ever.
An idempotency key on every leg, generated once by the payer app and carried end to end. Retries are certain, so exactly once has to come from the key rather than from hope.
What if a bank does not answer at all?
Treat it as unknown, not as failed.
A third outcome besides success and failure. Timeouts get resolved by asking the bank later, and never by guessing, which is the subject of a whole deep dive.
How fast does the payer need an answer?
Under a few seconds, or they pay in cash instead.
A synchronous request path with hard deadlines, and everything that can be done afterwards being done afterwards.
Who decides what actually happened, in a dispute?
The end of day settlement file. Not our database.
Reconciliation as a first class part of the design rather than an operations afterthought. Your ledger and theirs will disagree, daily, and the process for that is the product.
Published monthly volumes for UPI have been well north of 15 billion transactions a month,
and the figure keeps climbing, so treat any specific number as a checkpoint rather than a
constant. What matters is the shape of the arithmetic. (New to estimates like this?
Back of the envelope walks through the method.)
A worked example first, with small numbers. Say a rail handles 2.6 million payments in a
month. A month is about 2.6 million seconds (30 days x 86,400), so that is one payment a
second on average. If the busiest day is 2x an average day and the busiest hour of that day
is 3x an average hour, the peak is 1 x 2 x 3 = 6 payments a second. The slider below does
the same sum with real sized numbers.
Peak throughput
18 billion
2.5x
3x
Average per second18B / 2.6M seconds = 6,944 tps
Peak per second6,944 x 2.5 x 3 = 52,083 tps
Ledger writes at peak4 per transaction = 208,333 per second
Ledger rows per month18B x 4 = 72 billion
52,083 tps
peak, and every one of them moves money in two legs at two banks
Four ledger writes per transaction, because each of the two legs is a double entry pair. That turns 52,083 payments a second into 208,333 appends a second, and it is the number that decides the ledger design.
The multipliers are not decoration. Payment traffic in India is spiky in ways an average
hides completely: salary day, the first of the month, festival evenings, the last hour
before a bill deadline. Designing for the average is designing to fail on the days that
matter most.
1/saverage2/sbusiest day, x26/sbusiest hour, x3design for the peak, not the averagesalary day, festival evenings, the hour before a deadline