Two hundred thousand drivers are moving around a city, each one reporting where they are
every four seconds. A rider opens the app and wants the five nearest cars, then thirty
seconds later wants exactly one of them assigned to nobody else.
Ride hailing service / at a glance
Those are two completely different systems sharing a screen. One is a write heavy
geospatial index taking fifty thousand location updates a second. The other is a
matching problem where the same car must never be promised twice. Most candidates design
the first one and hand wave the second, and the second is where the interview is.
Ask about the matching before you ask about the map. The map is a solved problem with a
known answer, and the matching is where the real decisions are.
You ask
They say
What it settles
How many drivers are online, and how often do they report position?
A couple of hundred thousand in a large city, every four seconds.
Fifty thousand writes a second into whatever holds locations. That number alone rules out a relational table with a spatial index and pushes you to an in memory grid.
How fresh does a driver location have to be for matching?
Within a few seconds.
Location is soft state. It can live in memory, be lost on restart and rebuild itself in four seconds, which removes durability from the hottest write path in the system.
Can two riders be matched to the same driver?
Never.
A serialisation point per driver, and an offer that locks the driver while they decide. This is the hard requirement, and it is not solved by the geospatial index.
What happens if a driver ignores the request?
Move to the next one, within seconds.
Offers with a short timeout and automatic reassignment, so the matching state machine has a timer in it. It also means a rider is waiting while you try three drivers in sequence.
Does the price change while the rider is looking?
It is quoted up front and honoured.
A pricing service that reads supply and demand per area on a slow loop, and a quote that is frozen into the trip. Recomputing at accept time is how you get a support ticket.
Is a trip record allowed to be lost?
Absolutely not. It is money and it is a legal record.
Two different storage systems with two different guarantees. Locations are disposable, trips are durable and transactional, and confusing the two is the most common mistake in this design.
Location writes a day if you stored them4.3 billion
50,000 writes/sec
against a working set that fits in the memory of one laptop
Those two numbers together are the design. 24MB of live state means the whole city fits in memory, and 50,000 writes a second means you must never touch a disk to record one. The moment somebody suggests persisting every location update, ask what reads it.
The number people get wrong
Candidates size storage for location history and end up designing a time series
database. Nothing in the matching path ever reads a location older than a few seconds.
Keep the live position in memory, and if the business wants trip traces for analytics,
publish them to a log the matching system never queries. Two systems, two lifetimes.
Why: No response body, no acknowledgement of durability, and it should be a persistent connection rather than a fresh request every four seconds. This is the highest volume call in the system and every byte in the response is multiplied by fifty thousand a second.
Why: The quote id comes back with the list. Freezing the price here rather than at accept time is a product decision that removes an entire class of complaint, and it means the pricing read happens once per search rather than once per accept.
Why: 202, because no driver has agreed yet. Returning 200 with a driver would mean holding the request open while you offer the ride to three people in sequence, and the client would time out before the matching did.
POST/v1/offers/{id}/accept
returns 200 { tripId, rider, pickup } or 409 if it expired
Why: The 409 is the important response. Two drivers can tap accept on offers that were valid when they were sent, and exactly one of them can win. The loser needs a clean answer, not a trip they cannot see.
Nothing here is durable and that is deliberate. Lose the whole store and it rebuilds in four seconds from the next round of updates, which is a far better property than any replication scheme would give you.
tripsrelational, sharded by city, replicated
trip_id
uuid
PK
Generated at request time, before any driver exists.
rider_id
bigint
IDX
driver_id
bigint
IDX
Null until an offer is accepted. The unique constraint on active trips per driver is the last line of defence against double assignment.
state
enum
matching, assigned, arriving, in_progress, completed, cancelled. Transitions are guarded, not free.
quote_id
varchar(32)
The price the rider was shown. Copied into the trip so it cannot drift.
version
int
Optimistic lock. Every state change is a compare and set, which is how two concurrent accepts resolve.
Sample row
t_9f31 | 4471 | 88213 | in_progress | q_71a2 | 7
Trips are the money and the legal record, so they get a real database with real transactions. Putting trips in the same store as locations, to save a system, is the mistake this table exists to prevent.
location1overwrite2near me?3nearby4quote5request6candidates7create8offer, 15s9ETAon completionDriver appevery 4 secondsDriver gatewayholds connectionsGeo indexin memory, by cellOffer timerexpire, reassignRider appRider APIstatelessMatching serviceone owner per cellTrip storedurable, transactionalPaymentson completionPricingsupply vs demandMaps serviceroutes and ETAs
location1overwrite2near me?3nearby4quote5request6candidates7create8offer, 15s9ETAon completionDriver appevery 4 secondsDriver gatewayholds connectionsGeo indexin memory, by cellOffer timerexpire, reassignRider appRider APIstatelessMatching serviceone owner per cellTrip storedurable, transactionalPaymentson completionPricingsupply vs demandMaps serviceroutes and ETAs
Figure 1. The top half takes 50,000 writes a second and is allowed to lose everything. The bottom half takes a thousand requests a second and is not allowed to lose anything. Almost every decision here follows from that split.
Walking Figure 1:
Driver apps hold a persistent connection and stream position. Opening a fresh HTTPS
request every four seconds for 200,000 drivers is a self inflicted denial of service.
The gateway overwrites the driver's entry in the geo index. No append, no history, no
disk. If the driver moved into a new cell, that is a remove and an add.
A rider opens the app and asks who is nearby.
That search reads the same in memory index the writes go into, filtered to available
drivers.
The price is quoted once, here, and frozen into a quote id, so it cannot drift between
looking and accepting.
Requesting a ride hands the problem to the matching service, which returns immediately
with a trip in the matching state.
Matching reads candidate drivers from the index, ranked by ETA rather than by straight
line distance, because a river or a one way street makes those very different numbers.
The trip row is created before any driver is contacted. If everything crashes now,
there is a durable record saying a rider is waiting.
An offer goes to one driver with a short expiry, and a timer owns what happens if they
do not answer. That timer is the part people forget, and it is the difference between
a system and a demo.
Your answer
Step 9 offers the ride to one driver at a time with a 15 second timer. The alternative is broadcasting to the nearest five and letting them race. Which would you pick, and what does the other one cost?
Finding nearby drivers without a spatial database#
The obvious answer is a relational table with a spatial index and a bounding box query.
It works, and it falls over at fifty thousand writes a second, because every one of those
writes updates an R-tree and R-trees do not enjoy that.
The answer that scales is to give up on exact geometry and bucket the world.
Nearby lookup
encode123Rider pointCell tdr1vx~1.2km8 neighboursfixed setUnion, thensort by ETA
encode123Rider pointCell tdr1vx~1.2km8 neighboursfixed setUnion, thensort by ETA
Encode the point to a fixed precision string, then read that cell and its eight neighbours. Nine set reads from memory, no index to maintain, and a write is a set remove plus a set add. The cost is that cell size is fixed: in a dense market nine cells returns two thousand drivers, and in a rural one it returns nobody.
Start with geohash cells and say why: it is a hash map of sets, a write is two operations,
and it needs no rebalancing under load. Then name its weakness before the interviewer
does, which is that a fixed cell size is wrong somewhere in every city, and say that S2
is what you move to when that starts costing you matches.
What actually gets returned
Sorting candidates by straight line distance is the beginner version and it produces
visibly stupid matches: the closest car is across a river with a ten minute detour to
the bridge. Rank by ETA from the maps service, on the handful of candidates the cells
gave you. Nine cells might return two hundred drivers, you take the twenty closest, and
you ask for real ETAs only for those twenty. That is the shape of nearly every
geospatial ranking problem: cheap filter, expensive rank, small candidate set.
This is the part that separates the answers. Two riders, two matching workers, one driver
sitting between them, and nothing in the geo index stops both from picking him.
Matcher AMatcher BDriver stateDriver 88213candidates near pickupcandidates near pickupCAS 88213: available to offeredCAS 88213: available to offeredoffer, 15s to answeraccept
1/6 Matcher A is handling rider 4471 and gets a list of nearby drivers. Driver 88213 is at the top, ranked by ETA.
Figure 2. The offer is a lock with a timer on it. The second matcher does not fail because the driver is busy, it fails because a compare and set on the driver's state did not succeed.
Two details worth saying out loud. The lease has to expire on its own, so a matcher that
crashes between the compare and set and the offer does not strand a driver as
permanently offered. And the accept has to be idempotent, because a driver on a patchy
connection will tap it twice and both taps will arrive.
The follow up you will get
"Why not broadcast to the five nearest drivers and let the fastest one win?" It matches
faster and riders like it. It also means four drivers get a notification for a ride they
cannot have, which is the single most complained about behaviour in every driver app
that has tried it, and it turns acceptance into a race decided by phone hardware and
network latency. The usual compromise is sequential offers with a short timeout, and
broadcast only when sequential has already failed twice.
One matching service cannot own a whole country, and the obvious shard key is geography.
Assign each region to one matcher, and that matcher is the single serialisation point for
drivers inside it, which is what makes the compare and set cheap and local.
The problem is the boundary. A rider standing on the line between two regions has
candidates owned by two different matchers, and now the thing that made assignment simple
is gone.
The workable answer is that the matcher for the rider's region owns the trip, and it may
read candidates from neighbouring regions but must acquire the driver through the matcher
that owns that driver. Cross region matches are a small percentage of traffic and they
are allowed to be slower. What you must not do is let both matchers assign, which is the
double booking you spent the last section preventing.
Sharding by city rather than by grid is usually the honest answer, because cities are
natural boundaries. Almost nobody takes a ride from one city into another. Where they do,
that trip is a business event worth handling specially, not a hot path.
50k/scandidatesassignDriver gatewayGeo indexin memoryMatchingper regionTrip store
50k/scandidatesassignDriver gatewayGeo indexin memoryMatchingper regionTrip store
Healthy. 200,000 drivers, a thousand requests a second, most drivers idle. The geo index is doing 50,000 memory writes a second, which is nothing, and matching is mostly waiting.
Read every step as text
tuesday 3pm · Healthy. 200,000 drivers, a thousand requests a second, most drivers idle. The geo index is doing 50,000 memory writes a second, which is nothing, and matching is mostly waiting.
friday 6pm · Degraded. Demand triples and supply does not. Matching is now working hard, but the bottleneck is not CPU: it is that there are no available drivers in the cells near most riders, so candidate lists come back short and matchers walk further out.
stadium empties · Down. Twenty thousand people request a ride from four adjacent cells inside two minutes. Those cells are one hot partition of the geo index and one matcher owns the region, so a citywide system is bottlenecked on one process while the rest idle. This is a hot key problem wearing a map.
gateway restart · Degraded. A gateway deploy drops 60,000 driver connections. They reconnect over the next few seconds, and the geo index has stale entries for all of them until they do. Because entries expire after 30 seconds rather than being deleted, the worst case is briefly offering rides to drivers who are reconnecting.
repaired · Healthy. For the surge: split the hot region into smaller matcher shards dynamically, and queue riders with a visible wait rather than failing them. For the restart: stagger the deploy so no more than a few percent of connections drop at once, which is a deployment policy rather than an architecture change.
The stadium case is the one to talk about unprompted. Every geospatial system has it, the
map makes it look like a new problem, and it is the same hot partition you would get from
a celebrity account or a viral link.
A rider opens the app. Walk me through finding nearby drivers.
Warm up, and a check on whether you reach for PostGIS.
You
Driver positions live in memory, bucketed by geohash cell. The rider point encodes to a cell, we read that cell plus its eight neighbours, filter to available drivers, take the twenty closest by straight line, and only then ask the maps service for real ETAs on those twenty. Cheap filter, expensive rank, small candidate set. I would avoid a spatial index in a database here because we are doing fifty thousand position writes a second and that is not what R-trees are for.
Gives the shape and the reason against the obvious alternative, with the write number as the justification.
Interviewer
Two riders request at the same moment and the same driver is nearest to both.
The actual question.
You
Both matchers will see him, and that is fine. The assignment is a compare and set on the driver state from available to offered with a short lease. One wins and sends the offer, the other gets a failed CAS and moves to its next candidate. The lease has to expire on its own so a matcher crashing does not strand the driver, and the trip row carries a version so the accept is also a compare and set.
Does not try to prevent the race, resolves it. That is the senior answer.
Interviewer
Why not just broadcast to five drivers?
Testing whether you can argue a product trade, not only a technical one.
You
It matches faster and riders prefer it. The cost is that four drivers get buzzed for a ride they will not get, which drivers hate, and acceptance becomes a race decided by whose phone is faster. I would do sequential offers with a 15 second timeout by default and only broadcast after two failed offers, when the rider has already been waiting thirty seconds and the calculus changes.
Answers with a policy rather than a preference, including when the policy flips.
Interviewer
A concert ends and twenty thousand people request a ride from the same block.
The failure question. They want to see if you recognise the shape.
You
That is a hot partition. Those requests all land in a handful of cells owned by one matcher, so one process is saturated while the rest of the city idles. Adding matchers does not help unless you split that region, so I would support dynamic subdivision of a hot region into smaller shards. I would also queue riders with an honest visible wait rather than failing them, because supply is genuinely finite at that moment and a queue is a better experience than a retry loop.
Names it as a hot partition, rejects the non fix, and includes the product answer alongside the technical one.
Interviewer
How long should the offer timeout be?
Open ended, checking for a confident invention.
You
I do not know, and I would not guess it in the room. It is a trade between rider wait time and driver acceptance rate, and the right way to find it is to measure the distribution of how long drivers actually take to respond and cut somewhere in the tail, then A/B it. My instinct is somewhere between ten and twenty seconds, and I would expect it to differ by market and by time of day.
Gives a method and a range, then admits it is an instinct. Inventing "12 seconds because that is optimal" is the answer that loses points.
1. Why keep driver locations in memory rather than in a replicated database?
2. Two matchers pick the same driver from the index at the same instant. What prevents a double assignment?
3. Same system, but now it delivers food instead of people. What changes first?
Worth memorising
200,000 drivers reporting every 4 seconds is 50,000 writes a second.
The entire live location state is a few hundred megabytes. It fits in memory and it is disposable.
Nine cells cover a search: your own, plus the eight neighbours.
One driver, one rider, settled by a compare and set with an expiring lease. The loser just takes the next candidate.
Say this in 60 seconds
A ride hailing service is two systems that share a screen. Driver locations are 50,000 writes a second into an in memory index bucketed by geohash cell, with no durability at all, because nothing reads a position older than a few seconds and the whole state rebuilds itself in one reporting interval. Trips are the opposite: a real database with transactions, because that is money and a legal record. A nearby search reads nine cells, filters to available drivers, and ranks the top twenty by real ETA rather than straight line distance. Matching is the hard half. Two matchers will see the same driver, so I resolve it with a compare and set on the driver state from available to offered with an expiring lease, and the loser moves to its next candidate. Offers go to one driver at a time with a short timeout and automatic reassignment. The failure mode I would call out is a concert letting out, which is a hot partition in a handful of cells owned by one matcher, and the fix is splitting that region dynamically plus an honest queue for riders rather than a retry loop.