Open a food app and type "restaurants near me". In under a tenth of a second you get twenty
places, closest and best first. A proximity service is the system behind that answer:
give it a point on the map, and it returns the places around that point worth showing.
"Restaurants near me" looks like the same question as "drivers near me", and it is almost the
opposite one. Drivers move all the time and are looked up now and then. Restaurants never
move and are looked up all the time. Watch one search go through.
CDN first. A phone asks for restaurants within 5 km. Many searches end right here, because the CDN already holds the answer for this cell.
drivers: always movinglooked up now and thenrestaurants: never movelooked up all the time
That one flip, rarely changes and is read constantly, decides almost everything. The
ride hailing design (Design Uber) keeps everything in memory and
throws it away freely. This one works things out ahead of time, keeps answers for minutes,
and treats a restaurant changing address as a rare event that is allowed to be slow. Saying
this out loud in the first two minutes is most of the interview.
A few words you will see all through the page:
A cell is one square of a grid laid over the map, like the squares on a city map in an
old street atlas. Every place belongs to exactly one cell.
The index is the lookup table "cell to the places inside it". It is the heart of the
design.
Hydrate means fetching the full details (name, address, hours) for an id you already
have. You do it only for the few results you actually show.
Six questions turn "find places near me" into something you can design. The first one
matters most.
You ask
They say
What it settles
How often does the data change?
A business changes address a handful of times ever. New ones open daily.
Everything. Data that barely changes and is read all the time can be worked out ahead of time, copied everywhere and cached for minutes. If you just designed ride hailing, say out loud that this is its mirror image.
How many places, and how many searches?
A couple of hundred million places, tens of thousands of searches a second.
The napkin math below shows the index fits on one machine, so every server can hold a full copy. There is no partitioning problem to solve. That is a rare luxury, and the design should use it.
What does "near" mean: a radius, or a count?
The twenty best within about five kilometres, whichever limit comes first.
A search whose real size changes with the place. In a busy city five kilometres holds thousands of places and you need the best twenty. In a village it holds three, and you should search wider instead of returning three.
What makes a result good, besides being close?
Rating, whether it is open now, and how well it matches what was typed.
Two stages. Location is a filter that throws most places away. The actual order is a scoring job over a small set that survives.
Does the edge of the circle have to be exact?
No. Nobody notices a place at 5.1 km being included.
Permission to use rough squares (cells) instead of exact distance while filtering. That turns the filter into a quick set lookup instead of geometry.
Is it personalised?
Lightly. Mostly everyone in the same spot sees the same thing.
Results cache well if you round the search location. Two people fifty metres apart should hit the same cache entry, and that one decision multiplies your hit ratio.
Three inputs. Watch the headline: it decides whether this is a sharding problem at all. (New
to estimates like this? The back of the envelope chapter
walks through the method.)
Index size and read pressure
200 million
30 thousand
5 km
Geo index only200M x 60 bytes = 12.0 GB
Full place records200M x 400 bytes = 80 GB
Cells to read at this radius(5km / 1.2km)^2 = 19 cells
Average places per cell4.8
Searches per second30,000
12.0 GB
for the whole world, so every server can hold all of it
That headline is the design. At 12.0 GB the index fits in memory on any server you would rent, so you copy it to every server instead of splitting it, and a search never crosses the network to find candidates. The 80 GB of full records stays in a normal database and is read only for the twenty results you return.
The number that changes the design
Watch what a bigger radius does to the cell count: it grows with the square. Five
kilometres is about twenty cells, twenty five kilometres is about five hundred. If clients
can pass any radius, one request on the same endpoint can cost a thousand times another.
Either cap the radius or use bigger cells for wide searches, and say which before someone
finds it in production.
5 km: about 20 cells25 km: about 500cap the radius, or one request costs a thousand others
Why: The coordinates are rounded to three decimal places, about a hundred metres, before they become a cache key, so two people on the same street share an entry. Passing full precision through is the most common reason these endpoints have a 3% cache hit ratio and nobody can work out why.
GET/v1/nearby?cell=tdr1vx&category=restaurant
returns 200, cacheable for 10 minutes
Why: Asking by cell is what a CDN can actually cache: the world has a fixed number of cells but endless possible coordinates. Worth offering to clients that are fine with cell sized answers, which most map screens are.
Why: 202 means "accepted, not done yet", with an honest estimate. A write here means rebuilding part of an index and clearing caches, so promising it is searchable immediately would be a lie. Telling the caller how long it takes stops them from polling.
DELETE/v1/places/{id}
returns 204
Why: Closures travel faster than additions, on purpose. Showing a place that opened yesterday a few minutes late costs nothing. Sending someone to a restaurant that shut down is the failure people remember and write reviews about.
Here is the rounding as a worked example. One person stands at 12.93524, 77.62451 and a
friend at 12.93548, 77.62467, about thirty metres away. Rounded to three decimals, both
become 12.935, 77.625. Same key, one cache entry, one computation for both of them.
The whole structure is rebuilt and shipped to every search node, exactly like the autocomplete index. It is small enough that copying beats splitting, which removes fan out, hot shards and rebalancing from the design in one decision.
Two words in that table. S2 is a Google library that cuts the globe into cells at levels
0 to 30, each level splitting a cell into four smaller ones. A geohash turns a point into
a short string like tdr1vx, and nearby points usually share the same first letters. Either
gives you integer cell ids, which is all this design needs. A bitmap is a row of bits, one
per place: 1 if the place is a restaurant, 0 if not. ANDing the "restaurant" row with the
"open late" row keeps the places that are both, in a single pass.
restaurantopen lateboth (AND)one bit per place, one pass, no lookup per place
placesrelational or document store, sharded by place id
place_id
bigint
PK
name, address, phone
text
The heavy fields. Read only for the twenty results you return, never during the filter.
lat, lng
double
The true position. The cell in the index comes from this, not the other way round.
hours
json
IDX
Open now is worked out at read time in the user timezone. Baking it into the index would mean rebuilding the index every hour for no reason.
status
enum
open, temporarily_closed, permanently_closed. Checked at serve time, so a stale index can never bring a closed business back.
Figure 1 is the same path as the picture at the top, with the update pipeline added.
often ends herelat, lng, radius1cell + neighbours2category AND3a few hundred candidates4store the top 20hydrate 20 rows56swap in a new version7closed, immediatelyClienta map, usuallyCDNcell keyed resultsSearch nodesfull index eachCell indexin memory, replicatedCategory bitmapsAND, then truncateRankingdistance, quality, openPlace storenames, hours, statusResult cachequantised locationPlace updatesowners, importsIndex builderevery few minutesClosure feedfast path
Figure 1. Filter then rank, cheap then expensive. The geo index narrows two hundred million places to a few hundred using set operations, and only then does anything look at a real record.
Walking Figure 1:
A search arrives with a point and a radius. Many never get further than the CDN, a
network of servers near users that keeps copies of recent answers (the CDN section of
push vs pull explains how they fill). The rest reach a
search node, which rounds the coordinates first. That is what makes the next steps
cacheable.
The node works out which cells cover the circle and reads them from its own memory. No
network call, because every node holds the whole index.
Category filters are bitmap ANDs over the candidates, not a lookup per place.
A few hundred candidates go to ranking, which combines real distance, precomputed quality
and whether the place is open right now.
Only the top twenty are hydrated from the place store. This is the only database read in
the request. The answer is also kept in the result cache (the layers of caching are laid
out in caching patterns).
Separately, place updates trickle in from owners and bulk imports.
Every few minutes a builder produces a new index version and ships it to every node, which
swaps it in all at once. The autocomplete design
uses the same build and swap trick.
The dashed closure feed exists because the batch cycle is too slow for one case. A
permanently closed business must disappear now, not in three minutes, so closures go
straight to the serving path instead of waiting for a rebuild. Being minutes behind is fine
for everything else, which is a choice called eventual consistency
(strong vs eventual consistency).
closed for goodclosure feed: noweverything else: the next rebuildsearch nodes
Your answer
Step 2 reads the cells covering a 5km circle. A user in Manhattan and a user in rural Rajasthan run the same query. Describe what goes wrong for each of them with a single fixed cell size.
Cells are squares and a search is a circle, so every cell scheme is an approximation. The
real question is which way you round.
The rule is over cover, then filter. Take every cell the circle touches, gather all their
places, then compute the true distance for each one and drop the ones outside the radius.
Under covering means missing a place that really was nearby, and users notice that. Over
covering means a little extra work, which they do not. True distance on a round planet uses
the haversine formula, which takes two latitude and longitude pairs and returns the distance
along the surface. You only run it on the candidates, never on the whole world.
Try it. Grey dots are places, orange ones came back from a cell you read, green ones are
really inside the circle.
Cover the circle with cells
2 km
Cell size
Cells read
24
Candidates
254
Really inside
205
24 cells read, 254 places come back from them, and 205 are really inside 2 km. The other 49 are thrown away after a true distance check. Nobody inside the circle was missed, because every cell the circle touches was read.
The city is made up: a busy centre just left of the search point and a thin spread everywhere else. Try a big radius with small cells and count the cells, then the same radius with big cells and count the wasted places.
Two things to notice. Shrink the cells and the wasted orange dots drop, but the cell count
climbs fast. Grow the radius and everything grows with its square. Now the harder part: one
cell size for the whole world.
How big should a cell be?
1
First try
One fixed cell size everywhere
Pick one level, say cells about a kilometre wide, and use it on every search. Simple, cheap, and easy to reason about, which is why you start here.
Where it breaks: Both ends of the map at once. In a dense city twenty cells return thousands of candidates and ranking becomes the bottleneck. In a rural area the same twenty cells return almost nothing and the user gets a nearly empty page.
2
Better
Expand in rings until you have enough
Start with the cell the user is in. Not enough places? Read the ring of cells around it, then the next ring, until you have enough candidates or hit a maximum radius. Empty areas get a wider search on their own, with no extra data.
Where it breaks: The number of memory reads per search now varies. Your p99 gets worse than your median, and it gets worst exactly in the sparse places where you have fewest users to notice and fewest servers nearby.
3
What you ship
Cell size from a density map, plus rings as a backstop
At build time, count places per region and pick the cell level per region so every cell holds roughly the same number of places: small cells in Manhattan, big ones in rural Rajasthan. Candidate counts become even everywhere, so latency becomes even. Ring expansion stays as the backstop for the cases the density map got wrong.
The cost of the last rung: the cell level is now data. It has to ship with the index, and the
search node and the builder have to agree on it.
The most common mistake in this question is treating distance as the answer. It is a filter.
The ranking that follows is where the product lives.
How do you pick the twenty results?
1
First try
Return the twenty closest
Sort the places in the circle by distance and return the top twenty. It is correct in the most literal sense.
Where it breaks: The closest place is a shop that closed an hour ago with two stars. Close is not the same as good, and the user scrolls past all twenty.
2
Better
Score every place in the circle using its full record
Load each candidate from the place store, check hours, rating and relevance, and score it properly.
Where it breaks: A dense area has thousands of candidates. That is thousands of database reads per search, times tens of thousands of searches a second.
3
What you ship
Filter cheaply in memory, score a few hundred, hydrate twenty
Cells and bitmaps cut two hundred million places to a few hundred using set operations on integers. Ranking scores those from what is already in memory: distance, precomputed quality, open now. Only the final twenty are read from the place store.
Figure 2 plays the third rung message by message.
ClientSearch nodeCell indexRankerPlace storelat, lng, 5km, restaurantscells covering the circle~2,400 place idscategory bitmap AND, then distance~300 candidateshydrate 20 by id
1/6 The first thing the node does is round the coordinates. Full precision would make every request a unique cache key and wreck the hit ratio for no benefit at all.
Figure 2. Two hundred million places become a few hundred candidates using nothing but set operations, and only twenty of them ever cause a database read.
That shape, cheap filter into expensive rank into a tiny hydration, is the same one behind a
search engine's leaf shards and the ride hailing service's candidate list. Once you see it,
most "find me the best few" questions become the same question.
200 million placesa few hundred, from the cells20 ranked and hydratedcheap filter, expensive rank, tiny hydration
The follow up you will get
"How do you handle 'open now' if hours are in the index?" Do not put them in the index.
Opening hours change the answer every hour, so baking them in means rebuilding constantly.
Keep hours on the place record, check them at serve time in the user's timezone, and fetch
extra candidates so that dropping the closed ones still leaves twenty. Fetching twice as
many is far cheaper than an hourly index rebuild.
At a few gigabytes, the whole world fits on every search node. Replicating means every
node keeps a full copy. Sharding means splitting the data so each node holds one slice
(the sharding chapter covers it from the start). The reflex is to
shard. Here it is the wrong reflex.
Where does the cell index live?
1
First try
In the database, queried on every search
Store places in a database with a location index and run "places within 5 km" as a query. Nothing new to build.
Where it breaks: Tens of thousands of searches a second, each one a geometry query against the same database. It becomes the bottleneck long before anything else is busy.
2
Better
In memory, sharded by region
Load the index into memory and split the world into regions, one region per group of servers. Sounds natural for map data.
Where it breaks: Traffic is wildly uneven: the shard holding Tokyo is far busier than the one holding the Sahara. Searches near a boundary must ask several shards and merge. Rebalancing means moving cells while queries run.
3
What you ship
In memory, a full copy on every node
Every search node holds the whole index. A search is local, any node can serve any request, and handling more traffic means adding another identical machine. Fan out, hot shards and rebalancing all disappear in one decision.
Adding identical machines is horizontal scaling, compared with buying a bigger machine in
vertical vs horizontal scaling. The cost of the
last rung is that every node needs memory for the whole index, and every rebuild ships to
every node. Both are fine at this size. Say where the line is, though: once the index stops
fitting comfortably on one machine, you shard by cell range, with busy regions split more
finely than empty ones (hash, range, or a directory).
Where the caching actually happens
The most valuable line of code in this system is the one that rounds the coordinates before
building the cache key. Full precision means every request is unique and your cache does
nothing. Rounding to about a hundred metres means everyone on the same block shares an
entry, and a hit ratio that was three percent becomes seventy. It is a one line change with
a bigger effect than any amount of index cleverness, and saying so shows you know where
the wins are.
Push each failure and watch which box goes red first.
Search load
30k/s
30k/sradius 50kmfestival in one cityindex rebuildrepaired
misshydrate 20ClientsResult cacheSearch nodesfull indexPlace store
misshydrate 20ClientsResult cacheSearch nodesfull indexPlace store
Healthy. Design load with rounded cache keys. Most searches are answered from cache, a miss is a memory scan of twenty cells, and the store sees twenty key reads per miss. Everything is comfortable.
Read every step as text
30k/s · Healthy. Design load with rounded cache keys. Most searches are answered from cache, a miss is a memory scan of twenty cells, and the store sees twenty key reads per miss. Everything is comfortable.
radius 50km · Degraded. Someone sets a wide radius. The cell count grows with the square of the radius, so this one request reads five hundred cells and produces tens of thousands of candidates. One user can now cost a thousand times another on the same endpoint.
festival in one city · Down. Everybody searches from the same few blocks. The rounded keys collapse to a handful of entries, which is good, but those entries live on one cache node and it saturates. A hot key, created by the very thing that gave you a good hit ratio.
index rebuild · Degraded. A new index version ships to every node. Memory doubles during the swap, and the first queries after it run with cold CPU caches. Stagger it, and never let more than a fraction of the fleet swap at once.
repaired · Healthy. Cap the radius and use bigger cells above a threshold, so a wide search is cheap and slightly less precise rather than expensive and exact. For the hot city, keep a small local cache inside each search node in front of the shared one, the same one second trick as the URL shortener.
The festival case shows that optimisations create their own failures. Rounding locations gave
you a high hit ratio by pushing requests onto fewer keys, and pushing requests onto fewer keys
is the definition of a hot key: one key so popular that the single machine holding it
overloads. The URL shortener meets the same problem in
the read path, and the one key that ruins it,
and consistent hashing
explains why spreading keys evenly does not help.
One moment from this round, answered three ways. Pick the one you think is staff level before
the labels appear. (What changes at senior and staff
explains the gap.)
Pick the staff level answer
InterviewerA user sets the search radius to 50 km. What happens?
Show which is which without guessing
Answer A: Mid level. Cell count grows with the square of the radius, so this one request reads thousands of cells, and the answer never notices one user now costs what a hundred do.
Answer B: Senior. Has the arithmetic and a guard. The cap simply refuses what the user asked for.
Answer C: Staff. Makes the wide search cheap instead of forbidding it, and argues which property the user does not need.
Then run the question cold, with a 45 minute timer and no notes, and score the run on the
five things this question is really testing.
Score your last run
0 / 10
Read or write heavy
Where the index lives
Cells
Ranking
The cache key
Pick 0, 1 or 2 on every row. Be the strict interviewer, not the kind friend.
A user opens the app and wants restaurants near them. Walk me through it.
The opening. Watching for whether you reach for a database query with a location index.
You
We round the coordinates first, to about a hundred metres, so everyone on the same block shares a cache key. On a miss, the search node works out the cells covering the radius and reads them from its own memory, because the whole geographic index is only a few gigabytes and every node holds a full copy. That gives a few thousand candidates. We AND a category bitmap, compute real distance on what is left, rank a few hundred, and hydrate only the top twenty from the place store. One database read, for twenty keys.
Leads with the rounding, the decision with the biggest effect, and ends with the number of database reads. Both are memorable.
Interviewer
Why copy the index everywhere instead of sharding it by region?
Checking whether that was a decision or an accident.
You
Because it fits. A few gigabytes for two hundred million places means every node can hold everything, and that removes fan out at boundaries, hot shards over dense cities and rebalancing, all at once. Sharding by region is unbalanced by design, since Tokyo and the Sahara can be the same area and nowhere near the same traffic. I would shard only when the index stops fitting in memory, and then by cell range with busy regions split finer.
Names the three problems copying avoids and says when the answer flips.
Interviewer
How is this different from finding nearby drivers?
They want to know whether you understand why the same data structure gives a different design.
You
The read to write ratio flips. Drivers send a flood of location updates and are searched comparatively rarely, so that index lives in memory precisely because it is disposable and rebuilt every few seconds. Places almost never change, so this index is built ahead of time, copied everywhere and cached for minutes. Same cells, opposite reasoning: there I optimise for write throughput and accept losing state, here I optimise for read cost and accept minutes of staleness.
Comparing two systems by their read to write ratio is exactly the kind of transfer interviewers test for.
Interviewer
A restaurant permanently closes. When does it stop appearing?
A correctness question hiding inside a freshness question.
You
Not on the normal three minute cycle. Sending someone to a business that shut down is the failure people remember, so closures get a fast path: the status lives on the place record, it is checked at hydration time, and a closure feed pushes it immediately. The index can keep the stale id for a few minutes, because a candidate dropped at serve time costs nothing. Additions can wait, removals cannot.
Separates the two directions of freshness, which most candidates treat as one requirement.
Interviewer
What cell size would you pick?
Open ended, and the honest answer is "it depends" with a method attached.
You
I would not pick one for the whole world. I would target a number of places per cell rather than a distance, something like a few hundred, and let the level vary by region from a density map computed at build time. That keeps candidate counts, and so latency, even everywhere. I do not know the right target without measuring, and I would tune it by looking at the p99 candidate count per query rather than by reasoning about geometry.
Turns a geometry question into a latency question, which is what it really is, and admits what needs measuring.
1. Why is one fixed cell size wrong in both dense and sparse areas?
2. What does rounding the search coordinates before building the cache key actually achieve?
3. Same design, but the places are now delivery riders moving through the city. What changes first?
Worth memorising
200 million places is a few gigabytes of index, so every node holds all of it and nothing is sharded.
Cell count grows with the square of the radius: 5 km is about 20 cells, 25 km is about 500.
Round the search location to about 100 metres (three decimal places). That one line is your cache hit ratio.
Filter with cells, rank by distance and quality, hydrate twenty rows. Cheap, then expensive, then tiny.
Say this in 60 seconds
A proximity service is the mirror image of a ride hailing index. The data barely changes and is read constantly, so everything is worked out ahead of time. Two hundred million places is only a few gigabytes of geographic index, which means every search node holds a full copy and a search never leaves the machine, removing fan out and hot shards entirely. A query rounds its coordinates to about a hundred metres first, which is the single most valuable decision on the page because it turns endless possible cache keys into a bounded set. Then it reads the cells covering the radius, ANDs a category bitmap, computes true distance on what survives, ranks a few hundred candidates on distance, quality and whether they are open, and hydrates only the top twenty from the place store. Cell size adapts to density so candidate counts stay even, because one fixed size is wrong in Manhattan and wrong in rural Rajasthan for opposite reasons. Index builds run every few minutes, with one exception: a permanently closed business is removed immediately through a fast path, because additions can wait and removals cannot.