Library
33 min read

Design a proximity service

Open a food app and type "restaurants near me". In under a tenth of a second you get twenty places, closest and best first. A proximity service is the system behind that answer: give it a point on the map, and it returns the places around that point worth showing.

"Restaurants near me" looks like the same question as "drivers near me", and it is almost the opposite one. Drivers move all the time and are looked up now and then. Restaurants never move and are looked up all the time. Watch one search go through.

ClientCDNSearch nodesCell indexCategory bitmapsRankingPlace storeResult cacheIndex builder

CDN first. A phone asks for restaurants within 5 km. Many searches end right here, because the CDN already holds the answer for this cell.

That one flip, rarely changes and is read constantly, decides almost everything. The ride hailing design (Design Uber) keeps everything in memory and throws it away freely. This one works things out ahead of time, keeps answers for minutes, and treats a restaurant changing address as a rare event that is allowed to be slow. Saying this out loud in the first two minutes is most of the interview.

A few words you will see all through the page:

  • A cell is one square of a grid laid over the map, like the squares on a city map in an old street atlas. Every place belongs to exactly one cell.
  • The index is the lookup table "cell to the places inside it". It is the heart of the design.
  • Hydrate means fetching the full details (name, address, hours) for an id you already have. You do it only for the few results you actually show.

Step 1: Understand the problem#

Six questions turn "find places near me" into something you can design. The first one matters most.

You askThey sayWhat it settles
How often does the data change?A business changes address a handful of times ever. New ones open daily.Everything. Data that barely changes and is read all the time can be worked out ahead of time, copied everywhere and cached for minutes. If you just designed ride hailing, say out loud that this is its mirror image.
How many places, and how many searches?A couple of hundred million places, tens of thousands of searches a second.The napkin math below shows the index fits on one machine, so every server can hold a full copy. There is no partitioning problem to solve. That is a rare luxury, and the design should use it.
What does "near" mean: a radius, or a count?The twenty best within about five kilometres, whichever limit comes first.A search whose real size changes with the place. In a busy city five kilometres holds thousands of places and you need the best twenty. In a village it holds three, and you should search wider instead of returning three.
What makes a result good, besides being close?Rating, whether it is open now, and how well it matches what was typed.Two stages. Location is a filter that throws most places away. The actual order is a scoring job over a small set that survives.
Does the edge of the circle have to be exact?No. Nobody notices a place at 5.1 km being included.Permission to use rough squares (cells) instead of exact distance while filtering. That turns the filter into a quick set lookup instead of geometry.
Is it personalised?Lightly. Mostly everyone in the same spot sees the same thing.Results cache well if you round the search location. Two people fifty metres apart should hit the same cache entry, and that one decision multiplies your hit ratio.

What you are building, and what you cut#

Functional requirements
What the system does.
  • Find places near a point, optionally filtered. By category, open now, rating.
  • Rank the candidates. Distance is one signal among several, not the answer.
  • Add, update and remove places. Rare, and allowed to take minutes to show up.
  • Serve tens of thousands of searches a second. Mostly from cache, close to the user.
Non-functional requirements
How well it has to do it. The numbers you commit to.
  • 200 million places worldwide.
  • Under 100ms at p99 for a nearby search (99 searches out of 100 are faster than that).
  • A new place is searchable within a few minutes.
  • Results may be minutes old, but must never include a permanently closed business.
Out of scope
What you cut, and say so out loud.
  • Turn by turn navigation and routing. A graph problem with its own literature.
  • Reviews and photos. A content system that this one links to.
  • Live crowd levels or wait times. A streaming problem bolted onto the side.
  • Indoor positioning. Different sensors, different everything.

Back of the envelope#

Three inputs. Watch the headline: it decides whether this is a sharding problem at all. (New to estimates like this? The back of the envelope chapter walks through the method.)

Index size and read pressure
200 million
30 thousand
5 km
Geo index only200M x 60 bytes = 12.0 GB
Full place records200M x 400 bytes = 80 GB
Cells to read at this radius(5km / 1.2km)^2 = 19 cells
Average places per cell4.8
Searches per second30,000
12.0 GB
for the whole world, so every server can hold all of it

That headline is the design. At 12.0 GB the index fits in memory on any server you would rent, so you copy it to every server instead of splitting it, and a search never crosses the network to find candidates. The 80 GB of full records stays in a normal database and is read only for the twenty results you return.

The number that changes the design

Watch what a bigger radius does to the cell count: it grows with the square. Five kilometres is about twenty cells, twenty five kilometres is about five hundred. If clients can pass any radius, one request on the same endpoint can cost a thousand times another. Either cap the radius or use bigger cells for wide searches, and say which before someone finds it in production.

Step 2: Propose the high level design#

The core entities
  • PlaceA business with its true position, opening hours and status. It changes rarely.opentemporarily_closedpermanently_closed
  • CellA square of about 1 km, holding the places inside it best first, with a category bitmap for each.
  • SearchA point rounded to about 100 metres, a radius and a category. The rounding is what makes it cacheable.
Never breaks: A permanently closed business never appears in results, whatever the index says.

The API#

GET/v1/nearby?lat=12.9352&lng=77.6245&radius=5000&category=restaurant&open=now
returns 200 { results: [{ id, name, distanceM, rating, openUntil }], nextCursor }
Why: The coordinates are rounded to three decimal places, about a hundred metres, before they become a cache key, so two people on the same street share an entry. Passing full precision through is the most common reason these endpoints have a 3% cache hit ratio and nobody can work out why.
GET/v1/nearby?cell=tdr1vx&category=restaurant
returns 200, cacheable for 10 minutes
Why: Asking by cell is what a CDN can actually cache: the world has a fixed number of cells but endless possible coordinates. Worth offering to clients that are fine with cell sized answers, which most map screens are.
PUT/v1/places/{id}
{ "name": "...", "lat": 12.9352, "lng": 77.6245, "categories": ["restaurant"], "hours": {...} }
returns 202 { indexedIn: "~3 minutes" }
Why: 202 means "accepted, not done yet", with an honest estimate. A write here means rebuilding part of an index and clearing caches, so promising it is searchable immediately would be a lie. Telling the caller how long it takes stops them from polling.
DELETE/v1/places/{id}
returns 204
Why: Closures travel faster than additions, on purpose. Showing a place that opened yesterday a few minutes late costs nothing. Sending someone to a restaurant that shut down is the failure people remember and write reviews about.

Here is the rounding as a worked example. One person stands at 12.93524, 77.62451 and a friend at 12.93548, 77.62467, about thirty metres away. Rounded to three decimals, both become 12.935, 77.625. Same key, one cache entry, one computation for both of them.

The data model#

Two stores, split by how they are read. The index answers "which places", the store answers "what are they".

place_indexin memory, copied in full to every search node
cell_iduint64PKAn S2 or geohash cell at a fixed level, roughly 1 km on a side. Everything hangs off this key.
place_idssorted arrayThe places in this cell, best first by a precomputed quality score, so reading only the first part still gets the good ones.
category_bitsbitmap per placeFiltering by category becomes one bitmap AND instead of a lookup per place, which matters when a cell holds a few thousand places.
qualityfloatWorked out ahead of time from rating and popularity, so the expensive part of scoring is already done.
Sample row
tdr1vx | [88213, 41902, 7714, ...] | restaurant|cafe | 0.82
The whole structure is rebuilt and shipped to every search node, exactly like the autocomplete index. It is small enough that copying beats splitting, which removes fan out, hot shards and rebalancing from the design in one decision.

Two words in that table. S2 is a Google library that cuts the globe into cells at levels 0 to 30, each level splitting a cell into four smaller ones. A geohash turns a point into a short string like tdr1vx, and nearby points usually share the same first letters. Either gives you integer cell ids, which is all this design needs. A bitmap is a row of bits, one per place: 1 if the place is a restaurant, 0 if not. ANDing the "restaurant" row with the "open late" row keeps the places that are both, in a single pass.

placesrelational or document store, sharded by place id
place_idbigintPK
name, address, phonetextThe heavy fields. Read only for the twenty results you return, never during the filter.
lat, lngdoubleThe true position. The cell in the index comes from this, not the other way round.
hoursjsonIDXOpen now is worked out at read time in the user timezone. Baking it into the index would mean rebuilding the index every hour for no reason.
statusenumopen, temporarily_closed, permanently_closed. Checked at serve time, so a stale index can never bring a closed business back.
Sample row
88213 | "Airlines Hotel" | 12.9352, 77.6245 | {mon: 06:00-22:00} | open
Only twenty rows are ever fetched from this store per search.

(SQL vs NoSQL covers how to choose the place store. For this design, either works.)

The whole system on one whiteboard#

Figure 1 is the same path as the picture at the top, with the update pipeline added.

Figure 1. Filter then rank, cheap then expensive. The geo index narrows two hundred million places to a few hundred using set operations, and only then does anything look at a real record.

Walking Figure 1:

  1. A search arrives with a point and a radius. Many never get further than the CDN, a network of servers near users that keeps copies of recent answers (the CDN section of push vs pull explains how they fill). The rest reach a search node, which rounds the coordinates first. That is what makes the next steps cacheable.
  2. The node works out which cells cover the circle and reads them from its own memory. No network call, because every node holds the whole index.
  3. Category filters are bitmap ANDs over the candidates, not a lookup per place.
  4. A few hundred candidates go to ranking, which combines real distance, precomputed quality and whether the place is open right now.
  5. Only the top twenty are hydrated from the place store. This is the only database read in the request. The answer is also kept in the result cache (the layers of caching are laid out in caching patterns).
  6. Separately, place updates trickle in from owners and bulk imports.
  7. Every few minutes a builder produces a new index version and ships it to every node, which swaps it in all at once. The autocomplete design uses the same build and swap trick.

The dashed closure feed exists because the batch cycle is too slow for one case. A permanently closed business must disappear now, not in three minutes, so closures go straight to the serving path instead of waiting for a rebuild. Being minutes behind is fine for everything else, which is a choice called eventual consistency (strong vs eventual consistency).

Your answer

Step 2 reads the cells covering a 5km circle. A user in Manhattan and a user in rural Rajasthan run the same query. Describe what goes wrong for each of them with a single fixed cell size.

Step 3: Design deep dive#

Covering a circle with cells#

Cells are squares and a search is a circle, so every cell scheme is an approximation. The real question is which way you round.

The rule is over cover, then filter. Take every cell the circle touches, gather all their places, then compute the true distance for each one and drop the ones outside the radius. Under covering means missing a place that really was nearby, and users notice that. Over covering means a little extra work, which they do not. True distance on a round planet uses the haversine formula, which takes two latitude and longitude pairs and returns the distance along the surface. You only run it on the candidates, never on the whole world.

Try it. Grey dots are places, orange ones came back from a cell you read, green ones are really inside the circle.

Cover the circle with cells
2 km
Cell size
Cells read
24
Candidates
254
Really inside
205
24 cells read, 254 places come back from them, and 205 are really inside 2 km. The other 49 are thrown away after a true distance check. Nobody inside the circle was missed, because every cell the circle touches was read.
The city is made up: a busy centre just left of the search point and a thin spread everywhere else. Try a big radius with small cells and count the cells, then the same radius with big cells and count the wasted places.

Two things to notice. Shrink the cells and the wasted orange dots drop, but the cell count climbs fast. Grow the radius and everything grows with its square. Now the harder part: one cell size for the whole world.

How big should a cell be?

  1. First try
    One fixed cell size everywhere
    Pick one level, say cells about a kilometre wide, and use it on every search. Simple, cheap, and easy to reason about, which is why you start here.
    Where it breaks: Both ends of the map at once. In a dense city twenty cells return thousands of candidates and ranking becomes the bottleneck. In a rural area the same twenty cells return almost nothing and the user gets a nearly empty page.
  2. Better
    Expand in rings until you have enough
    Start with the cell the user is in. Not enough places? Read the ring of cells around it, then the next ring, until you have enough candidates or hit a maximum radius. Empty areas get a wider search on their own, with no extra data.
    Where it breaks: The number of memory reads per search now varies. Your p99 gets worse than your median, and it gets worst exactly in the sparse places where you have fewest users to notice and fewest servers nearby.
  3. What you ship
    Cell size from a density map, plus rings as a backstop
    At build time, count places per region and pick the cell level per region so every cell holds roughly the same number of places: small cells in Manhattan, big ones in rural Rajasthan. Candidate counts become even everywhere, so latency becomes even. Ring expansion stays as the backstop for the cases the density map got wrong.

The cost of the last rung: the cell level is now data. It has to ship with the index, and the search node and the builder have to agree on it.

Filter is not ranking#

The most common mistake in this question is treating distance as the answer. It is a filter. The ranking that follows is where the product lives.

How do you pick the twenty results?

  1. First try
    Return the twenty closest
    Sort the places in the circle by distance and return the top twenty. It is correct in the most literal sense.
    Where it breaks: The closest place is a shop that closed an hour ago with two stars. Close is not the same as good, and the user scrolls past all twenty.
  2. Better
    Score every place in the circle using its full record
    Load each candidate from the place store, check hours, rating and relevance, and score it properly.
    Where it breaks: A dense area has thousands of candidates. That is thousands of database reads per search, times tens of thousands of searches a second.
  3. What you ship
    Filter cheaply in memory, score a few hundred, hydrate twenty
    Cells and bitmaps cut two hundred million places to a few hundred using set operations on integers. Ranking scores those from what is already in memory: distance, precomputed quality, open now. Only the final twenty are read from the place store.

Figure 2 plays the third rung message by message.

1/6 The first thing the node does is round the coordinates. Full precision would make every request a unique cache key and wreck the hit ratio for no benefit at all.
Figure 2. Two hundred million places become a few hundred candidates using nothing but set operations, and only twenty of them ever cause a database read.

That shape, cheap filter into expensive rank into a tiny hydration, is the same one behind a search engine's leaf shards and the ride hailing service's candidate list. Once you see it, most "find me the best few" questions become the same question.

The follow up you will get

"How do you handle 'open now' if hours are in the index?" Do not put them in the index. Opening hours change the answer every hour, so baking them in means rebuilding constantly. Keep hours on the place record, check them at serve time in the user's timezone, and fetch extra candidates so that dropping the closed ones still leaves twenty. Fetching twice as many is far cheaper than an hourly index rebuild.

Replicate, do not shard#

At a few gigabytes, the whole world fits on every search node. Replicating means every node keeps a full copy. Sharding means splitting the data so each node holds one slice (the sharding chapter covers it from the start). The reflex is to shard. Here it is the wrong reflex.

Where does the cell index live?

  1. First try
    In the database, queried on every search
    Store places in a database with a location index and run "places within 5 km" as a query. Nothing new to build.
    Where it breaks: Tens of thousands of searches a second, each one a geometry query against the same database. It becomes the bottleneck long before anything else is busy.
  2. Better
    In memory, sharded by region
    Load the index into memory and split the world into regions, one region per group of servers. Sounds natural for map data.
    Where it breaks: Traffic is wildly uneven: the shard holding Tokyo is far busier than the one holding the Sahara. Searches near a boundary must ask several shards and merge. Rebalancing means moving cells while queries run.
  3. What you ship
    In memory, a full copy on every node
    Every search node holds the whole index. A search is local, any node can serve any request, and handling more traffic means adding another identical machine. Fan out, hot shards and rebalancing all disappear in one decision.

Adding identical machines is horizontal scaling, compared with buying a bigger machine in vertical vs horizontal scaling. The cost of the last rung is that every node needs memory for the whole index, and every rebuild ships to every node. Both are fine at this size. Say where the line is, though: once the index stops fitting comfortably on one machine, you shard by cell range, with busy regions split more finely than empty ones (hash, range, or a directory).

Where the caching actually happens

The most valuable line of code in this system is the one that rounds the coordinates before building the cache key. Full precision means every request is unique and your cache does nothing. Rounding to about a hundred metres means everyone on the same block shares an entry, and a hit ratio that was three percent becomes seventy. It is a one line change with a bigger effect than any amount of index cleverness, and saying so shows you know where the wins are.

Break it#

Push each failure and watch which box goes red first.

Search load
30k/s
30k/sradius 50kmfestival in one cityindex rebuildrepaired
Healthy. Design load with rounded cache keys. Most searches are answered from cache, a miss is a memory scan of twenty cells, and the store sees twenty key reads per miss. Everything is comfortable.
Read every step as text
  1. 30k/s · Healthy. Design load with rounded cache keys. Most searches are answered from cache, a miss is a memory scan of twenty cells, and the store sees twenty key reads per miss. Everything is comfortable.
  2. radius 50km · Degraded. Someone sets a wide radius. The cell count grows with the square of the radius, so this one request reads five hundred cells and produces tens of thousands of candidates. One user can now cost a thousand times another on the same endpoint.
  3. festival in one city · Down. Everybody searches from the same few blocks. The rounded keys collapse to a handful of entries, which is good, but those entries live on one cache node and it saturates. A hot key, created by the very thing that gave you a good hit ratio.
  4. index rebuild · Degraded. A new index version ships to every node. Memory doubles during the swap, and the first queries after it run with cold CPU caches. Stagger it, and never let more than a fraction of the fleet swap at once.
  5. repaired · Healthy. Cap the radius and use bigger cells above a threshold, so a wide search is cheap and slightly less precise rather than expensive and exact. For the hot city, keep a small local cache inside each search node in front of the shared one, the same one second trick as the URL shortener.

The festival case shows that optimisations create their own failures. Rounding locations gave you a high hit ratio by pushing requests onto fewer keys, and pushing requests onto fewer keys is the definition of a hot key: one key so popular that the single machine holding it overloads. The URL shortener meets the same problem in the read path, and the one key that ruins it, and consistent hashing explains why spreading keys evenly does not help.

Trade-offs#

ChoiceWhat you gainWhat you payPick it when
Copy the index to every nodeA search is a local memory read, any node serves any request, and scaling out is copying a machine.Every node needs the whole index in memory, and every rebuild ships to all of them.While the index fits comfortably on one machine. Past that, shard by cell range and split the busy regions finely.
Cells instead of true geometryThe filter becomes set operations on integers, with no spatial index to maintain.Rough at the edge of the circle, and one fixed cell size is wrong somewhere in every country.Always for the filter stage. Compute true distance afterwards on the few hundred candidates that survive.
Rounding the search locationTurns endless possible keys into a bounded set and takes the cache hit ratio from single digits to most of your traffic.Results are computed for a nearby point rather than the exact one, and popular blocks become hot keys.Any location keyed cache. Round to less than the user would notice, around a hundred metres.
Batch index builds with a closure fast pathA simple, testable pipeline for the common case, and instant updates for the one case that matters.Two paths into the serving layer, and the fast path needs its own correctness argument.Whenever one kind of update is far more urgent than the rest. Making the whole pipeline fast to serve one case is the expensive mistake.

The level bar#

One moment from this round, answered three ways. Pick the one you think is staff level before the labels appear. (What changes at senior and staff explains the gap.)

Pick the staff level answer

InterviewerA user sets the search radius to 50 km. What happens?

Show which is which without guessing
  • Answer A: Mid level. Cell count grows with the square of the radius, so this one request reads thousands of cells, and the answer never notices one user now costs what a hundred do.
  • Answer B: Senior. Has the arithmetic and a guard. The cap simply refuses what the user asked for.
  • Answer C: Staff. Makes the wide search cheap instead of forbidding it, and argues which property the user does not need.

Then run the question cold, with a 45 minute timer and no notes, and score the run on the five things this question is really testing.

Score your last run
0 / 10
  1. Read or write heavy
  2. Where the index lives
  3. Cells
  4. Ranking
  5. The cache key
Pick 0, 1 or 2 on every row. Be the strict interviewer, not the kind friend.

Interview replay#

Interviewer
A user opens the app and wants restaurants near them. Walk me through it.
The opening. Watching for whether you reach for a database query with a location index.
You
We round the coordinates first, to about a hundred metres, so everyone on the same block shares a cache key. On a miss, the search node works out the cells covering the radius and reads them from its own memory, because the whole geographic index is only a few gigabytes and every node holds a full copy. That gives a few thousand candidates. We AND a category bitmap, compute real distance on what is left, rank a few hundred, and hydrate only the top twenty from the place store. One database read, for twenty keys.
Leads with the rounding, the decision with the biggest effect, and ends with the number of database reads. Both are memorable.
Interviewer
Why copy the index everywhere instead of sharding it by region?
Checking whether that was a decision or an accident.
You
Because it fits. A few gigabytes for two hundred million places means every node can hold everything, and that removes fan out at boundaries, hot shards over dense cities and rebalancing, all at once. Sharding by region is unbalanced by design, since Tokyo and the Sahara can be the same area and nowhere near the same traffic. I would shard only when the index stops fitting in memory, and then by cell range with busy regions split finer.
Names the three problems copying avoids and says when the answer flips.
Interviewer
How is this different from finding nearby drivers?
They want to know whether you understand why the same data structure gives a different design.
You
The read to write ratio flips. Drivers send a flood of location updates and are searched comparatively rarely, so that index lives in memory precisely because it is disposable and rebuilt every few seconds. Places almost never change, so this index is built ahead of time, copied everywhere and cached for minutes. Same cells, opposite reasoning: there I optimise for write throughput and accept losing state, here I optimise for read cost and accept minutes of staleness.
Comparing two systems by their read to write ratio is exactly the kind of transfer interviewers test for.
Interviewer
A restaurant permanently closes. When does it stop appearing?
A correctness question hiding inside a freshness question.
You
Not on the normal three minute cycle. Sending someone to a business that shut down is the failure people remember, so closures get a fast path: the status lives on the place record, it is checked at hydration time, and a closure feed pushes it immediately. The index can keep the stale id for a few minutes, because a candidate dropped at serve time costs nothing. Additions can wait, removals cannot.
Separates the two directions of freshness, which most candidates treat as one requirement.
Interviewer
What cell size would you pick?
Open ended, and the honest answer is "it depends" with a method attached.
You
I would not pick one for the whole world. I would target a number of places per cell rather than a distance, something like a few hundred, and let the level vary by region from a density map computed at build time. That keeps candidate counts, and so latency, even everywhere. I do not know the right target without measuring, and I would tune it by looking at the p99 candidate count per query rather than by reasoning about geometry.
Turns a geometry question into a latency question, which is what it really is, and admits what needs measuring.

Checkpoint#

Checkpoint

1. Why is one fixed cell size wrong in both dense and sparse areas?

2. What does rounding the search coordinates before building the cache key actually achieve?

3. Same design, but the places are now delivery riders moving through the city. What changes first?

Worth memorising
  • 200 million places is a few gigabytes of index, so every node holds all of it and nothing is sharded.
  • Cell count grows with the square of the radius: 5 km is about 20 cells, 25 km is about 500.
  • Round the search location to about 100 metres (three decimal places). That one line is your cache hit ratio.
  • Filter with cells, rank by distance and quality, hydrate twenty rows. Cheap, then expensive, then tiny.
Say this in 60 seconds

A proximity service is the mirror image of a ride hailing index. The data barely changes and is read constantly, so everything is worked out ahead of time. Two hundred million places is only a few gigabytes of geographic index, which means every search node holds a full copy and a search never leaves the machine, removing fan out and hot shards entirely. A query rounds its coordinates to about a hundred metres first, which is the single most valuable decision on the page because it turns endless possible cache keys into a bounded set. Then it reads the cells covering the radius, ANDs a category bitmap, computes true distance on what survives, ranks a few hundred candidates on distance, quality and whether they are open, and hydrates only the top twenty from the place store. Cell size adapts to density so candidate counts stay even, because one fixed size is wrong in Manhattan and wrong in rural Rajasthan for opposite reasons. Index builds run every few minutes, with one exception: a permanently closed business is removed immediately through a fast path, because additions can wait and removals cannot.