LearnHLDCaching patterns

Caching patterns

“Add a cache” is the most common sentence in system design rounds and the least informative. A cache is four decisions, not one: where it sits, how writes reach it, when entries leave, and what happens on the day it is empty. Get the last one wrong and the cache is the reason you go down rather than the reason you stay up.

Your answer

Your cache hit ratio slips from 99% to 95%. Traffic has not changed at all. Roughly what happens to the load on your database?

The layers

There is never one cache. By the time a request reaches your code it has passed several, and knowing which layer is responsible for a stale page is most of debugging.

Figure 1. Each layer catches what the one before it missed, so the traffic reaching the database is a small fraction of a small fraction.

The interesting property of Figure 1 is that each layer is smaller, faster and more stale than the one behind it. Moving a value one layer left makes it faster and more wrong, and that is the only trade you are ever making with a cache.

The number nobody expects

What the hit ratio actually buys
100 thousand
99%
Requests reaching the cache100,000 per second
Misses reaching the database100,000 x 1.0% = 1,000 per second
Compared with a 99% hit ratio1.0x the database load
1,000 to the database
and the cache is doing all the rest

This is the part that surprises people. Hit ratio and database load are not linear: going from 99% to 95% quintuples the misses, because what matters is the miss rate and 5% is five times 1%. A cache tuning change that sounds like a rounding error is a 5x load change on the thing least able to absorb it.

Drag the hit ratio from 99 down to 95 and watch the last row. Four percentage points is a five times increase in database load. This is why “the cache hit rate dipped a bit” is an incident and not a note in a dashboard.

How writes reach the cache

Three patterns, and the choice is about which failure you prefer.

Write pattern
The application owns the cache. Read it, and on a miss read the database and put the value back. Writes go to the database and delete the cache entry. This is the default and it should be your answer unless you have a reason. A cache outage degrades latency rather than breaking writes, and you never store anything the database has not already accepted.
Delete, do not update

On a write, delete the cache entry rather than writing the new value into it. Two concurrent writers that both update the cache can land in the opposite order to how they landed in the database, and the cache is then permanently wrong with no TTL short enough to make that acceptable. Deleting is idempotent and the next reader repopulates from the source of truth.

The three ways a cache takes you down

Each has a name, a mechanism and a fix. Interviewers ask about the first one constantly.

Stampede, also called the thundering herd. A popular key expires. A thousand concurrent requests all miss, all query the database for the same row at the same instant, and the database falls over serving one value a thousand times. The fix is single flight: the first miss takes a short lock and fetches, everyone else waits for that result. Serving slightly stale data while a refresh is in progress is a cheaper option again.

Penetration. Requests arrive for keys that do not exist anywhere, often from a scanner walking ids. Every one is a miss, so every one hits the database, and the cache provides no protection at all because there is nothing to cache. The fix is to cache the absence: store a null marker with a short TTL. A bloom filter in front is the heavier version for when the key space is genuinely huge.

Avalanche. You populate a cache at deploy time, or a large batch job writes many keys in the same second, and they all carry the same TTL. Some minutes later they all expire at once, and every one of them stampedes together. The fix is one line: add jitter to the TTL, so a nominal ten minutes becomes a random value between nine and eleven.

Break it

Cache health
99% hit
99% hit95% hithot key expirescache restartsprotected
Healthy. 100,000 requests a second, 1,000 of them reaching the database. Everything is comfortable and nobody thinks about the cache.

Eviction, briefly

Pick the policy from the access pattern
General purpose, recency predicts reuse
A small set of keys is popular forever
One large scan touches every key once
Data that is only valid for a known period

Trade-offs

ChoiceWhat you gainWhat you payPick it when
Cache asideCache failures degrade latency instead of breaking writes, and only requested data is ever stored.Every miss pays a round trip, and there is a brief window after a write where a reader can repopulate stale data.The default. Choose this and justify anything else.
Short TTLStaleness is bounded without any invalidation logic to get wrong.More misses, so a lower hit ratio and more database load.Data where a few seconds of staleness is invisible, which is most read paths.
Explicit invalidation on writeNear immediate consistency and a high hit ratio at the same time.Every code path that writes has to remember, and the one that forgets produces a bug nobody can reproduce.Data users edit and expect to see change immediately, like a profile.
Versioned keysNever invalidate anything. Bump a version in the key and old entries age out on their own.Wasted memory holding entries nobody will read again, and a version to propagate.Large derived objects that change wholesale, like a rendered page or a computed feed.

Checkpoint

Checkpoint

1. Hit ratio falls from 99% to 95% with no change in traffic. What happens to database load?

2. A row is updated. Should you write the new value into the cache or delete the key?

3. Your Redis restarts empty during peak traffic. Which defence keeps the site up?

Say this in 60 seconds

A cache is four decisions: where it sits, how writes reach it, when entries leave, and what happens when it is empty. I would default to cache aside, because a cache failure then costs latency rather than breaking writes, and I would delete on write rather than update, since two concurrent updates can land in the cache in the opposite order to the database. TTLs get jitter so a batch of keys does not expire together. The number I would want everyone to know is that database load tracks the miss rate, so a hit ratio slipping from 99 to 95 is five times the load, not four percent more. And the failure I design for is the cache coming back empty at peak: single flight so duplicate misses collapse into one query, plus a concurrency limit in front of the database so overload sheds instead of queues.

IndGeek provides solutions in the software field, and is a hub for ultimate Tech Knowledge.