Design YouTube
In December 2014 the view counter under Gangnam Style ran out of room. The video had passed 2,147,483,647 views, the largest number a signed 32 bit integer can hold, and YouTube had to move the counter to 64 bits. Nobody had planned for one video to be watched more than two billion times. That is the kind of problem this chapter is about.
Watch one video's life below: it is published, people are told, someone opens it, and the number under it goes up.
Transcoded. The upload pipeline from the video chapter hands over a finished, transcoded video. It is still private.
You already know how to store and stream the video file itself. That is the video streaming chapter: upload in parts, transcode in chunks, serve segments from a CDN (a network of servers close to viewers that keep copies of popular files). This chapter is everything around the player. The number under the video, the subscriptions tab, the bell, the comments, and the copyright check that runs before anyone sees an upload. In an interview, "Design YouTube" is mostly these parts, and they are where the hard problems sit.
A few words come up all through this page:
- A hot row is one database row that far too many requests want to change at the same moment. Each change locks the row, so they wait in a line.
- Fan out means one event (a new upload) turning into many writes or messages (one per subscriber).
- A shard is one slice of a database that has been split across many machines. The sharding chapter explains how the split is chosen.
Step 1: Understand the problem#
"Design YouTube" is huge, so the first job is to agree on which part you are building. Ask these in the first three minutes.
| You ask | They say | What it settles |
|---|---|---|
| Is the video pipeline part of this, and the home page recommendations? | No. Assume upload, transcoding and playback exist. Treat the home page ranking as a service another team owns. | You can spend the round on the platform around the player. Say in one sentence how the pipe works and point at it, so the interviewer knows you are cutting it on purpose. |
| How big? | Assume a billion watch page loads a day. Some channels have hundreds of millions of subscribers. | The watch page is a read problem a cache solves. The big channels are what break a simple subscriptions feed, so they get their own deep dive. |
| How exact does the view count have to be? | Close enough, within a few minutes, for viewers. Exact, with fake views removed, for paying creators. | Two numbers from two pipelines: a fast one for display and a slow, careful one for money. This single answer shapes the whole counting design. |
| Who gets told when a channel uploads? | Every subscriber sees it in their subscriptions tab. Only those who turned the bell on get a push notification, within minutes. | Two different audiences of two different sizes. The feed can be slightly late, the push can be spread over a few minutes, and neither has to be instant. |
| How are comments ordered? | Top comments by default, newest first as an option. | Top order cannot be a sort at read time on a video with a million comments. It becomes a list you compute ahead of time. |
| Do we check uploads for copyright? | Yes. Match each upload against reference files that rights holders have supplied, before it goes public. | A gate in the publish path that can block a video or hand its money to someone else. How the matching works is its own system, so you design the gate, not the matcher. |
What you are building, and what you cut#
- The watch page: title, channel, counts, and your own state. One call that must be fast for a billion loads a day.
- View and like counts. Approximately right in minutes, exactly right by the next day.
- Subscriptions tab and bell notifications. A news feed with very uneven channel sizes.
- Comments, with top and newest ordering. Write light, read heavy, and very skewed.
- The Content ID gate before publish. A step in the publish path, with the matcher treated as a black box.
- Watch page metadata in under 200ms at p95 (95 out of 100 requests are faster than that).
- Displayed counts no more than a few minutes behind.
- Bell notifications delivered within about 10 minutes of publish.
- Counts used for creator payouts are exact and published daily.
- Upload, transcoding and the CDN. Covered in the video streaming chapter.
- Recommendations and the home page. A ranking system, and it belongs with the AI path.
- Search. The same shape as the web search engine chapter, over titles and captions.
- Live streaming and ads. Each one is its own design.
Back of the envelope#
Move the sliders and watch which number grows fastest. (New to these estimates? The back of the envelope chapter shows the method.)
The watch page is 11,574 reads a second, and nearly all of them are the same few thousand popular videos, so a cache absorbs it. The heartbeats are 400 times the likes and every one is a write. That stream must never touch a database row directly, which is what the first deep dive is about.
A heartbeat is the small message the player sends every 30 seconds while a video plays, saying "this session is still watching, and it is at this second". It is how the system knows a view really happened, and there are a lot of them.
Candidates average the views across all videos and conclude a counter column is fine. The average hides the shape. Ten million views of one trailer in its first hour is about 2,800 a second, all on one row. A single row in a relational database manages a few thousand updates a second at best before the lock queue becomes the bottleneck, and that is one video on one good day.