Design a video streaming service
Read firstCDNsDesign object storage
A phone on a train plays a video at 1080p, goes through a tunnel, drops to 480p for eight seconds, and comes back up. Nobody notices. That is the product.
Below, one video makes the whole trip: uploaded once across the top, then watched along the bottom.
Upload. The uploader sends the file in parts straight to object storage, using pre signed URLs. Our own servers never touch the bytes.
The interesting thing about this question is that almost none of the hard work happens while somebody is watching. Playback is a static file download from a cache near the user, and it is meant to be boring. Everything expensive happens once, when the video is uploaded, and everything clever happens on the client. Candidates who spend the interview designing the streaming path are designing the easy half.
Think of a bakery. The baking happens once, early, in one kitchen, and it can take hours. The selling happens all day, in hundreds of corner shops close to where people live. Nobody bakes a loaf when a customer walks in. A video service works the same way: transcode once, then hand out finished files from shelves near the viewer.
Words this page uses a lot:
- Transcoding means re-encoding a video into a different size or format. One upload becomes several versions.
- A rendition is one of those versions: the same video at one resolution and quality, like 240p, 480p, 1080p or 4K.
- Bitrate is how many bits a second of video takes, written in Mbps (megabits per second). Higher quality means a higher bitrate and a bigger download.
- A segment is a few seconds of video saved as its own small file. Here, six seconds.
- A manifest is a small text file that lists the renditions and the URL of every segment. The player reads it first, then fetches segments.
- A CDN (content delivery network) is a set of servers spread around the world that keep copies of popular files close to viewers. Each location is an edge. The origin is where the master copy lives, here object storage.
Step 1: Understand the problem#
The first question separates this into two completely different systems, so ask it first.
| You ask | They say | What it settles |
|---|---|---|
| On demand video, or live? | On demand. | Everything. On demand means transcode once and serve a static file forever. Live means transcoding while people wait, inside a tight time budget, which is a different page and you should say so. |
| How many hours are uploaded, and how many watched? | Thousands of hours uploaded a day. Hundreds of millions of hours watched. | At the default numbers in the napkin below, about fifty thousand hours are watched for every hour uploaded. Upload is a batch problem you can queue and watching is a bandwidth problem you cannot, so they get completely separate infrastructure. |
| What devices and what networks? | Everything from a 3G phone to a 4K television. | Several renditions of every video, and switching between them while the video plays. It is also why a single file per video is not an option. |
| How soon after upload must a video be watchable? | Minutes, not hours. | Transcoding is split into chunks and done in parallel, not as one long job per file. A two hour film transcoded one frame after another is a very long wait. |
| How exact does the view count have to be? | Roughly right, quickly. Exact eventually, for payouts. | A streaming counter for the number on screen and a batch recount for the number creators are paid on. The same shape as the ad click aggregator, and worth saying so. |
| Do we need DRM and geo restrictions? | Signed URLs and region blocking, yes. Full DRM, assume not. | DRM (digital rights management) is copy protection built into the player. Without it, short lived signed URLs checked at the CDN edge do the job, instead of an authorisation call per segment. An auth check on every six second chunk would be the busiest endpoint you own. |
That view count answer is exactly the "two pipelines, one number" idea in the ad click aggregator, and the YouTube page builds the counter in full. This page stays on the video itself.
What you are building, and what you cut#
- Upload a video and make it watchable in minutes. Upload in parts that can resume, then transcode in parallel.
- Transcode into several renditions. From 240p to 4K, in more than one compression format.
- Stream adaptively to any device. The player picks the rendition, segment by segment.
- Count views. Approximately now, exactly later.
- Playback starts in under 2 seconds at p95 (95 out of 100 plays start at least this fast).
- Rebuffering under 0.5% of watch time.
- A video is watchable within 5 minutes of upload finishing.
- Around a hundred terabits a second of egress at peak, almost all of it from cache.
- Recommendations and the home page. It is a ranking system that happens to point at videos.
- Live streaming, which puts transcoding on the latency path and needs its own design.
- Comments, subscriptions and the social layer. A feed problem, covered on the YouTube page.
- Content moderation and copyright matching. Enormous, and it runs beside the pipeline rather than inside it.
Back of the envelope#
Egress is data sent out to viewers, and cloud providers and networks charge by it. A quick way to feel the size: 500 million hours watched a day, spread over 24 hours, is about 21 million people watching at any moment. At 2.5 Mbps each, that is about 52 terabits a second, on average. The napkin does the same sum with sliders. (The method is in back of the envelope.)
Storage grows by 175TB a day and never shrinks, but it is cheap and boring. The number that dominates the bill is egress. Everything in this design that looks like an optimisation is really an attempt to serve that 114.6 Tbps from a cache close to the user rather than from your own machines.
Candidates compute storage, find a large number, and design around it. Storage is the cheapest part of this system by an order of magnitude. Egress is the bill, and the cache hit ratio at the edge (the share of requests the edge answers from its own copy) is the single metric that moves it. Out of a million segment requests, a 99% hit ratio sends 10,000 to the origin and a 95% hit ratio sends 50,000. That is five times the origin traffic, the difference between a normal origin fleet and a heroic one.
Step 2: Propose the high level design#
- VideoAn id given at upload start, the source file, and which renditions are ready.uploadingtranscodingreadyfailedblocked
- Chunk jobThirty seconds of one rendition. A two hour film at seven renditions is 1,680 of them, each independent.
- SegmentA few seconds of one rendition, at a URL that never changes, cached for a year.
- ManifestThe list of renditions and segments for one video. The only part of playback that is per user.
The API#
A pre signed URL is a link that carries its own short lived permission, so the client can upload straight to object storage without any storage password. Uploading a big file in parts like this is the multipart upload from the object storage write path.
{ "filename": "final_cut_v3.mov", "sizeBytes": 8140000000, "sha256": "..." }{ "positionMs": 184000, "rendition": "720p", "bufferMs": 12000, "sessionId": "..." }The data model#
| video_id | uuid | PK | Assigned at upload start, before a single byte arrives. |
| state | enum | IDX | uploading, transcoding, ready, failed, blocked. The client polls this, and it is the only thing standing between a user and a broken player. |
| source_key | varchar(256) | Where the original lives. Kept forever, because every future compression format means transcoding it again. | |
| duration_ms | bigint | From the probe step, not from the uploader. | |
| renditions | json | Which resolutions and formats actually completed. A video can be watchable at 480p while 4K is still encoding. | |
| region_policy | json | Where it may be played. Enforced at the edge, not in the player. |
The whole system on one whiteboard#
Figure 1 is the picture at the top, flattened, with the view counter added.
Walking Figure 1:
- The client uploads parts directly to object storage with pre signed URLs. Your API never touches the bytes, which is the single biggest saving in the upload path.
- Completion triggers the pipeline. The source is probed (read to find its real length, resolution and format) and split into chunks of about thirty seconds, cut on keyframe boundaries so each chunk can be decoded on its own. Keyframes are explained in the first deep dive.
- Each chunk becomes one job per rendition, waiting in a queue (the message queues chapter covers how that works). A two hour film is 7,200 seconds, so 240 chunks, and at seven renditions that is 1,680 small jobs instead of one enormous one.
- Workers pick them up. This fleet is the largest compute cost in the company and it is a perfect fit for spot capacity: spare cloud machines sold cheaply that the provider can take back at short notice. A lost job is just a retried chunk.
- The packager assembles encoded chunks into playable segments and writes the manifests.
- Metadata flips to ready, per rendition, so 480p can be live while 4K is still going.
- Playback: the viewer fetches segments from the nearest edge. Almost all of this never reaches your infrastructure.
- The manifest comes from your API, because it carries signed URLs and region rules and is the only per user part of playback.
- Heartbeats feed the view counting pipeline, entirely off the delivery path.
When an edge does not have a segment, it asks the origin shield, and only the shield reads object storage. The third deep dive explains why that middle step exists.
Step 4 says a lost transcode job is 'just a retried chunk'. What property of step 2 makes that true, and what would break if the splitter cut chunks at fixed byte offsets instead?
Step 3: Design deep dive#
Transcoding is a fan out problem, not a video problem#
First, the one piece of video knowledge this needs. Compressed video does not store every frame as a full picture. A keyframe is stored whole. The frames after it are stored as differences: "same as before, but the ball moved left". Think of a flipbook where only some pages are fully drawn and the rest say "like the last page, with these changes". You can only start reading at a fully drawn page. That is why a chunk must begin on a keyframe.
How do you turn one uploaded film into seven renditions, quickly?
- First tryOne machine runs ffmpeg over the whole file, seven timesffmpeg is the standard open source tool for converting video. Point it at the file once per rendition and wait.Where it breaks: A two hour film takes many hours of wall clock time on one machine, idle machines elsewhere cannot help, and a crash at 90% loses everything.
- BetterSeven machines, one rendition each, in parallelEvery rendition runs at the same time on its own machine, so you wait for one encode instead of seven in a row.Where it breaks: You still wait for the slowest single encode of the whole film, usually the 4K one. And if a cheap spot machine is taken back near the end, that whole rendition starts again.
- What you shipCut on keyframes, one job per chunk per renditionSplit into chunks of about thirty seconds, cut on keyframes. Each chunk times each rendition is its own small job: 1,680 for the two hour film. Encode the low renditions first and publish each rendition the moment it is complete. Under about a minute of video, skip the cutting and run one job.
Figure 2 plays the third rung one message at a time.
Idempotent means doing it twice has the same effect as doing it once. A job that writes its result to a fixed name is idempotent: run it again and it overwrites the same file. That is the "setting is safe to repeat" idea in idempotency keys, and it is what makes retries free here.
Two details worth saying unprompted. Transcode the low renditions first, because being watchable at 480p in ninety seconds is worth far more than being watchable at 1080p in six minutes. And keep the source forever: every new compression format, and there is always a new one, means running this whole pipeline again over the back catalogue.
"How would you handle a two second video and a four hour one in the same pipeline?" Chunking a two second file is pure overhead, so below some threshold you transcode it whole in one job. Above it, chunk. The threshold exists because the fixed cost of a job is roughly constant, so state it as a rule with a number attached rather than as a special case: below about a minute, one job, and above it, thirty second chunks.
The client decides, not the server#
Adaptive bitrate (ABR) means the quality changes during playback to match the network. It sounds like a server feature and is almost entirely a client one. The server publishes a manifest describing what exists. The player measures how fast its last segment arrived and how much video it has buffered (downloaded but not yet shown), and picks the next segment. It is a driver changing gear for the road, rather than someone at head office choosing the gear for every car.
How does the video keep playing when the network gets worse?
- First tryOne file per video, at one qualityEncode a single 1080p file and let everyone download it.Where it breaks: A phone on 3G cannot download it as fast as it plays, so it freezes. A 4K television gets a blurry picture. One size fits nobody.
- BetterSeveral renditions, and the server picks for each viewerA session server tracks every viewer and streams the quality it thinks is right.Where it breaks: Every response is now personal, so nothing can be cached at the edge, and millions of viewers need session state somewhere. You lose the one property that makes this affordable.
- What you shipRenditions in a manifest, and the player picks each segmentEvery segment of every rendition is an ordinary file with its own URL. The player reads the manifest, watches its own buffer and download speed, and chooses. The server keeps no state per viewer at all. HLS and DASH, the two common streaming formats, both work this way.
Try it. The same train journey, first locked to 1080p, then with the player choosing.
The two rules the player followed are worth naming, because they show you have debugged a player. Start at a low rendition and climb, since a fast first segment matters more than a beautiful one. And switch down at once but up slowly, because an unnecessary downgrade is invisible and a rebuffer is not.
The consequence to state explicitly: because the client decides, every segment is an ordinary file that never changes, with a URL. That is why a CDN can hold 99% of your traffic, and that is why the egress bill is survivable. Statelessness is not an aesthetic preference here, it is the business model. (Each segment is a plain HTTP download. How those bytes travel underneath is in TCP vs UDP vs QUIC.)
Getting the bytes near the user#
At tens of terabits a second, the only question that matters is what fraction is served from cache. A TTL (time to live) is how long a cache may keep a copy before checking again. The caching patterns chapter shows where the CDN sits among the other cache layers.
How do you serve tens of terabits a second?
- First tryServe segments straight from object storageThe player fetches every segment from the origin.Where it breaks: Every byte of the egress leaves your own machines, and a viewer on the other side of the world waits on a long round trip for every segment.
- BetterCDN edges in front, segments cached for a yearSegments never change, are addressed by URL, are a few megabytes each, and popular ones are extremely popular. Set a one year TTL and never invalidate: if a video changes, it gets new URLs.Where it breaks: The long tail. Most videos have almost no views, so the first viewer near each edge always misses. With hundreds of edge sites, one unpopular video can send hundreds of origin fetches for the same segment.
- What you shipCDN edges, plus an origin shield in the middleEdges fetch through one middle tier, the origin shield, instead of going to the origin directly. Three hundred edge misses for the same segment become one origin fetch. At the very largest scale, the caches move inside the internet providers themselves.
A shield is one person doing the grocery run for the whole office, instead of three hundred people each driving to the same shop for the same thing. This kind of CDN, which fetches a file the first time someone asks and keeps it, is a pull CDN, compared with the push kind in push vs pull.
Netflix ships physical caching appliances into internet providers' own data centres, so the popular catalogue is already inside the network your customer is connected to before they press play. YouTube does the same thing with its own edge nodes. The interview version of this insight is short and worth saying: at this scale, delivery stops being a software problem and becomes a question of where you put hardware.
Break it#
Read every step as text
- normal evening · Healthy. 99% hit ratio at the edge. Origin sees a trickle, mostly the long tail, and the whole system is doing what it was built to do: almost nothing.
- new release · Healthy. A popular video drops. The first request near each edge misses, so the shield takes a burst of a few hundred fetches, then everything is warm. This is the shield earning its place: without it that burst hits origin from every edge independently.
- catalogue refresh · Degraded. A move to a new compression format changes URLs across the library at once. Every cache is cold at the same time and the hit ratio collapses from 99% to near zero, which multiplies origin load by a hundred. Roll a change like this out over days, per region, and never in one push.
- edge site down · Degraded. A whole edge location fails. Traffic reroutes to neighbouring sites which are cold for that population, so latency rises and origin traffic doubles for a while. Survivable, and it is why you never run an edge tier at more than about 60% of capacity.
- repaired · Healthy. Caches refill within minutes because segments never change and popularity is concentrated. The recovery property of this design is that it needs no coordination: every cache independently pulls what its own users ask for.
The catalogue refresh state is the one worth raising unprompted, because it is a self inflicted outage that looks like a routine deploy, and every candidate who has lived through one mentions it before being asked. The cure is the same idea as a gradual code rollout, a little at a time and watched, which blue-green vs canary vs rolling covers for deploys.
Trade-offs#
| Choice | What you gain | What you pay | Pick it when |
|---|---|---|---|
| Chunked parallel transcoding | Minutes instead of hours, and interruptible machines become usable because a lost job is one chunk. | Cuts must land on keyframes, and quality across chunk boundaries needs care. | Any file long enough that encoding it in one go is measured in hours. Below about a minute, do it in one job. |
| Client side adaptive bitrate | Every segment is a file that never changes, so the CDN serves nearly everything and the server holds no session state. | You give up central control of quality, and a badly written player degrades the experience with no server side fix. | On demand video, always. Server side selection only makes sense when you must control quality centrally, and it costs you cacheability. |
| Pre signed direct upload | Eight gigabyte files never pass through your API tier, and resumability is free. | Less control over what arrives, so validation moves to a probe step after the fact. | Any upload above a few megabytes. Proxying large files through application servers is a habit worth breaking. |
| Origin shield in front of storage | Hundreds of edge misses for the same object collapse into one origin fetch. | Another tier to operate, and one more hop of latency on a cold read. | Any large catalogue with many edge locations and a long tail, which is every video service. |
The level bar#
One moment from this round, answered three ways. Pick the one you think is staff level before the labels appear. (What changes at senior and staff explains the gap.)
InterviewerYou are moving every video to a new compression format. How do you roll it out?
Show which is which without guessing
- Answer A: Senior. Knows why a single push is dangerous and stages it. Treats the whole library as one block of work.
- Answer B: Mid level. Every cache goes cold at the same moment, the hit ratio falls from 99% to near zero, and origin load goes up a hundred times.
- Answer C: Staff. Names the real risk, orders the work by where the money is, and makes the metric that matters the gate.
Then run the question cold, with a 45 minute timer and no notes, and score the run on the five things this question is really testing.
- Two systems
- Transcoding
- Who picks the quality
- The bill
- Cold caches
Interview replay#
Checkpoint#
1. Why must the splitter cut chunks on keyframe boundaries?
2. What does client side bitrate selection actually buy the system?
3. Same design, but now it is live streaming with a five second delay target. What breaks first?
- Egress is the bill. Storage is cheap and transcoding is a one time cost per video.
- A 95% hit ratio instead of 99% is five times the origin traffic. Hit ratio is the metric that moves money.
- A two hour film is 240 thirty second chunks, and at 7 renditions that is 1,680 independent jobs.
- Six second segments that never change, with a one year TTL. Anything that changes their URLs is a self inflicted outage.
Video on demand is two systems. The upload pipeline runs once per video and is allowed to take minutes: the client uploads parts directly to object storage with pre signed URLs, a splitter probes the file and cuts it on keyframes into thirty second chunks, and each chunk times each rendition becomes an independent job on a large spot fleet. Low renditions encode first and each rendition publishes as it completes, so a two hour film is watchable at 480p in about two minutes. The playback path is deliberately boring. The player fetches a manifest, which is the only per user part, and then pulls segments that never change from the nearest CDN edge, choosing quality itself from its own buffer and throughput. That client side decision is what keeps every segment cacheable, and cacheability is the whole economic model, because egress at tens of terabits a second is the bill and storage is not. An origin shield collapses hundreds of cold edge misses into one. The failure I would call out is anything that changes segment URLs across the catalogue at once, because it takes the hit ratio from ninety nine percent to zero and multiplies origin load by a hundred.