Full Stack • Java • System Design • Cloud • AI Engineering

YouTube System Design - 1 Hour Interview Guide

Design a scalable video streaming platform like YouTube. Covers requirements, capacity estimation, video upload pipeline, transcoding, adaptive bitrate streaming, CDN, search, recommendations, database design, caching, and trade-offs for a 1-hour system design interview.

1-Hour Interview Roadmap

Time Topic
0 – 5 min Requirements clarification
5 – 10 min Capacity estimation
10 – 18 min High-level architecture + core components
18 – 30 min Video upload + transcoding pipeline
30 – 40 min Video streaming — HLS, CDN, adaptive bitrate
40 – 48 min Database design + search
48 – 55 min Caching + scaling + recommendations
55 – 60 min Trade-offs + failure scenarios

What Are We Building?

A video upload, processing, and streaming platform where users can:

  • Upload videos from any device
  • Watch videos in adaptive quality based on network speed
  • Search for videos by title, tags, or description
  • Subscribe to channels and receive new video notifications
  • Like, dislike, comment on, and share videos
  • Receive personalized video recommendations

Scale reference: YouTube has 2.5 billion monthly active users, 500 hours of video uploaded every minute, 1 billion hours watched per day, and delivers video globally in under 2 seconds of startup latency.

YouTube's unique design challenge: unlike social feeds or messaging, videos are large binary assets (gigabytes each) that must be processed, stored in multiple formats, and streamed efficiently at massive scale.

Step 1 — Requirements

Functional Requirements

# Requirement
1 Users can upload videos (any format, up to 12 hours / 256 GB)
2 Uploaded videos are transcoded into multiple resolutions for adaptive streaming
3 Users can stream videos with adaptive bitrate (quality adjusts to network speed)
4 Users can search videos by keyword, title, tag, or channel name
5 Users can like, dislike, comment on, and share videos
6 Users can subscribe to channels and see subscription feeds
7 Users receive personalized video recommendations on the home page
8 Creators see view counts, watch time, and engagement analytics
9 Videos support captions/subtitles and chapters

Non-Functional Requirements

# Requirement
1 Video playback starts within 2 seconds (time-to-first-frame)
2 No buffering on stable connections — seamless adaptive streaming
3 High availability — 99.99% uptime for video playback
4 Video upload durable once ACKed — no data loss
5 Transcoding completes within 5 minutes for standard HD videos
6 Search results returned within 500ms
7 Read-heavy — 300:1 watch-to-upload ratio
8 Global delivery via CDN — serve from edge nodes closest to viewer

Out of Scope

  • YouTube Shorts algorithm
  • Live streaming architecture
  • YouTube Premium / DRM
  • Ads serving pipeline
  • Creator monetization

Step 2 — Capacity Estimation

Assumptions

Metric Value
Daily Active Users (DAU) 1 billion
Monthly Active Users (MAU) 2.5 billion
Videos uploaded per day 720,000 (500/min)
Average raw video size 2 GB
Average compressed video size 400 MB
Transcoded resolutions per video 5 (240p–4K)
Average views per video per day 1,000
Video watch time per user per day 60 minutes
Read-to-write ratio ~300:1

Storage Estimation

Raw uploads:   720K/day × 2 GB         = 1.44 PB/day
Transcoded:    720K/day × 400 MB × 5   = 1.44 PB/day (with all resolutions)
Thumbnails:    720K/day × 100 KB       = ~72 GB/day

Total new storage per day:  ~3 PB
5-year storage:             ~5.5 EB

Bandwidth Estimation

Watch requests: 1B DAU × 60 min/day × 5 Mbps avg
= ~375 TB/hour outgoing bandwidth

CDN serves 95% of this → origin bandwidth = ~18 TB/hour

Transcoding Throughput

720K videos/day
= ~8.3 videos/second uploaded
= ~40 transcode jobs/second (5 resolutions each)
= requires ~200+ parallel transcoding workers

Key Insight

YouTube is a read-heavy media delivery problem, not a data management problem. The critical path is:
upload → transcode → CDN distribution → viewer playback.
The bottleneck is transcoding throughput and CDN cache hit rate, not database reads.

Step 3 — High-Level Architecture

flowchart TD
    Creator --> AG[API Gateway\nAuth + Rate Limiting]
    Viewer --> AG

    AG --> UPS[Upload Service]
    AG --> VMS[Video Metadata Service]
    AG --> SS[Search Service]
    AG --> CS[Comment Service]
    AG --> RS[Recommendation Service]

    UPS --> S3R[Raw Video Storage\nS3]
    S3R --> KF[Kafka\nvideo.uploaded events]
    KF --> TC[Transcoding Service\nFFmpeg Workers]
    TC --> S3T[Processed Video Storage\nS3]
    S3T --> CDN[CDN\nEdge Delivery]

    TC -->|update status| VMS
    VMS --> VDB[(Video Metadata DB\nPostgreSQL)]
    VMS --> VC[(Video Cache\nRedis)]

    SS --> ES[(Elasticsearch\nVideo Index)]
    CS --> CDB[(Comment Store\nCassandra)]
    RS --> RDB[(Recommendation Store\nRedis + BigQuery)]

    Viewer -->|stream| CDN
    CDN -->|cache miss| S3T

Component Responsibilities

Component Responsibility
API Gateway Auth, rate limiting, routing all client requests
Upload Service Accept video file, issue pre-signed S3 URL, trigger transcoding pipeline
Transcoding Service Convert raw video to multiple resolutions + HLS segments; generate thumbnails
Video Metadata Svc Store and serve video title, description, status, URLs, counts
Search Service Full-text search over video metadata via Elasticsearch
Comment Service Append-only comment storage and retrieval
Recommendation Svc Serve personalized video recommendations based on watch history
CDN Cache and deliver video segments globally from edge nodes
Raw Video Storage Durable object storage for uploaded raw files (pre-transcoding)
Processed Storage Object storage for HLS segments and manifests (post-transcoding)

Step 4 — Video Upload Pipeline

The Problem

A raw video upload can be 2 GB or larger. Routing this through your application servers would:

  • Saturate server memory and network
  • Block the server thread for minutes
  • Fail if the connection drops mid-upload

Upload Flow

sequenceDiagram
    participant Creator
    participant UPS as Upload Service
    participant VDB as Metadata DB
    participant S3 as Raw S3 Storage
    participant KF as Kafka

    Creator->>UPS: POST /videos/initiate {title, description, tags}
    UPS->>VDB: INSERT video {status: PROCESSING}
    VDB-->>UPS: video_id
    UPS->>S3: Generate pre-signed upload URL
    UPS-->>Creator: {video_id, upload_url, expires_in: 15min}

    Creator->>S3: PUT (direct multipart upload to S3)
    S3-->>Creator: Upload complete
    Creator->>UPS: POST /videos/{video_id}/complete

    UPS->>VDB: UPDATE status = UPLOADED
    UPS->>KF: Publish video.uploaded {video_id, raw_s3_key}

Multipart Upload for Large Files

flowchart LR
    Raw[2 GB Video] --> P1[Part 1\n100 MB]
    Raw --> P2[Part 2\n100 MB]
    Raw --> P3[Part 3\n100 MB]
    Raw --> PN[Part N\n...]
    P1 & P2 & P3 & PN -->|parallel upload| S3[S3 Multipart]
    S3 -->|CompleteMultipartUpload| S3_DONE[Single S3 Object]

Benefits of multipart:

  • Each part uploaded concurrently — faster for large files
  • If one part fails, only that part is retried (not the full file)
  • S3 assembles all parts into one object on CompleteMultipartUpload

Why Pre-signed URLs?

  • Raw video bytes never flow through your backend — eliminates bottleneck
  • S3 handles the bandwidth — your Upload Service only processes metadata
  • Pre-signed URL expires in 15 minutes — security window limited

Step 5 — Transcoding Pipeline

Why Transcoding Is Essential

A viewer on mobile 3G cannot stream a 4K file at 50 Mbps. Transcoding converts one raw upload into multiple versions so every device and network condition is served optimally.

What Transcoding Produces

flowchart LR
    RAW[Raw Upload\n2 GB .mp4] --> TC[Transcoding Service\nFFmpeg]
    TC --> R1[240p / 400 Kbps\nHLS segments]
    TC --> R2[480p / 1 Mbps\nHLS segments]
    TC --> R3[720p / 2.5 Mbps\nHLS segments]
    TC --> R4[1080p / 5 Mbps\nHLS segments]
    TC --> R5[4K / 20 Mbps\nHLS segments]
    TC --> TH[Thumbnail\nJPEG 1280×720]
    TC --> MF[Master Manifest\nvideo.m3u8]
    R1 & R2 & R3 & R4 & R5 & MF --> S3T[Processed S3\nStorage]
    S3T --> CDN[CDN]

Transcoding Pipeline Steps

flowchart TD
    KF[Kafka\nvideo.uploaded] --> TCW[Transcoding Worker\npoll job]
    TCW -->|download raw| S3R[Raw S3]
    TCW --> SEG[Split into\n10-sec segments]
    SEG --> PAR[Parallel Encode\nper resolution\nper segment]
    PAR --> MNF[Generate\nHLS Manifest\n.m3u8]
    MNF --> S3T[Store to\nProcessed S3]
    S3T --> VMS[Update Metadata Service\nstatus = LIVE\nresolution_urls updated]
    VMS --> CDN[CDN warms\nedge caches]

Why Parallel Encoding?

1-hour video → 360 × 10-second segments
Each segment encoded at 5 resolutions = 1,800 jobs

Sequential:  1,800 × 5 sec = 2.5 hours
Parallel (200 workers): ~1,800 / 200 = ~9 jobs each = ~45 seconds

Splitting the video into segments and encoding each segment in parallel on separate workers reduces a 2.5-hour sequential job to under 1 minute.

HLS — How Adaptive Streaming Works

HLS (HTTP Live Streaming) is the industry standard adaptive streaming protocol.

File Structure

video_abc/
├── master.m3u8          ← master manifest (lists all resolutions)
├── 240p/
│   ├── stream.m3u8      ← playlist for this resolution
│   ├── segment_001.ts
│   ├── segment_002.ts
│   └── ...
├── 720p/
│   ├── stream.m3u8
│   ├── segment_001.ts
│   └── ...
└── 1080p/
    └── ...

Master Manifest (master.m3u8)

#EXTM3U
#EXT-X-STREAM-INF:BANDWIDTH=400000,RESOLUTION=426x240
240p/stream.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=1000000,RESOLUTION=854x480
480p/stream.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=2500000,RESOLUTION=1280x720
720p/stream.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=5000000,RESOLUTION=1920x1080
1080p/stream.m3u8

How the Player Uses This

flowchart TD
    Player -->|1. Fetch master.m3u8| CDN
    CDN -->|Master manifest| Player
    Player -->|2. Choose starting quality| RES{Network\nspeed check}
    RES -->|Slow 3G| SEG240[Fetch 240p segments]
    RES -->|Good WiFi| SEG1080[Fetch 1080p segments]
    Player -->|3. Continuously monitor buffer| ABR{Buffer < 10s\nor download slow?}
    ABR -->|Yes| DOWN[Switch to lower resolution]
    ABR -->|No| SAME[Continue or upgrade resolution]

The player never downloads the full video. It fetches 10-second segments one at a time, switching quality based on current network speed — that is why YouTube keeps playing smoothly when your connection fluctuates.

Step 6 — Video Streaming Flow

What Happens When a Viewer Presses Play

sequenceDiagram
    participant Viewer
    participant VMS as Metadata Service
    participant CDN
    participant S3T as Processed S3

    Viewer->>VMS: GET /videos/{video_id}
    VMS-->>Viewer: {title, thumbnail, manifest_url: "cdn.yt.com/abc/master.m3u8"}

    Viewer->>CDN: GET master.m3u8
    CDN-->>Viewer: Master manifest (resolution list)

    Viewer->>CDN: GET 720p/stream.m3u8 (select initial quality)
    CDN-->>Viewer: Segment playlist

    loop Every 10 seconds
        Viewer->>CDN: GET 720p/segment_001.ts
        CDN-->>Viewer: Segment data (served from edge)
    end

    Note over CDN,S3T: On cache miss: CDN fetches from S3, caches at edge

CDN Cache Strategy

flowchart LR
    Viewer -->|GET segment| CDN_EDGE[Nearest CDN Edge\ne.g. Mumbai PoP]
    CDN_EDGE -->|Cache HIT| Viewer
    CDN_EDGE -->|Cache MISS| CDN_ORIGIN[CDN Origin\nUS-East PoP]
    CDN_ORIGIN -->|Still MISS| S3[Processed S3\nus-east-1]
    S3 --> CDN_ORIGIN
    CDN_ORIGIN --> CDN_EDGE
    CDN_EDGE --> Viewer
Content Type CDN TTL Reasoning
Video segments (.ts) 30 days Immutable — never change after transcoding
HLS manifests (.m3u8) 5 minutes Can be updated (e.g. new resolutions added)
Thumbnails 7 days Occasionally updated by creator
Video metadata 60 seconds Like/view counts change frequently

CDN Hit Rate Optimization

Popular videos get millions of plays — their segments are cached everywhere. Long-tail videos (uploaded once, watched 10 times) should not waste CDN capacity.

Tiered strategy:

  • Viral/popular videos → Pre-warm CDN edge nodes in all regions
  • Average videos → Cache on demand (pull-through caching)
  • Long-tail videos → Serve from regional S3 + single CDN tier

Step 7 — Database Design

Database Selection

Data Database Reason
Video metadata PostgreSQL Structured, relational, complex queries (search by tag, date)
User profiles / channels PostgreSQL ACID, strong consistency
Comments Cassandra Append-only, high write volume, time-ordered
Watch history Cassandra Per-user time-series, very high volume
Like / dislike Cassandra High write volume, idempotent (one per user per video)
Subscriptions Cassandra Wide rows — (channel_id, subscriber_id)
Search index Elasticsearch Full-text, tags, NRT indexing
Recommendations Redis + BigQuery Real-time serving (Redis) + offline model training (BigQuery)
View count / like count Redis Counter — async flush to PostgreSQL

Video Metadata Table (PostgreSQL)

Table: videos

  video_id          BIGINT        PRIMARY KEY  (Snowflake)
  channel_id        BIGINT        NOT NULL  FK → channels.channel_id
  title             VARCHAR(200)  NOT NULL
  description       TEXT
  tags              TEXT[]
  duration_seconds  INT
  visibility        VARCHAR       (PUBLIC | UNLISTED | PRIVATE)
  status            VARCHAR       (PROCESSING | TRANSCODING | LIVE | FAILED | DELETED)
  raw_s3_key        TEXT          (raw upload path)
  manifest_url      TEXT          (CDN URL to master.m3u8)
  thumbnail_url     TEXT
  resolution_urls   JSONB         {"240p": "...", "480p": "...", "720p": "...", "1080p": "..."}
  view_count        BIGINT        DEFAULT 0
  like_count        BIGINT        DEFAULT 0
  dislike_count     BIGINT        DEFAULT 0
  comment_count     BIGINT        DEFAULT 0
  language          VARCHAR(10)
  uploaded_at       TIMESTAMP
  published_at      TIMESTAMP

Comment Table (Cassandra)

Table: comments

Partition key:  video_id       BIGINT
Clustering key: comment_id     BIGINT DESC  (Snowflake — newest first)

Columns:
  user_id        BIGINT
  text           TEXT
  like_count     COUNTER
  is_deleted     BOOLEAN
  created_at     TIMESTAMP

PRIMARY KEY (video_id, comment_id)

Query: SELECT * FROM comments WHERE video_id = ? ORDER BY comment_id DESC LIMIT 20

Watch History Table (Cassandra)

Table: watch_history

Partition key:  user_id        BIGINT
Clustering key: watched_at     TIMESTAMP DESC

Columns:
  video_id       BIGINT
  watch_percent  FLOAT    (how much of the video was watched)
  last_position  INT      (seconds — for resume playback)

PRIMARY KEY (user_id, watched_at)

Like / Dislike Table (Cassandra)

Table: video_reactions

Partition key:  video_id    BIGINT
Clustering key: user_id     BIGINT

Columns:
  reaction     VARCHAR  (LIKE | DISLIKE)
  created_at   TIMESTAMP

PRIMARY KEY (video_id, user_id)

Unique per (video_id, user_id) — UPSERT pattern prevents double likes.

Channel and Subscription Tables (PostgreSQL)

Table: channels
  channel_id      BIGINT    PRIMARY KEY
  user_id         BIGINT    FK → users
  channel_name    VARCHAR   UNIQUE
  subscriber_count BIGINT   DEFAULT 0
  created_at      TIMESTAMP

Table: subscriptions
  subscriber_id   BIGINT
  channel_id      BIGINT
  subscribed_at   TIMESTAMP
  PRIMARY KEY (subscriber_id, channel_id)

Step 8 — Search

Requirements

  • Search by title, description, tags, channel name
  • Results ranked by relevance + popularity (views, likes, watch time)
  • Autocomplete on search queries
  • Filter by upload date, duration, video type

Search Architecture

flowchart LR
    KF[Kafka\nvideo.published] --> IDX[Search Indexer]
    IDX --> ES[Elasticsearch\nVideo Index]

    Viewer -->|query: "system design"| SS[Search Service]
    SS -->|bool + function_score query| ES
    ES -->|Top 50 results| SS
    SS -->|Enrich from Redis cache| VC[(Redis\nVideo Cache)]
    SS --> Viewer

Elasticsearch Video Document

{
  "video_id":       12345678,
  "channel_id":     999,
  "channel_name":   "CodeWithVenu",
  "title":          "System Design Interview Guide",
  "description":    "Learn how to ace system design...",
  "tags":           ["system design", "interview", "backend"],
  "duration":       3600,
  "view_count":     4200000,
  "like_count":     180000,
  "language":       "en",
  "published_at":   "2025-01-15T10:00:00Z",
  "thumbnail_url":  "https://cdn.yt.com/thumb/12345678.jpg"
}

Ranking Formula

relevance_score = BM25(query, title, description, tags)
                × log(view_count + 1)            ← popularity boost
                × log(like_count + 1)            ← engagement boost
                × recency_factor(published_at)   ← newer content boosted
                × watch_time_ratio               ← % of video watched on avg

Step 9 — Caching Strategy

Redis Cache Map

Cache Key Pattern TTL Contents
Video metadata video:{video_id} 1 hr Full video object (title, manifest_url, counts)
View count views:{video_id} Eventual Counter — async flush to PostgreSQL
Like count likes:{video_id} Eventual Counter
User watch history history:{user_id} 24 hrs Last 100 watched video IDs
Recommendation list recs:{user_id} 30 min Pre-computed list of 20 video IDs
Channel profile channel:{channel_id} 1 hr Channel metadata + subscriber count
Trending videos trending:{region} 10 min Sorted set — video_id → trending score
Search autocomplete suggest:{prefix} 1 hr Top 10 completions for typed prefix

View Count at Scale

YouTube receives ~5 million view events per minute on popular videos. Row-level incrementing a PostgreSQL column would cause catastrophic lock contention.

Solution:

flowchart LR
    ViewEvent -->|INCR views:{video_id}| RC[(Redis)]
    RC -->|every 30 seconds| BATCH[Batch Flush Worker]
    BATCH -->|UPDATE view_count = view_count + delta| PG[(PostgreSQL)]
    PG -->|sync| ES2[(Elasticsearch\nview_count updated)]
  1. View event hits Redis INCR — atomic, sub-millisecond
  2. Batch flush worker reads and resets Redis counter every 30 seconds
  3. Applies the delta to PostgreSQL in a single UPDATE statement
  4. Elasticsearch updated periodically for search ranking

Step 10 — Recommendations

Why Recommendations Are Critical

70% of YouTube watch time comes from the recommendation system. A viewer who clicks a video is shown 20 suggestions on the right sidebar and in autoplay. Getting these right drives engagement.

Recommendation Pipeline

flowchart TD
    WH[Watch History\nLikes\nSearch Queries] --> ML[Offline ML Training\nBigQuery + TensorFlow]
    ML -->|User embedding vectors\nVideo embedding vectors| VS[(Vector Store\nFAISS / Pinecone)]

    Viewer -->|GET /recommendations| RS[Recommendation Service]
    RS -->|Lookup user vector| VS
    VS -->|Nearest neighbor search| CANDS[Top 500 candidate videos]
    CANDS --> RANK[Re-ranking\nML model\napply context + freshness]
    RANK --> TOP20[Top 20 recommendations]
    TOP20 --> RC[(Redis\nrecs:{user_id})]
    RC --> Viewer

Two Stages

Stage Method Candidates In Out Goal
Retrieval Approximate nearest neighbor All videos 500 Find broadly relevant candidates fast
Ranking Deep neural network 500 20 Rank by predicted watch probability

Retrieval uses collaborative filtering — "users who watched what you watched also liked these videos."
Ranking uses the candidate videos' features + user context (time of day, device, recent watches) to predict which 20 the user will most likely watch next.

Step 11 — Scaling

Upload Scaling

flowchart LR
    Creators --> S3[S3\nMulti-region\nDirect Upload]
    S3 --> KF[Kafka\nvideo.uploaded\n100+ partitions]
    KF --> TC1[Transcoding Worker 1]
    KF --> TC2[Transcoding Worker 2]
    KF --> TC3[Transcoding Worker N]
    TC1 & TC2 & TC3 --> S3T[Processed S3]
  • Upload workers are stateless — scale to 500+ pods during peak upload hours
  • Kafka partitioned by video_id — each video processed by one worker set
  • Transcoding workers auto-scale based on Kafka consumer lag

Streaming Scaling

flowchart TD
    Viewers --> CDN[CDN\nGlobal PoPs\n200+ locations]
    CDN -->|Cache hit ~95%| Viewer
    CDN -->|Cache miss 5%| S3T[Processed S3\nRegional]
  • CDN absorbs 95%+ of streaming traffic — origin (S3) sees only 5% of requests
  • Each CDN PoP (Point of Presence) independently serves local viewers
  • Add CDN capacity (new PoPs) to scale streaming — no backend changes needed

Database Scaling

Database Sharding Strategy
PostgreSQL Partition videos by uploaded_at range; shard by channel_id
Cassandra Consistent hashing — comments/likes partitioned by video_id
Elasticsearch Index shards across cluster nodes — shard by video_id range
Redis Redis Cluster — slot-based sharding

Transcoding Parallelism

720K videos/day = 8.3 uploads/second
Each video = 5 resolutions × ~100 segments = 500 tasks per video

8.3 × 500 = 4,150 transcoding tasks/second at peak
With 200-worker farm: each worker handles ~21 tasks/second

Workers are ephemeral containers (Kubernetes Jobs) — scale up/down with job queue depth.

Step 12 — Multi-Region Deployment

flowchart TD
    GLB[Global Load Balancer\nAnyCast DNS] --> US[US-East\nPrimary]
    GLB --> EU[EU-West]
    GLB --> APAC[APAC]
    GLB --> IN[India]

    US -->|async replication| EU
    US -->|async replication| APAC
    US -->|async replication| IN

    CDN_US[CDN US PoPs] --- US
    CDN_EU[CDN EU PoPs] --- EU
    CDN_APAC[CDN APAC PoPs] --- APAC
Concern Strategy
Video storage S3 geo-replication to 3+ regions
CDN coverage 200+ PoPs globally — viewers always within 50ms of an edge node
Metadata Primary in US-East; read replicas in each region
Upload Upload to nearest region; async replicate to primary
Transcoding Triggered in region where raw file was uploaded

Step 13 — Failure Scenarios

Failure Impact Mitigation
Transcoding worker crashes Video stuck in PROCESSING Kafka consumer group auto-rebalances; job retried from last offset
S3 region outage Uploads and playback fail for that region S3 geo-replication; CDN serves cached segments; fallback to another region
CDN PoP outage Viewers in that region get higher latency CDN routes to next nearest PoP automatically
Redis cache down View counts delayed; recommendations slow Redis Sentinel/Cluster failover; metadata falls back to PostgreSQL
Elasticsearch down Search returns 503 Graceful degradation — show trending/subscriptions instead
PostgreSQL primary down Metadata writes fail Read replica promotes to primary; minimal downtime with pgBouncer
Kafka down Transcoding jobs not dispatched Upload Service buffers job locally and retries; Kafka cluster RF=3
Recommendation service down Home feed shows generic trending Serve cached recommendations from Redis; fall back to trending list

Final Architecture

flowchart TD
    Creator --> AG[API Gateway]
    Viewer --> AG

    AG --> UPS[Upload Service]
    AG --> VMS[Video Metadata Service]
    AG --> SS[Search Service]
    AG --> CS[Comment Service]
    AG --> RS[Recommendation Service]

    UPS -->|pre-signed URL| S3R[Raw S3\nMulti-region]
    S3R --> KF[Kafka Cluster]
    KF --> TC[Transcoding Workers\n200+ pods]
    TC --> S3T[Processed S3\nMulti-region]
    S3T --> CDN[CDN\n200+ Global PoPs]

    TC -->|status update| VMS
    VMS --> PG[(PostgreSQL\nVideo Metadata)]
    VMS --> RC[(Redis Cluster\nCache + Counters)]

    SS --> ES[(Elasticsearch)]
    CS --> CDB[(Cassandra\nComments)]
    RS --> VEC[(Vector Store\nFAISS)]

    Viewer -->|stream| CDN
    CDN -->|5% cache miss| S3T

Technology Stack

Layer Technology
API Gateway NGINX / Envoy / AWS API Gateway
Upload Service Go / Python (handles pre-signed URL generation)
Transcoding FFmpeg on Kubernetes Jobs / AWS Elemental MediaConvert
Message queue Apache Kafka
Video storage (raw) Amazon S3 / Google Cloud Storage
Video storage (CDN) S3 → CloudFront / Fastly / Akamai
Metadata DB PostgreSQL (primary + read replicas)
Comments / likes Apache Cassandra
Metadata cache Redis Cluster
Search Elasticsearch
Recommendations TensorFlow + FAISS / Pinecone (vector search)
Recommendation cache Redis
Monitoring Prometheus + Grafana + ELK
Deployment Kubernetes (multi-region)

Key Trade-Offs

Decision Option A Option B Choice & Reason
Upload routing Through app servers Pre-signed S3 + direct upload Pre-signed S3 — app servers cannot handle 2 GB files at scale
Streaming protocol MP4 progressive download HLS adaptive bitrate HLS — adapts to network; CDN-optimized 10-sec segments
View count storage PostgreSQL row increment Redis counter + async flush Redis counter — 5M views/min would deadlock PostgreSQL
Search backend PostgreSQL full-text Elasticsearch Elasticsearch — NRT indexing, relevance scoring, autocomplete
Recommendation generation Real-time per-request Pre-computed + cached Pre-computed — ML inference at 1B users/day cannot be real-time per-request
Transcoding Sequential single worker Parallel segment-level workers Parallel — reduce 2.5 hours to under 1 minute
CDN cache TTL for segments Short TTL (1 hr) Long TTL (30 days) 30 days — segments are immutable; long TTL maximizes CDN hit rate
Database for comments PostgreSQL Cassandra Cassandra — append-only, time-ordered, high write throughput

YouTube vs Netflix — Key Design Differences

Dimension YouTube Netflix
Content model User-generated (UGC) Licensed + original content
Upload volume 500 hours/minute by millions of users Hundreds of titles per month by Netflix
Transcoding Fully automated pipeline Human-supervised encoding pipeline
DRM Optional (for paid content) Mandatory — Widevine, FairPlay, PlayReady
Recommendation Collaborative filtering (UGC cold start) Rich editorial + viewing signals
CDN strategy External CDN (Akamai, Fastly) Own CDN — Open Connect Appliances
Live streaming Yes (YouTube Live) No (VOD only)

Common Interview Mistakes

  • ❌ Routing large video uploads through your application servers
  • ❌ Not mentioning HLS / adaptive bitrate — this is the core streaming technology
  • ❌ Not explaining why transcoding into multiple resolutions is necessary
  • ❌ Forgetting CDN — serving video from a single S3 region to 1B users is impossible
  • ❌ Using PostgreSQL row-level locks for view counts at millions of events per minute
  • ❌ Sequential transcoding — a 1-hour video would take 2.5 hours to transcode
  • ❌ No search design — YouTube search is a distinct Elasticsearch-backed system
  • ❌ No recommendation architecture — 70% of YouTube views come from recommendations
  • ❌ No failure scenario for transcoding worker crash (common question)
  • ❌ Using offset pagination for comments — use cursor-based (?before_id=...)

Interview Questions

  1. Why are pre-signed S3 URLs used for video upload instead of routing through your backend?
  2. What is HLS and how does adaptive bitrate streaming work?
  3. Why is video transcoded into multiple resolutions?
  4. How do you transcode a 1-hour video in under 5 minutes?
  5. What is the role of the CDN in video streaming?
  6. How do you handle 5 million view events per minute without overloading the database?
  7. What is the difference between the raw video storage and the processed video storage?
  8. Why is Cassandra used for comments instead of PostgreSQL?
  9. How does YouTube's search work? What makes it different from SQL LIKE queries?
  10. How do video recommendations work at a high level?
  11. What happens if a transcoding worker crashes mid-job?
  12. How do you handle a CDN PoP outage for viewers in that region?
  13. How would you design a resume-playback feature (continue where you left off)?
  14. How do you deduplicate re-uploaded videos to save storage?
  15. How would you design the notification system for new uploads from subscribed channels?

Summary

Concern Solution
Video upload Pre-signed S3 URL → direct multipart upload from client
Transcoding Kafka → FFmpeg workers → parallel segment encoding → HLS output
Adaptive streaming HLS with 5 resolution tiers — player switches quality based on bandwidth
Video delivery CDN with 200+ PoPs — 95%+ cache hit rate; 30-day TTL on immutable segments
Video metadata PostgreSQL — structured, ACID, complex queries
Comments Cassandra — append-only, time-ordered, high write volume
View / like counts Redis counters — async batch flush to PostgreSQL every 30 seconds
Search Elasticsearch — NRT indexing, BM25 relevance, popularity and freshness boost
Recommendations Two-stage ML: vector retrieval (FAISS) + neural net re-ranking
Scale Stateless services + Kafka + parallel transcoding + CDN
Multi-region S3 geo-replication + CDN global PoPs + metadata read replicas per region

The core principle: YouTube is a video processing and delivery problem. Upload goes direct to S3. Transcoding is massively parallel. Streaming is CDN-first. The backend handles only metadata — never raw video bytes.

CodeWithVenu

Master Java • Spring Boot • System Design • Enterprise Architecture

Website: https://codewithvenu.com