Full Stack • Java • System Design • Cloud • AI Engineering

Blue-Green Deployment

A deep dive into blue-green deployment — how it works, when to use it, how to handle database migrations, traffic switching, rollback, Kubernetes implementation, and the trade-offs every production engineer should know.

Introduction

Blue-green deployment is a release strategy that maintains two identical production environments running in parallel.

At any point in time, only one environment — Blue — is receiving live traffic. The other — Green — is idle and available for the next deployment.

When a new version is ready to release:

  1. The new version is deployed to the idle environment (Green)
  2. Green is tested and verified in isolation
  3. All traffic is switched from Blue to Green in a single atomic step
  4. Blue is retained as the immediate rollback target

If anything goes wrong after the switch, rollback is a single traffic switch back to Blue — taking seconds, not minutes.

This is the defining advantage of blue-green: instant, zero-risk rollback at any point after a release.

The Core Problem Blue-Green Solves

Every deployment strategy faces the same fundamental question: what happens if the new version is broken?

Strategy Rollback Mechanism Rollback Time
Recreate Redeploy previous image 5–15 minutes
Rolling Update Roll forward with fixed version 5–15 minutes
Canary Set traffic weight to 0% 1–3 minutes
Blue-Green Switch traffic back to idle environment < 60 seconds

Blue-green achieves the fastest rollback of any deployment strategy because the previous version is already running and fully warmed up. No containers need to start, no dependencies need to initialize.

Architecture

The Two Environments

flowchart TB
    DNS["DNS / Load Balancer\n(Controls traffic routing)"]

    subgraph BlueEnv["Blue Environment — LIVE"]
        B_GW["API Gateway"]
        B_S1["Service A: v1"]
        B_S2["Service B: v1"]
        B_S3["Service C: v1"]
    end

    subgraph GreenEnv["Green Environment — IDLE"]
        G_GW["API Gateway"]
        G_S1["Service A: v2"]
        G_S2["Service B: v2"]
        G_S3["Service C: v2"]
    end

    SharedDB[("Shared Database")]

    DNS -->|"100% traffic"| BlueEnv
    DNS -.->|"0% traffic"| GreenEnv

    BlueEnv --> SharedDB
    GreenEnv --> SharedDB

Both environments:

  • Are identical in configuration — same instance types, same networking, same secrets
  • Share the same database — schema must be backward compatible during the cutover window
  • Are independently deployable — Green is updated while Blue serves 100% of traffic

State Machine

Blue-green alternates which environment is live with each release.

flowchart LR
    S1["Release 1\nBlue: v1 LIVE\nGreen: idle"]
    S2["Deploy Release 2\nBlue: v1 LIVE\nGreen: v2 ready"]
    S3["Switch Traffic\nBlue: v1 idle\nGreen: v2 LIVE"]
    S4["Deploy Release 3\nBlue: v3 ready\nGreen: v2 LIVE"]
    S5["Switch Traffic\nBlue: v3 LIVE\nGreen: v2 idle"]

    S1 --> S2 --> S3 --> S4 --> S5

Each release cycle flips which colour is live. Blue and Green swap roles with every deployment.

The Complete Deployment Flow

sequenceDiagram
    participant Engineer
    participant Pipeline
    participant Green
    participant LoadBalancer
    participant Blue
    participant Database

    Note over Blue: Serving 100% live traffic

    Engineer->>Pipeline: Trigger release
    Pipeline->>Database: Run schema migration (expand phase)
    Pipeline->>Green: Deploy v2 to idle environment
    Pipeline->>Green: Run smoke tests
    Green-->>Pipeline: All tests pass

    Pipeline->>LoadBalancer: Switch 100% traffic → Green
    LoadBalancer-->>Green: Live traffic begins
    LoadBalancer-->>Blue: Traffic stops

    Note over Green: Serving 100% live traffic
    Note over Blue: Idle — ready for instant rollback

    Engineer->>Pipeline: Confirm release stable (after soak)
    Pipeline->>Blue: Scale down or prepare for next deploy

Traffic Switching Methods

How traffic is switched between Blue and Green depends on the infrastructure layer used.

DNS-Based Switching

Update the DNS record to point to the Green load balancer.

flowchart LR
    Client --> DNS["DNS Record\npayments.company.com"]
    DNS -->|"Before"| BlueLB["Blue Load Balancer\n10.0.1.100"]
    DNS -.->|"After switch"| GreenLB["Green Load Balancer\n10.0.2.100"]

Consideration: DNS TTL (Time To Live) determines how long clients cache the old address. Use a low TTL (30–60 seconds) ahead of the deployment window. DNS switching is not instant — it takes TTL time to propagate globally.

Load Balancer Target Group Switching

Switch at the load balancer level by changing which target group receives traffic.

flowchart TB
    Internet --> ALB["AWS Application Load Balancer"]
    ALB -->|"Listener Rule\n(before switch)"| BlueTG["Blue Target Group\n(v1 instances)"]
    ALB -.->|"Listener Rule\n(after switch)"| GreenTG["Green Target Group\n(v2 instances)"]

This is the preferred approach for AWS deployments:

  • Switch is instant — no DNS propagation delay
  • Single API call changes the listener rule
  • Rollback is the same operation in reverse
  • Health checks on Green target group must pass before switching

Kubernetes Service Selector Switch

In Kubernetes, traffic is controlled by pod label selectors. Switching is an update to the Service object.

flowchart TB
    Service["Kubernetes Service\n(payments-svc)"]

    subgraph Before["Before Switch"]
        Selector1["selector:\n  app: payments\n  version: blue"]
    end

    subgraph After["After Switch"]
        Selector2["selector:\n  app: payments\n  version: green"]
    end

    Service --> Before
    Service -.-> After

    Before --> BluePods["Blue Pods (v1)\n✅ Receiving traffic"]
    After --> GreenPods["Green Pods (v2)\n✅ Receiving traffic"]

Before deployment:

# Service selector points to Blue
selector:
  app: payments
  version: blue

After switch:

# Single patch command switches all traffic
selector:
  app: payments
  version: green

The switch takes effect immediately — Kubernetes updates the endpoint slice within seconds.

Database Strategy

The database is the most complex aspect of blue-green deployment. Both environments share one database, which means the schema must simultaneously support both versions during the transition.

Phase 1 — Expand (Before Switch)

Add the new schema elements without removing or modifying existing ones. Both v1 and v2 can operate against the same schema.

flowchart LR
    DB["Database"]
    DB --> OldCol["existing columns\n(v1 reads these)"]
    DB --> NewCol["new columns added\n(v2 reads these, v1 ignores)"]

Safe operations in this phase:

  • Add new nullable columns
  • Add new tables
  • Add indexes (using CONCURRENTLY)
  • Add new enum values

Phase 2 — Backfill (During Soak Period)

After the switch, v2 is live. Run background jobs to backfill data into new columns for existing rows.

sequenceDiagram
    participant V2App
    participant BackfillJob
    participant Database

    Note over V2App: Live — writing to both old and new columns
    BackfillJob->>Database: UPDATE rows SET new_col = derive(old_col) WHERE new_col IS NULL
    BackfillJob-->>Database: Batch complete
    Note over Database: All rows now have new_col populated

Phase 3 — Contract (Next Release Cycle)

Once all v1 instances are retired and the blue environment is confirmed decommissioned, clean up:

  • Remove old columns no longer read by any version
  • Add NOT NULL constraints now that all rows are backfilled
  • Drop compatibility views or shims
flowchart LR
    Phase1["Expand\nAdd new_col\n(nullable)"]
    Phase2["Backfill\nPopulate new_col\nfor all rows"]
    Phase3["Contract\nDrop old_col\nAdd NOT NULL"]

    Phase1 --> Phase2 --> Phase3

Schema Change Safety Table

Migration Type Safe During Switch? Notes
Add nullable column ✅ Yes v1 ignores the new column
Add new table ✅ Yes v1 is unaware of the table
Add index (CONCURRENTLY) ✅ Yes Non-blocking
Drop unused column ✅ Yes (post-retire) Only after v1 is fully decommissioned
Rename column ❌ No (directly) Use expand + contract across two releases
Change column type ❌ No (directly) Add new column, migrate, drop old
Remove NOT NULL constraint ✅ Yes Relaxing constraints is safe
Add NOT NULL constraint ❌ No (directly) Backfill nulls first, then add constraint

Rollback

Rollback in blue-green is the fastest available for any deployment strategy.

sequenceDiagram
    participant OnCall
    participant Alerting
    participant LoadBalancer
    participant Blue
    participant Green

    Note over Green: Live — serving 100% traffic
    Alerting->>OnCall: Error rate spike — 5% on /payments
    OnCall->>LoadBalancer: Switch traffic back to Blue
    LoadBalancer-->>Blue: 100% traffic (< 5 seconds)
    LoadBalancer-->>Green: 0% traffic

    Note over Blue: Fully restored
    Note over Green: Idle — debugging begins

Rollback properties:

  • Blue environment is already warm — no cold start penalty
  • All Blue connections to the database are pre-established
  • Blue pods are in Running state — not Pending or ContainerCreating
  • SLO recovery time: typically under 60 seconds from decision to restored traffic

What rollback does NOT undo:

  • Database schema expansions (new nullable columns remain — they are backward compatible)
  • Data written by v2 (new columns populated by v2 remain — v1 ignores them)
  • Audit logs created during the v2 window

These are safe to leave in place. The contract phase of the next release cycle will clean them up.

Blue-Green with Databases — Full Timeline

flowchart TB
    T1["T-1: Expand migration\nDeploy schema additions\n(new columns, new tables)"]
    T2["T0: Deploy v2 to Green\nRun smoke tests"]
    T3["T0+5min: Switch traffic\nGreen becomes live"]
    T4["T0+30min: Soak period\nMonitor SLOs on Green"]
    T5["T0+60min: Confirm stable\nBlue kept for rollback window"]
    T6["T+24hrs: Decommission Blue\nBegin contract phase"]
    T7["Next release: Contract\nDrop old columns, add constraints"]

    T1 --> T2 --> T3 --> T4 --> T5 --> T6 --> T7

Never decommission Blue immediately after switching. Keep Blue running for at least 1–2 hours to preserve the instant rollback capability through the highest-risk window.

Blue-Green vs Canary

These two strategies are often compared. They solve different problems.

flowchart LR
    subgraph BlueGreen["Blue-Green"]
        BG1["100% traffic on v1"]
        BG2["Switch → 100% on v2"]
        BG1 --> BG2
    end

    subgraph Canary["Canary"]
        C1["100% on v1"]
        C2["5% on v2, 95% on v1"]
        C3["50% on v2, 50% on v1"]
        C4["100% on v2"]
        C1 --> C2 --> C3 --> C4
    end
Dimension Blue-Green Canary
Traffic exposure All-at-once switch Gradual percentage shift
Rollback speed Instant (< 60s) Instant (set to 0%)
Blast radius if broken 100% of users briefly Small % of users for longer
Infrastructure cost 2x during deployment window Small overhead for canary pods
Best for Clean cutover, full environment test High-traffic, risk-sensitive changes
Real traffic validation After 100% switch Before full rollout

Combined pattern: Some teams use blue-green for the environment switch and canary for the traffic shift — deploy to Green, shift 5% → 100% while keeping Blue as the rollback target.

Kubernetes Implementation

A complete blue-green setup in Kubernetes using two Deployments and one Service.

Blue Deployment (current live)

apiVersion: apps/v1
kind: Deployment
metadata:
  name: payments-blue
  labels:
    app: payments
    version: blue
spec:
  replicas: 4
  selector:
    matchLabels:
      app: payments
      version: blue
  template:
    metadata:
      labels:
        app: payments
        version: blue
    spec:
      containers:
      - name: payments
        image: payments:v1.4.2

Green Deployment (new version)

apiVersion: apps/v1
kind: Deployment
metadata:
  name: payments-green
  labels:
    app: payments
    version: green
spec:
  replicas: 4
  selector:
    matchLabels:
      app: payments
      version: green
  template:
    metadata:
      labels:
        app: payments
        version: green
    spec:
      containers:
      - name: payments
        image: payments:v1.5.0

Service (controls which version receives traffic)

apiVersion: v1
kind: Service
metadata:
  name: payments-svc
spec:
  selector:
    app: payments
    version: blue   # Change to "green" to switch traffic
  ports:
  - port: 80
    targetPort: 8080

Traffic Switch Command

# Switch to Green
kubectl patch service payments-svc \
  -p '{"spec":{"selector":{"app":"payments","version":"green"}}}'

# Rollback to Blue
kubectl patch service payments-svc \
  -p '{"spec":{"selector":{"app":"payments","version":"blue"}}}'

Monitoring During a Blue-Green Release

After switching traffic to Green, watch these signals for the soak period (minimum 15–30 minutes):

flowchart LR
    Switch["Traffic → Green"]
    Switch --> M1["Error rate\n(compare vs Blue baseline)"]
    Switch --> M2["p99 latency\n(must not regress)"]
    Switch --> M3["Business metrics\n(payment success rate)"]
    Switch --> M4["JVM / Runtime\n(memory, GC, threads)"]
    Switch --> M5["Database\n(connection pool, query latency)"]

    M1 --> Decision{"All signals\nhealthy?"}
    M2 --> Decision
    M3 --> Decision
    M4 --> Decision
    M5 --> Decision

    Decision -->|"Yes"| Confirm["Confirm release\nDecommission Blue"]
    Decision -->|"No"| Rollback["Rollback to Blue\n< 60 seconds"]

Key comparisons to make:

Metric Blue (baseline) Green (post-switch) Action if regressed
Error rate (5xx) 0.02% < 0.05% Rollback immediately
p99 latency 180ms < 220ms Investigate
Payment success rate 99.8% > 99.7% Rollback if declining
DB connection pool 40% utilised < 70% Investigate

When to Use Blue-Green

Strong fit:

  • Regulated systems (banking, healthcare) requiring a clean, auditable cutover
  • Major version releases with significant behaviour changes
  • High-value services (payments, auth) where instant rollback must be available
  • Full environment testing is required before any production traffic touches the new version
  • Compliance requirements mandate that the previous version be immediately restorable

Weak fit:

  • Services with very frequent deploys (multiple times per hour) — infrastructure cost becomes significant
  • Systems that cannot afford 2x resource cost even temporarily
  • Services that benefit more from gradual traffic exposure (use canary instead)
  • Environments with strict database coupling that makes backward-compatible migrations impractical

Production Readiness Checklist

Environment Setup

  • Blue and Green environments are identical in configuration (instance type, memory, CPU)
  • Both environments share the same database with backward-compatible schema
  • Health checks are passing on Green before any traffic switch
  • Smoke tests pass on Green against production database

Traffic Switching

  • Load balancer or service selector switch mechanism is tested in staging
  • Rollback switch command is documented and tested
  • DNS TTL is set low ahead of deployment (if DNS-based switching)
  • Traffic switch takes effect within 60 seconds

Database

  • Schema migration is expand-only (no destructive changes before switch)
  • Backfill jobs are idempotent and can be re-run safely
  • Contract phase is scheduled for the release after v1 is decommissioned

Monitoring

  • Dashboards are open before switch with Blue baseline captured
  • Alert thresholds defined for error rate and latency
  • Soak period defined (minimum 15 minutes post-switch before confirming)
  • Rollback decision criteria documented (what metric threshold triggers rollback)

Post-Switch

  • Blue retained for minimum 1-hour rollback window
  • Blue scaled down only after stability confirmed
  • Contract migration scheduled for next release cycle
  • Release documented in change log

Summary

Blue-green deployment is the gold standard for high-confidence, zero-downtime releases where instant rollback is a hard requirement.

flowchart LR
    Deploy["Deploy v2\nto Green\n(isolated)"]
    Test["Test Green\n(smoke tests\nfull env)"]
    Switch["Switch traffic\nBlue → Green\n(atomic)"]
    Monitor["Soak period\n(15–30 min)"]
    Confirm["Confirm stable\nRetire Blue"]

    Deploy --> Test --> Switch --> Monitor --> Confirm
    Monitor -->|"Issue detected"| Rollback["Rollback\nGreen → Blue\n< 60 seconds"]

Key principles:

  • Blue is always your rollback — never decommission it until stability is confirmed
  • Database migrations must be backward compatible — expand first, contract later
  • Switch at the load balancer level — not DNS, to avoid propagation delays
  • Soak before confirming — monitor SLOs for 15–30 minutes before decommissioning Blue
  • Smoke test Green before switching — never switch to an untested environment