Cost Optimization

Introduction

Cloud infrastructure costs grow faster than almost any other engineering expense. A system built under a startup's traffic load will cost 10× more when it reaches scale — not because the architecture changed, but because no one was watching.

Cost optimization is not about being cheap. It is about spending money where it creates value and eliminating spend where it does not.

A production system that is expensive because it is fast, reliable, and scalable is well-designed. A production system that is expensive because instances are over-provisioned, data is stored in the wrong tier, and network egress is unmetered is poorly engineered — and it compounds every month.

Cloud cost optimization is the practice of continuously aligning infrastructure spend with business value: right-sizing resources, eliminating waste, matching pricing models to usage patterns, and building cost awareness directly into engineering culture.

---

The Cost Visibility Problem

You cannot optimize what you cannot measure. Most cost problems are discovered late because teams lack visibility into what is being spent, by what, and why.

flowchart LR
    Spend["Cloud Bill\n$50,000/month"]
    Spend --> Unknown["Untagged resources\n(40% of spend)"]
    Spend --> Identified["Tagged and attributed\n(60% of spend)"]

    Identified --> Team_A["Platform Team: $12,000"]
    Identified --> Team_B["API Team: $9,000"]
    Identified --> Team_C["Data Team: $8,000"]
    Identified --> Team_D["Frontend Team: $1,000"]

    Unknown --> Problem["Cannot optimize\nwhat you cannot see"]

The first step in cost optimization is always visibility: tag every resource, attribute every dollar to a team or service, and surface cost data to the engineers who create it.

Cost Attribution Tags

Every cloud resource should carry tags that answer three questions:

Tag Key	Example Value	Purpose
team	platform	Which team owns this resource
service	user-api	Which service does this resource serve
environment	production	Is this production, staging, or dev
cost-center	engineering	Which budget does this belong to

Without consistent tagging, cost allocation is guesswork. With it, every spike in the bill is traceable to a team and a service.

Cost Allocation by Layer

Cost Category	Typical % of Cloud Bill	Common Waste Sources
Compute (EC2, VMs)	40–60%	Over-provisioned instances, idle resources
Storage (S3, disks)	10–20%	Data never deleted, wrong storage class
Database (RDS, managed)	10–20%	Over-provisioned DB, unused read replicas
Network (egress, NAT)	5–15%	Cross-region traffic, uncompressed data
Managed services	5–15%	Unused queues, idle functions, orphaned LBs

---

Compute Cost Optimization

Compute is the largest cost category for most production systems and the one with the most levers to pull.

Rightsizing Instances

The most common form of compute waste: instances sized for peak load from two years ago, now running at 15% CPU utilization.

flowchart LR
    subgraph Before["Before Rightsizing"]
        B1["16 × m5.2xlarge\n(8 vCPU, 32GB RAM each)\nAvg CPU: 12%\nCost: $4,416/month"]
    end

    subgraph After["After Rightsizing"]
        A1["16 × m5.large\n(2 vCPU, 8GB RAM each)\nAvg CPU: 48%\nCost: $1,104/month"]
    end

    Before -->|"Rightsize based\non actual metrics"| After
    After --> Savings["Savings: $3,312/month\n($39,744/year)"]

Rightsizing process:

1. Collect 2–4 weeks of CPU, memory, and network utilization metrics
2. Identify instances running below 20–30% utilization
3. Target 60–70% average utilization after rightsizing (leave headroom for spikes)
4. Test with one instance type down before applying fleet-wide
5. Re-evaluate quarterly — workloads change

Utilization targets for rightsizing:

Metric	Too Low (Over-provisioned)	Target Range	Too High (Under-provisioned)
Average CPU	< 20%	40–70%	> 85%
Peak CPU	< 40%	60–85%	> 95%
Memory utilization	< 30%	50–75%	> 90%

Reserved Instances and Savings Plans

On-demand pricing is the most expensive way to run cloud infrastructure. For predictable, baseline workloads, committing to 1 or 3-year terms reduces cost by 30–72%.

flowchart TB
    Workload["Production Workload"]

    subgraph Pricing["Pricing Options"]
        OD["On-Demand\nNo commitment\nHighest price\n(Baseline cost: 100%)"]
        RI["Reserved Instances\n1-year: ~40% savings\n3-year: ~60% savings\nCommit to instance type + region"]
        SP["Savings Plans\n1 or 3-year\n~30–66% savings\nFlexible: any instance type"]
        Spot["Spot Instances\n~70–90% savings\nInterruptible — 2-min notice"]
    end

    Workload --> OD
    Workload --> RI
    Workload --> SP
    Workload --> Spot

Pricing Model	Savings vs On-Demand	Flexibility	Best For
On-Demand	0%	Full — start/stop anytime	Variable workloads, experiments
Reserved Instance (1yr)	~40%	Locked to instance type	Stable baseline workloads
Reserved Instance (3yr)	~60%	Locked to instance type	Long-running, stable infrastructure
Compute Savings Plan	~30–66%	Any instance type/size	Flexible compute with commitment
Spot Instance	~70–90%	Interruptible	Batch processing, stateless workers

Rule of thumb: Cover your stable baseline capacity with Reserved Instances or Savings Plans. Use Spot for batch jobs, background workers, and non-critical async processing. Use On-Demand only for the variable portion above baseline.

Spot Instances for Fault-Tolerant Workloads

Spot instances use spare cloud capacity at deep discounts, but can be reclaimed with a 2-minute warning. The key is designing workloads that tolerate interruption.

flowchart TB
    Queue["Job Queue\n(SQS / Kafka)"]

    subgraph SpotFleet["Spot Instance Fleet"]
        W1["Worker 1\n(Spot)"]
        W2["Worker 2\n(Spot)"]
        W3["Worker 3\n(Spot — interrupted)"]
    end

    OnDemand["On-Demand Worker\n(Fallback capacity)"]

    Queue --> W1
    Queue --> W2
    Queue --> W3
    Queue --> OnDemand

    Interrupt["Spot interruption\n(2-min notice)"] --> W3
    W3 -->|"Job returned\nto queue"| Queue

Workloads well-suited for Spot:

• CI/CD build runners
• Batch data processing (ETL, analytics)
• ML training jobs
• Video transcoding
• Stateless web tier workers (with On-Demand minimum)
• Load testing

Spot instance best practices:

• Use multiple instance types and availability zones to reduce interruption probability
• Checkpoint long-running jobs so they can resume after interruption
• Always maintain a minimum On-Demand or Reserved baseline for critical capacity
• Use Spot Fleet or Auto Scaling Groups with mixed instance policy

Auto-Scaling to Match Demand

A fleet sized for peak load that runs 24/7 pays for idle capacity at all hours outside of peak. Auto-scaling eliminates this by adjusting fleet size to actual demand.

flowchart LR
    subgraph Static["Static Fleet (No Auto-Scaling)"]
        S_Peak["Peak capacity: 20 instances\nRunning 24/7\nCost: 20 × 24h × 365 = 175,200 instance-hours"]
    end

    subgraph Dynamic["Dynamic Fleet (With Auto-Scaling)"]
        D_Night["Night: 2 instances"]
        D_Day["Business hours: 12 instances"]
        D_Peak["Peak: 20 instances"]
        D_Cost["Avg: 6 instances\nCost: 6 × 24h × 365 = 52,560 instance-hours"]
    end

    Static -->|"Auto-scaling\nreduces waste"| Dynamic
    Dynamic --> Savings["~70% cost reduction\non compute"]

Scale-to-zero for non-production environments is one of the highest-leverage optimizations available. Development and staging environments that run 24/7 when engineers work 8 hours a day waste approximately 65% of their running cost.

---

Storage Cost Optimization

Storage costs grow continuously and silently. Data written is rarely deleted. Storage tiers are rarely reviewed. The right strategy cuts storage costs by 60–80% without deleting a byte of data that is still needed.

Storage Tiering

Not all data needs the same access speed or the same cost. Cloud providers offer multiple tiers designed for different access patterns.

Storage Tier	Access Latency	Cost (relative)	Best For
Standard (hot)	Milliseconds	100%	Frequently accessed data (daily reads)
Infrequent Access (warm)	Milliseconds	~40%	Accessed monthly — older logs, backups
Glacier / Archive (cold)	Minutes to hours	~5–10%	Compliance archives, disaster recovery data
Deep Archive	12+ hours	~2–3%	Long-term retention with rare or no access

flowchart LR
    Data["All Data\n($100/month standard)"]

    Data --> Hot["Hot — 10%\nStandard tier\n($10/month)"]
    Data --> Warm["Warm — 30%\nInfrequent Access tier\n($12/month)"]
    Data --> Cold["Cold — 60%\nGlacier / Archive\n($6/month)"]

    Total["Total: $28/month\nvs $100/month\n72% reduction"]
    Hot --> Total
    Warm --> Total
    Cold --> Total

Lifecycle Policies

Lifecycle policies automate the movement of objects between storage tiers and the deletion of objects that are no longer needed. Without them, storage only ever grows.

# S3 Lifecycle Policy Example
Rules:
  - ID: "log-lifecycle"
    Filter:
      Prefix: "logs/"
    Transitions:
      - Days: 30
        StorageClass: STANDARD_IA       # Move to Infrequent Access after 30 days
      - Days: 90
        StorageClass: GLACIER           # Move to Glacier after 90 days
      - Days: 365
        StorageClass: DEEP_ARCHIVE      # Move to Deep Archive after 1 year
    Expiration:
      Days: 2555                        # Delete after 7 years

Apply lifecycle policies to:

• Application logs
• Access logs and audit trails
• Database backups and snapshots
• User-uploaded content that is rarely re-accessed
• Build artifacts and deployment packages

Deleting Orphaned Resources

Storage waste accumulates from resources that outlive their purpose:

Orphaned Resource	How It Accumulates
Unused EBS volumes	Instance terminated, volume not deleted
Old database snapshots	Manual snapshots never cleaned up
Stale AMIs and VM images	Old deployment images never deregistered
Abandoned S3 buckets	Projects ended, buckets remained
Incomplete multipart uploads	Failed uploads consuming storage indefinitely

Automated cleanup jobs scanning for orphaned resources typically recover 5–15% of storage spend immediately.

---

Database Cost Optimization

Managed databases are among the most expensive line items on a cloud bill. They are also among the most over-provisioned.

Rightsizing Database Instances

Database instances are typically provisioned at the size that was needed during a high-growth period and never revisited. CPU and memory utilization below 20–30% on a database is a clear signal to downsize.

flowchart TB
    subgraph Audit["Database Cost Audit"]
        CPU["CPU Utilization\n< 15% average"]
        MEM["Memory Utilization\n< 30% average"]
        CONN["Active Connections\n< 20% of max"]
        IOPS["IOPS Usage\n< 10% of provisioned"]
    end

    Audit --> Action["Downsize instance class\nReduce provisioned IOPS\nSwitch to gp3 storage"]
    Action --> Savings["Potential saving:\n40–60% of DB cost"]

Read Replicas vs Caching

A read replica costs as much as a second database instance. For read-heavy workloads with repeated queries on the same data, a caching layer (Redis, Memcached) serves the same traffic at a fraction of the cost.

Approach	Latency	Cost	Consistency	Best For
Read Replica	1–5ms	~100% DB cost	Near-real-time	Complex queries, reporting
Redis Cache	< 1ms	~10–20% DB cost	Eventual (TTL-based)	Repeated queries, session data, lists
CDN / Edge Cache	< 10ms	Very low	Stale by TTL	Public, cacheable API responses

Before adding a read replica, measure whether a cache layer can absorb the read load. For many workloads, a $200/month Redis cluster eliminates the need for a $2,000/month read replica.

Serverless and Paused Databases

For non-production environments, databases that pause when idle (Aurora Serverless, Cloud SQL for development) eliminate the cost of a database running 24/7 when only used 8 hours a day.

Environment	Standard DB Cost	Serverless / Paused Cost	Savings
Staging	$300/month (always-on)	$60/month (paused ~80%)	80%
Development	$150/month (always-on)	$15/month (paused ~90%)	90%
Preview envs	$100/month each	$5/month each (paused)	95%

---

Network Cost Optimization

Network egress — data leaving the cloud provider's network — is one of the most underestimated cost categories. Bandwidth is billed asymmetrically: ingress is free, egress is charged.

Egress Cost Sources

flowchart TB
    subgraph Expensive["High Egress Cost Paths"]
        E1["Data transferred\nout to the internet"]
        E2["Cross-region traffic\n(us-east-1 → eu-west-1)"]
        E3["NAT Gateway traffic\n(private subnet → internet)"]
    end

    subgraph Cheap["Low or Zero Cost Paths"]
        C1["Ingress from internet"]
        C2["Within same AZ"]
        C3["CloudFront → origin\n(AWS internal)"]
    end

Traffic Type	Typical Cost
Internet egress (first 10TB)	~$0.09/GB
Cross-region transfer	~$0.02–0.08/GB
NAT Gateway processing	~$0.045/GB
Same AZ (within VPC)	Free
Cross-AZ (within same region)	~$0.01/GB
CloudFront to origin (AWS)	Free

CDN for Egress Reduction

Serving static assets and cacheable API responses through a CDN eliminates direct origin egress. The CDN serves content from edge nodes; origin only pays for the initial cache fill.

flowchart LR
    Users["1,000 users\nrequesting same image"]

    subgraph Without["Without CDN"]
        Origin1["Origin server\n1,000 × 1MB = 1GB egress\nCost: ~$0.09"]
    end

    subgraph With["With CDN"]
        CDN["CDN Edge Node\n1 cache miss → origin (1MB)\n999 cache hits → edge (free egress)"]
        Origin2["Origin server\n1 × 1MB = 0.001GB\nCost: ~$0.0001"]
    end

    Users --> Without
    Users --> With

What to serve through a CDN:

• Static assets (JS, CSS, images, fonts)
• Video and audio files
• Public API responses with low churn
• Software download packages
• Documentation and marketing sites

NAT Gateway Cost Reduction

NAT Gateways charge per GB of processed traffic. Private subnet instances routing all traffic through a NAT Gateway can generate significant unexpected costs, especially for:

• Container image pulls from public registries
• Package manager downloads during deployment
• Telemetry data sent to external monitoring

Mitigations:

• Use VPC Endpoints for AWS service traffic (S3, DynamoDB, ECR) — removes NAT Gateway charges entirely
• Use ECR (Elastic Container Registry) in the same region to avoid cross-region image pulls
• Route DNS and monitoring traffic through the internet gateway directly for public endpoints

Data Compression

Compressing data before storing or transmitting it reduces both storage cost and egress cost simultaneously.

Compression	Ratio	CPU overhead	Best For
gzip	60–80%	Low	JSON, logs, HTML, text
Snappy	20–50%	Very low	High-throughput streaming data
Zstandard	60–80%	Very low	General purpose, balanced
Parquet	80–95%	Medium	Analytical data, columnar reads

Storing logs in compressed format and enabling HTTP compression for API responses are two of the simplest, highest-impact cost reductions available.

---

Environment Cost Optimization

Non-production environments are the most overlooked source of cloud waste. Development, staging, and CI environments often run at near-production scale but are idle for 16+ hours per day.

Schedule-Based Shutdown

flowchart LR
    subgraph Always_On["Always-On Environment"]
        A["Running 24 hours × 30 days\n= 720 hours/month\nCost: $1,000/month"]
    end

    subgraph Scheduled["Scheduled Shutdown"]
        S["Running 8 hours × 22 workdays\n= 176 hours/month\nCost: $244/month"]
    end

    Always_On -->|"Auto-shutdown 6pm\nAuto-start 8am"| Scheduled
    Scheduled --> Savings["Savings: $756/month\n75% reduction"]

Schedule-based shutdown candidates:

• Development environments
• Staging environments (if not used for 24/7 integration testing)
• QA environments
• Load testing clusters (provisioned on demand)
• Feature branch preview environments (destroyed after PR merge)

Infrastructure as Code for Ephemeral Environments

If environments are defined as code, they can be created on demand and destroyed when no longer needed. A feature branch environment that exists only while the PR is open costs a fraction of a permanent environment.

Environment Type	Lifecycle	Cost Model
Permanent staging	Always running	Full monthly cost
PR preview env	Created on PR open, destroyed on PR merge	Hours of cost per PR
Load test env	Provisioned before test, destroyed after	Minutes to hours per test run
Dev environment	Running only during business hours	~25% of always-on cost

---

FinOps: Cost as an Engineering Practice

Cost optimization that happens once a quarter in a review meeting will always lag behind infrastructure growth. The most effective approach treats cost as a continuous engineering concern, not a periodic finance task.

FinOps Principles

flowchart TB
    FinOps["FinOps Practice"]

    FinOps --> Visibility["Visibility\nEvery team sees\ntheir own cost"]
    FinOps --> Accountability["Accountability\nTeams own their\ncost budget"]
    FinOps --> Culture["Culture\nEngineers make\ncost-aware decisions"]
    FinOps --> Automation["Automation\nCost gates in CI/CD\nAutomatic cleanup jobs"]
    FinOps --> Review["Regular Review\nWeekly cost\nanomaly detection"]

Cost Anomaly Detection

Unexpected spikes in cloud spend are often caused by a single misconfigurations — a loop that writes data unintentionally, an auto-scaling policy without a maximum cap, or a misconfigured data transfer. Automated anomaly detection catches these before they become large bills.

Anomaly Type	Example	Detection Method
Sudden spend spike	New service deployed with no cost limits	Alert on 20% day-over-day increase
Runaway auto-scaling	Fleet scaled to 1000 instances, no max cap	Alert on instance count threshold
Unexpected egress	New feature sending large payloads to external APIs	Alert on egress GB exceeding baseline
Orphaned resource accumulation	Hundreds of unused EBS volumes accumulating	Weekly orphan scan report

Unit Economics

The most actionable cost metric is cost per unit of business value — not total cloud spend.

Metric	Formula	Why It Matters
Cost per request	Monthly infra cost / monthly request count	Tracks efficiency as traffic grows
Cost per active user	Monthly infra cost / monthly active users	Compares cost efficiency across growth stages
Cost per transaction	Monthly infra cost / monthly transactions	Most relevant for payment and commerce systems
Cost per GB processed	Monthly infra cost / monthly data processed	Relevant for data pipelines and analytics platforms

If cost per request is decreasing as traffic grows, architecture is scaling efficiently. If it is flat or increasing, architecture has inefficiencies that scale with load.

---

Cost Optimization Checklist

Visibility

• All cloud resources are tagged with team, service, and environment
• Cost allocation reports are shared with each team weekly
• Cost anomaly detection alerts are configured
• Unit cost metrics (cost per request, per user) are tracked

Compute

• All instances have been reviewed for rightsizing in the last 90 days
• Stable baseline workloads are covered by Reserved Instances or Savings Plans
• Spot instances are used for batch, CI/CD, and fault-tolerant workloads
• Auto-scaling is configured with appropriate minimum and maximum bounds
• Non-production environments are scheduled to shut down outside business hours

Storage

• Lifecycle policies are configured on all S3 buckets and blob storage
• Storage tiers are reviewed — cold data is in Glacier or equivalent
• Orphaned volumes, snapshots, and AMIs are inventoried and cleaned up
• Incomplete multipart uploads are automatically aborted after 7 days

Database

• Database instances have been reviewed for rightsizing
• Provisioned IOPS are matched to actual IOPS usage (not over-provisioned)
• Non-production databases use pause-on-idle or serverless configurations
• Caching layer is evaluated before adding read replicas

Network

• CDN is configured for all static assets and cacheable API responses
• VPC Endpoints are used for AWS service traffic (S3, DynamoDB, ECR)
• Data compression is enabled for API responses and stored logs
• Cross-region data transfer is minimized by co-locating services that communicate heavily

Engineering Culture

• Engineers can see their service's cost in a self-service dashboard
• Cost impact is part of architecture review for new features
• Automated cleanup jobs run weekly for orphaned resources
• Cost budgets and alerts are configured per team

---

Summary

Cloud cost optimization is not a one-time project — it is a continuous engineering discipline.

flowchart TB
    Goal["Efficient Cloud Spend"]

    Goal --> Visibility["Visibility\nTag everything\nAttribute every dollar\nAnomaly detection"]
    Goal --> Compute["Compute\nRightsize instances\nReserved + Spot mix\nAuto-scaling"]
    Goal --> Storage["Storage\nLifecycle policies\nTiered storage\nDelete orphans"]
    Goal --> Database["Database\nRightsize DB instances\nCache before replica\nPause non-prod"]
    Goal --> Network["Network\nCDN for egress\nVPC endpoints\nCompress data"]
    Goal --> Culture["FinOps Culture\nTeam cost ownership\nUnit economics\nWeekly review"]

The highest-leverage actions by effort:

Action	Effort	Typical Savings
Rightsize over-provisioned compute	Low	30–60% of compute cost
Reserved Instances / Savings Plans	Low	30–60% of covered compute
Lifecycle policies on storage	Low	50–80% of storage cost
Schedule non-prod shutdown	Low	60–75% of non-prod cost
Spot instances for batch workloads	Medium	70–90% of batch compute
CDN for static asset egress	Medium	80–95% of static egress
VPC Endpoints for AWS service traffic	Low	30–60% of NAT Gateway cost
Caching layer before read replica	Medium	Replaces 1× DB instance

Start with visibility. You cannot optimize what you cannot see. Once spend is attributed and anomalies are alerted, the highest-cost and highest-waste items surface immediately. Address them in order of savings potential, and re-run the cycle quarterly.

Cost efficiency is not a constraint on engineering quality. A well-designed system that scales efficiently, wastes nothing, and matches infrastructure to demand is a better-engineered system in every measurable way.

---

This page is part of the Production Engineering learning path.

Return to the System Design Learning Path to continue with the complete roadmap.