Full Stack • Java • System Design • Cloud • AI Engineering

Cloud • 2024-01-29

AWS EC2 Complete Guide - From Basics to Advanced

Master Amazon EC2 with this comprehensive guide covering instance types, pricing models, security, networking, and best practices.

AWS EC2 Complete Guide

What is Amazon EC2?

Amazon Elastic Compute Cloud (EC2) is a web service that provides secure, resizable compute capacity in the cloud. It's designed to make web-scale cloud computing easier for developers.

Key Features

  • Elastic: Scale up or down in minutes
  • Flexible: Choose from various instance types
  • Secure: Multiple security layers
  • Cost-Effective: Pay only for what you use
  • Reliable: 99.99% availability SLA

EC2 Instance Types

1. General Purpose (T, M series)

T3/T3a (Burstable):

Use Cases:
- Web servers
- Development environments
- Small databases
- Microservices

Sizes: nano, micro, small, medium, large, xlarge, 2xlarge

Example: t3.medium
- 2 vCPUs
- 4 GB RAM
- Baseline: 20% CPU
- Burst: 100% CPU

M6i/M6a (Balanced):

Use Cases:
- Application servers
- Gaming servers
- Medium databases
- Enterprise applications

Example: m6i.xlarge
- 4 vCPUs
- 16 GB RAM
- Consistent performance

2. Compute Optimized (C series)

C6i/C6a:

Use Cases:
- High-performance web servers
- Batch processing
- Scientific modeling
- Gaming servers
- Ad serving

Example: c6i.2xlarge
- 8 vCPUs
- 16 GB RAM
- High CPU performance

3. Memory Optimized (R, X series)

R6i/R6a:

Use Cases:
- In-memory databases (Redis, Memcached)
- Real-time big data analytics
- High-performance databases
- SAP HANA

Example: r6i.xlarge
- 4 vCPUs
- 32 GB RAM
- High memory-to-CPU ratio

4. Storage Optimized (I, D series)

I3/I3en:

Use Cases:
- NoSQL databases (Cassandra, MongoDB)
- Data warehousing
- Elasticsearch
- Kafka

Example: i3.xlarge
- 4 vCPUs
- 30.5 GB RAM
- 950 GB NVMe SSD

5. Accelerated Computing (P, G, F series)

P4/P3 (GPU):

Use Cases:
- Machine learning training
- Deep learning
- High-performance computing
- Video encoding

Example: p3.2xlarge
- 8 vCPUs
- 61 GB RAM
- 1 NVIDIA V100 GPU

EC2 Pricing Models

1. On-Demand Instances

Pricing: Pay by the second (minimum 60 seconds)

Pros:

  • ✅ No upfront payment
  • ✅ No long-term commitment
  • ✅ Flexible
  • ✅ Good for unpredictable workloads

Cons:

  • ❌ Most expensive option
  • ❌ No discounts

Use Cases:

  • Development and testing
  • Short-term workloads
  • Unpredictable applications
  • First-time applications

Example Pricing (us-east-1):

t3.medium: $0.0416/hour = ~$30/month
m6i.xlarge: $0.192/hour = ~$140/month
c6i.2xlarge: $0.34/hour = ~$248/month

2. Reserved Instances (RI)

Commitment: 1 or 3 years

Discount: Up to 75% compared to On-Demand

Payment Options:

  • All Upfront (highest discount)
  • Partial Upfront (moderate discount)
  • No Upfront (lowest discount)

Types:

  • Standard RI (cannot change instance type)
  • Convertible RI (can change instance type)

Pros:

  • ✅ Significant cost savings
  • ✅ Capacity reservation
  • ✅ Predictable billing

Cons:

  • ❌ Long-term commitment
  • ❌ Less flexibility

Use Cases:

  • Steady-state workloads
  • Production applications
  • Predictable usage

Example Savings:

t3.medium On-Demand: $30/month
t3.medium 1-year RI: $18/month (40% savings)
t3.medium 3-year RI: $12/month (60% savings)

3. Spot Instances

Pricing: Up to 90% discount

How it Works:

  • Bid for unused EC2 capacity
  • Can be interrupted with 2-minute warning
  • Price fluctuates based on supply/demand

Pros:

  • ✅ Massive cost savings
  • ✅ Good for flexible workloads

Cons:

  • ❌ Can be interrupted
  • ❌ Not suitable for critical workloads

Use Cases:

  • Batch processing
  • Data analysis
  • CI/CD pipelines
  • Fault-tolerant applications
  • Containerized workloads

Best Practices:

# Use Spot Fleet for diversification
spot_fleet_config = {
    'AllocationStrategy': 'diversified',
    'InstancePools': 10,
    'SpotPrice': '0.05',
    'TargetCapacity': 20
}

4. Savings Plans

Commitment: 1 or 3 years

Discount: Up to 72%

Flexibility:

  • Change instance family
  • Change region
  • Change OS
  • Change tenancy

Types:

  • Compute Savings Plans (most flexible)
  • EC2 Instance Savings Plans (higher discount)

Pros:

  • ✅ Flexible
  • ✅ Significant savings
  • ✅ Automatic application

Cons:

  • ❌ Commitment required

EC2 Storage Options

1. EBS (Elastic Block Store)

Types:

gp3 (General Purpose SSD):

Use Case: Boot volumes, dev/test
IOPS: 3,000-16,000
Throughput: 125-1,000 MB/s
Size: 1 GB - 16 TB
Price: $0.08/GB-month

io2 (Provisioned IOPS SSD):

Use Case: Databases, critical applications
IOPS: 100-64,000
Throughput: Up to 1,000 MB/s
Size: 4 GB - 16 TB
Price: $0.125/GB-month + $0.065/IOPS

st1 (Throughput Optimized HDD):

Use Case: Big data, data warehouses
Throughput: Up to 500 MB/s
Size: 125 GB - 16 TB
Price: $0.045/GB-month

sc1 (Cold HDD):

Use Case: Infrequent access
Throughput: Up to 250 MB/s
Size: 125 GB - 16 TB
Price: $0.015/GB-month

2. Instance Store

Characteristics:

  • Temporary storage
  • Physically attached to host
  • High performance
  • Data lost on stop/terminate

Use Cases:

  • Cache
  • Buffers
  • Temporary data
  • Scratch data

3. EFS (Elastic File System)

Characteristics:

  • Shared file system
  • Multiple EC2 instances
  • Automatic scaling
  • Pay for what you use

Use Cases:

  • Content management
  • Web serving
  • Data sharing
  • Home directories

EC2 Networking

1. VPC (Virtual Private Cloud)

VPC Structure:
├── VPC (10.0.0.0/16)
│   ├── Public Subnet (10.0.1.0/24)
│   │   ├── Internet Gateway
│   │   └── NAT Gateway
│   ├── Private Subnet (10.0.2.0/24)
│   │   └── EC2 Instances
│   └── Database Subnet (10.0.3.0/24)
│       └── RDS Instances

2. Security Groups

Stateful Firewall:

Inbound Rules:
- SSH (22) from 0.0.0.0/0
- HTTP (80) from 0.0.0.0/0
- HTTPS (443) from 0.0.0.0/0
- Custom TCP (8080) from sg-12345678

Outbound Rules:
- All traffic to 0.0.0.0/0

Best Practices:

  • Principle of least privilege
  • Use specific IP ranges
  • Reference other security groups
  • Regular audits

3. Elastic IP

Characteristics:

  • Static public IPv4 address
  • Can be reassigned
  • Charged when not associated

Use Cases:

  • Whitelisting
  • DNS mapping
  • Failover

4. Elastic Network Interface (ENI)

Features:

  • Multiple private IPs
  • Multiple security groups
  • MAC address
  • Source/destination check

Use Cases:

  • Network appliances
  • Dual-homed instances
  • Low-budget HA

EC2 Security Best Practices

1. IAM Roles

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "s3:GetObject",
        "s3:PutObject"
      ],
      "Resource": "arn:aws:s3:::my-bucket/*"
    }
  ]
}

Benefits:

  • No hardcoded credentials
  • Automatic rotation
  • Fine-grained permissions

2. Key Pairs

Best Practices:

  • Use strong key pairs (RSA 2048+)
  • Store private keys securely
  • Rotate keys regularly
  • Use AWS Systems Manager Session Manager

3. Encryption

EBS Encryption:

  • Encrypt at rest
  • Encrypt in transit
  • Use AWS KMS
  • Enable by default

Data in Transit:

  • Use TLS/SSL
  • VPN connections
  • AWS PrivateLink

4. Monitoring

CloudWatch Metrics:

  • CPU Utilization
  • Network In/Out
  • Disk Read/Write
  • Status Checks

CloudWatch Alarms:

alarm = cloudwatch.put_metric_alarm(
    AlarmName='HighCPU',
    MetricName='CPUUtilization',
    Namespace='AWS/EC2',
    Statistic='Average',
    Period=300,
    EvaluationPeriods=2,
    Threshold=80,
    ComparisonOperator='GreaterThanThreshold'
)

EC2 Auto Scaling

Components

  1. Launch Template
  2. Auto Scaling Group
  3. Scaling Policies

Scaling Policies

Target Tracking:

{
  "TargetValue": 50.0,
  "PredefinedMetricSpecification": {
    "PredefinedMetricType": "ASGAverageCPUUtilization"
  }
}

Step Scaling:

CPU > 80%: Add 2 instances
CPU > 90%: Add 4 instances
CPU < 30%: Remove 1 instance

Scheduled Scaling:

Monday-Friday 9 AM: Scale to 10 instances
Monday-Friday 6 PM: Scale to 2 instances

EC2 Best Practices

1. Right-Sizing

  • Monitor utilization
  • Use CloudWatch metrics
  • Downsize underutilized instances
  • Use AWS Compute Optimizer

2. Cost Optimization

  • Use Reserved Instances for steady workloads
  • Use Spot Instances for flexible workloads
  • Stop instances when not needed
  • Use Auto Scaling
  • Delete unused EBS volumes

3. High Availability

  • Use multiple Availability Zones
  • Use Auto Scaling Groups
  • Use Elastic Load Balancer
  • Regular backups (AMIs, snapshots)

4. Performance

  • Choose right instance type
  • Use enhanced networking
  • Use placement groups
  • Optimize EBS performance

5. Security

  • Use IAM roles
  • Enable encryption
  • Regular patching
  • Use security groups properly
  • Enable CloudTrail logging

Common Use Cases

1. Web Application

Architecture:
├── Application Load Balancer
├── Auto Scaling Group
│   ├── EC2 (t3.medium) - Web Server
│   ├── EC2 (t3.medium) - Web Server
│   └── EC2 (t3.medium) - Web Server
├── RDS (db.t3.medium) - Database
└── ElastiCache (cache.t3.micro) - Cache

2. Batch Processing

Architecture:
├── S3 - Input Data
├── Spot Fleet (c6i.xlarge) - Processing
├── SQS - Job Queue
└── S3 - Output Data

3. Machine Learning

Architecture:
├── S3 - Training Data
├── EC2 (p3.2xlarge) - Training
├── S3 - Model Storage
└── EC2 (g4dn.xlarge) - Inference

Conclusion

Amazon EC2 is a powerful and flexible compute service that forms the backbone of many cloud applications.

Key Takeaways:

  • Choose the right instance type for your workload
  • Use appropriate pricing model to optimize costs
  • Implement security best practices
  • Use Auto Scaling for high availability
  • Monitor and optimize regularly

Next Steps:

  1. Create an AWS account
  2. Launch your first EC2 instance
  3. Experiment with different instance types
  4. Set up Auto Scaling
  5. Implement monitoring and alerts

Happy computing! 🚀