Rate Limiter Design - Complete Low-Level Design Guide

Design a scalable Rate Limiter using Java and Spring Boot. Learn Token Bucket, Leaky Bucket, Fixed Window, Sliding Window, Redis implementation, distributed rate limiting, SOLID principles, design patterns, and enterprise architecture.

Introduction

Every modern application needs to protect itself from excessive traffic.

Examples include:

Login APIs
Payment APIs
OTP Generation
Password Reset
Search APIs
Public REST APIs
AI APIs
Banking APIs
E-commerce Checkout

Without rate limiting, a single user or malicious bot can overwhelm the system, causing service degradation or outages.

Popular platforms like:

AWS API Gateway
Google APIs
GitHub
Stripe
PayPal
Twilio
OpenAI

all enforce rate limits to ensure fairness, security, and system stability.

What is Rate Limiting?

Rate Limiting controls how many requests a client can make within a given time period.

Example:

100 Requests

↓

Per Minute

↓

Per User

If the limit is exceeded, the server rejects additional requests.

Why Do We Need Rate Limiting?

Without Rate Limiting:

API abuse
DDoS attacks
Credential stuffing
Brute force attacks
Resource exhaustion
Higher infrastructure cost

With Rate Limiting:

Fair resource usage
Better availability
Improved security
Stable performance
Predictable system behavior

Real-World Examples

API	Limit Example
Login API	5 attempts / minute
OTP API	3 requests / 10 minutes
Payment API	20 requests / minute
Search API	100 requests / minute
AI API	500 tokens / minute
Public REST API	1000 requests / hour

Functional Requirements

The system should support:

Limit requests
Per-user limits
Per-IP limits
Per-API limits
Distributed deployment
Dynamic configuration
Retry information
Rate limit headers
Monitoring
Burst handling

Non-Functional Requirements

The system should be:

Highly Available
Highly Concurrent
Distributed
Low Latency
Fault Tolerant
Scalable
Configurable

High-Level Architecture

flowchart TD
    CLIENT["Client Request"]

    GATEWAY["API Gateway"]

    LIMITER["Distributed Rate Limiter"]

    REDIS["Redis Counter Store"]

    CONFIG["Rules & Configuration Service"]

    MONITOR["Monitoring & Metrics"]

    BACKEND["Application Service"]

    CLIENT --> GATEWAY --> LIMITER

    LIMITER --> REDIS
    LIMITER --> CONFIG
    LIMITER --> MONITOR
    LIMITER --> BACKEND

Request Flow

sequenceDiagram

participant Client

participant Gateway

participant RateLimiter

participant Redis

Client->>Gateway: HTTP Request

Gateway->>RateLimiter: Check Limit

RateLimiter->>Redis: Current Count

Redis-->>RateLimiter: Count

alt Allowed

RateLimiter-->>Gateway: Allow

Gateway-->>Client: Success

else Blocked

RateLimiter-->>Gateway: Reject

Gateway-->>Client: HTTP 429

end

HTTP Status Code

When requests exceed the configured limit:

429 Too Many Requests

Common response headers:

X-RateLimit-Limit

X-RateLimit-Remaining

X-RateLimit-Reset

Retry-After

Where Can Rate Limiting Be Applied?

API Gateway
Load Balancer
Reverse Proxy
Application Layer
Microservice
User Level
API Key Level
IP Address
Organization
Region

Rate Limiting Algorithms

The most common algorithms are:

Fixed Window Counter
Sliding Window Log
Sliding Window Counter
Token Bucket
Leaky Bucket

Fixed Window Counter

A counter is maintained for a fixed duration.

Example:

Limit

100 Requests

↓

Every Minute

flowchart LR
    WINDOW_START["Start Time Window (Minute 1)"]

    COUNTER["Request Counter Increment"]

    RESET["Window Reset Logic"]

    WINDOW_NEXT["Next Time Window (Minute 2)"]

    WINDOW_START --> COUNTER --> RESET --> WINDOW_NEXT

Advantages:

Simple
Fast

Disadvantages:

Burst traffic at window boundaries

Sliding Window Log

Every request timestamp is stored.

flowchart LR
    REQUEST["Incoming Request"]

    TIMESTAMP["Request Timestamp Generator"]

    WINDOW["Sliding Window Bucket Logic"]

    CLEANUP["Eviction of Old Entries"]

    REQUEST --> TIMESTAMP --> WINDOW --> CLEANUP

Advantages:

Accurate

Disadvantages:

High memory usage

Sliding Window Counter

Uses multiple counters to smooth request distribution.

flowchart LR
    CURRENT["Current Time Window"]

    PREV["Previous Window Data"]

    WEIGHT["Weighted Aggregation Engine"]

    DECISION["Allow / Reject Decision Engine"]

    CURRENT --> PREV --> WEIGHT --> DECISION

Advantages:

Better accuracy
Lower memory usage

Token Bucket Algorithm

One of the most popular algorithms.

flowchart TD
    BUCKET["Token Bucket"]

    REQUEST["Incoming Request"]

    ALLOW["Allow Request"]

    REJECT["Reject Request"]

    BUCKET --> REQUEST

    REQUEST --> ALLOW
    REQUEST --> REJECT

Tokens are added periodically.

Each request consumes one token.

If no tokens remain, requests are rejected.

Advantages:

Supports burst traffic
Efficient
Widely used

Example

Bucket Capacity

10 Tokens

↓

Refill

1 Token / Second

Current:

Tokens = 7

↓

Incoming Request

↓

Tokens = 6

Leaky Bucket Algorithm

Requests enter a queue.

They leave at a constant rate.

flowchart LR
    REQUESTS["Incoming Request Stream"]

    BUCKET["Token Bucket / Queue Buffer"]

    OUTPUT["Controlled Constant Output"]

    REQUESTS --> BUCKET --> OUTPUT

Advantages:

Smooth traffic

Disadvantages:

Less flexible for bursts

Algorithm Comparison

Algorithm	Burst Support	Memory	Accuracy
Fixed Window	Poor	Low	Medium
Sliding Log	Excellent	High	High
Sliding Counter	Good	Medium	High
Token Bucket	Excellent	Low	High
Leaky Bucket	Medium	Low	High

Domain Model

classDiagram

class RateLimiter

class Request

class Policy

class Bucket

class RedisStore

RateLimiter --> Policy

RateLimiter --> Bucket

Bucket --> RedisStore

Entity Responsibilities

RateLimiter

Validate requests
Apply algorithm
Return decision

Policy

Stores:

Limit
Duration
Algorithm
Scope

Bucket

Stores:

Available Tokens
Last Refill Time

RedisStore

Stores:

Counters
Expiration
Distributed state

Design Patterns

Strategy Pattern

Support multiple algorithms.

Examples:

Token Bucket
Leaky Bucket
Sliding Window
Fixed Window

Factory Pattern

Create algorithm implementations.

Singleton

Configuration Manager

Chain of Responsibility

Multiple rate limit checks.

Example:

IP

↓

User

↓

API Key

↓

Organization

SOLID Principles

SRP

Each algorithm has one responsibility.

OCP

Add new algorithms without modifying existing code.

LSP

Every algorithm behaves as a RateLimiterStrategy.

ISP

Separate interfaces for:

Counter Storage
Algorithm
Configuration

DIP

RateLimiter depends on abstractions.

Distributed Rate Limiting

Single-node counters fail in clustered deployments.

Solution:

flowchart TD
    APP1["Application Instance 1"]
    APP2["Application Instance 2"]
    APP3["Application Instance 3"]

    REDIS["Redis Atomic Counter"]

    COUNTER["Shared Counter State"]

    APP1 --> REDIS
    APP2 --> REDIS
    APP3 --> REDIS

    REDIS --> COUNTER

Redis becomes the centralized counter store.

Redis Implementation

Redis stores:

UserID

↓

Counter

↓

TTL

Example:

user123

↓

95 Requests

↓

Expires in 12 Seconds

TTL automatically resets counters after the configured time window.

API Gateway Integration

flowchart TD
    C["Client"]
    API["API Gateway"]
    RL["Rate Limiter"]
    APP["Spring Boot API"]

    C --> API --> RL --> APP

The gateway blocks requests before they reach the application.

Enterprise Architecture

flowchart TD
    C["Client"]

    LB["Load Balancer"]

    API["API Gateway"]

    RL["Rate Limiter"]

    REDIS["Redis Cluster"]

    APP["Spring Boot Service"]

    DB["PostgreSQL"]

    MON["Monitoring"]

    GRAF["Grafana"]

    C --> LB --> API

    API --> RL
    API --> APP

    RL --> REDIS
    RL --> MON

    APP --> DB

    MON --> GRAF

Monitoring Metrics

Track:

Allowed Requests
Rejected Requests
Requests Per Second
Active Buckets
Token Utilization
Average Response Time
Top Consumers

Scaling Considerations

Large systems may process:

Millions of Requests
Thousands of APIs
Millions of Users

Scaling techniques:

Redis Cluster
Horizontal Scaling
Sharding
Local Cache
Consistent Hashing
Async Metrics Collection

Future Enhancements

Possible features:

Dynamic Rate Limits
AI-based Abuse Detection
Premium User Limits
Geographic Limits
Time-based Limits
Adaptive Throttling
User Tier Policies
Global Rate Limits
API Key Management
Self-Service Configuration

Common Mistakes

❌ Using local memory in distributed systems.

❌ Ignoring clock synchronization.

❌ Hardcoding limits.

❌ No retry information.

❌ No Redis expiration.

❌ Using one global limit for every user.

❌ Ignoring burst traffic.

Interview Questions

What is rate limiting?
Why is Redis commonly used?
Explain the Token Bucket algorithm.
Explain the Leaky Bucket algorithm.
Fixed Window vs Sliding Window?
How would you implement distributed rate limiting?
Why should rate limiting be implemented at the API Gateway?
How would you support different limits for premium users?
How would you prevent brute-force login attacks?
How would you scale a rate limiter to millions of requests?

Summary

A Rate Limiter is a foundational component of modern distributed systems that protects services from abuse while ensuring fairness and high availability.

A production-ready implementation typically includes:

Layered Spring Boot architecture
SOLID principles
Strategy, Factory, Singleton, and Chain of Responsibility patterns
Token Bucket or Sliding Window algorithms
Redis for distributed counters
API Gateway integration
HTTP 429 responses with rate limit headers
Monitoring and alerting
Configurable policies
Horizontal scalability

Mastering this design prepares you for advanced infrastructure components such as API Gateway, Distributed Cache, Load Balancer, Service Mesh, Authentication Gateway, and Cloud-Native Microservices, where traffic management and resilience are critical.