HTTP 429 Too Many Requests: Rate Limiting

Intermediate

HTTP 429 Too Many Requests is the server's way of telling a client to slow down. When a server or CDN rate limiter detects that a client has exceeded the allowed request rate, it responds with 429 along with a Retry-After header indicating when the client can try again. Rate limiting is a critical defense mechanism that protects servers from being overwhelmed by traffic — whether from DDoS attacks, aggressive web scrapers, misconfigured API clients, or legitimate traffic spikes.

Understanding rate limiting algorithms, implementation strategies, and client-side handling of 429 responses is essential for both API providers who need to protect their services and API consumers who need to handle throttling gracefully.

How Do Rate Limiting Algorithms Work?

Four primary algorithms power modern rate limiters, each with different characteristics for burst handling, memory usage, and fairness.

Fixed Window Counter divides time into fixed intervals (e.g., 1 minute) and counts requests within each window. When the count exceeds the limit, subsequent requests receive 429. The simplicity comes with a drawback: a client can send double the intended rate by sending requests at the end of one window and the beginning of the next (the "boundary burst" problem).

Sliding Window Log maintains a log of all request timestamps and counts only those within the sliding window. This eliminates the boundary burst problem but requires storing every timestamp, consuming more memory. A variant — the sliding window counter — uses weighted averages of the current and previous windows to approximate the log approach with fixed-window memory efficiency.

Token Bucket maintains a bucket with a fixed capacity of tokens. Tokens are added at a constant rate (e.g., 10 per second). Each request consumes one token. If the bucket is empty, the request is rejected with 429. The bucket can hold more tokens than the per-second rate, allowing controlled bursts. This is the most widely used algorithm for API rate limiting because it handles bursty traffic naturally.

Leaky Bucket processes requests at a constant rate, like water leaking from a bucket at a fixed rate. Incoming requests fill the bucket; if the bucket overflows (queue full), requests are dropped. This produces the smoothest output rate but adds latency because requests may queue before processing.

How Do You Implement Rate Limiting in Nginx?

Nginx provides the limit_req module for rate limiting based on request rate and the limit_conn module for limiting concurrent connections.

# Define rate limit zone (10MB shared memory, 10 req/sec per IP)
limit_req_zone $binary_remote_addr zone=api:10m rate=10r/s;
limit_req_zone $binary_remote_addr zone=login:10m rate=1r/s;

server {
    # API endpoints: 10 req/sec with burst of 20
    location /api/ {
        limit_req zone=api burst=20 nodelay;
        limit_req_status 429;
        proxy_pass http://backend;
    }

    # Login endpoint: strict 1 req/sec
    location /auth/login {
        limit_req zone=login burst=5;
        limit_req_status 429;
        proxy_pass http://backend;
    }
}

The burst parameter defines how many excess requests can be queued. The nodelay flag processes queued requests immediately rather than spacing them out. Without nodelay, excess requests are delayed to match the configured rate, which adds latency but produces smoother traffic.

How Does CDN Edge Rate Limiting Work?

CDN rate limiting operates at edge servers distributed across the globe, providing several advantages over origin-only rate limiting:

Scale — CDN networks handle millions of requests per second across hundreds of PoPs. Origin rate limiters protect a single server; CDN rate limiters protect the entire infrastructure.

Proximity — rejecting requests at the nearest edge PoP (typically within 10ms) is far more efficient than sending the request across the globe to the origin (50-200ms) only to be rejected there.

Distributed state — CDN rate limiters can maintain per-client counters across all PoPs, preventing attackers from circumventing rate limits by rotating through different edge locations.

CDN rate limiting can be configured by IP address, API key, geographic region, request path, or custom request attributes. For example, an API might allow 100 requests per minute for free-tier API keys and 10,000 per minute for enterprise keys, with enforcement at the CDN edge.

How Should Clients Handle 429 Responses?

API clients should implement robust 429 handling with exponential backoff and jitter:

Honor Retry-After — always check for and respect the Retry-After header. Ignoring it risks escalating the rate limit to a longer ban or permanent block.

Exponential backoff — if no Retry-After header is present, use exponential backoff: wait 1 second, then 2, then 4, then 8, up to a maximum delay. This prevents thundering herd problems where many clients retry simultaneously after a rate limit window resets.

Jitter — add random jitter to the backoff delay to desynchronize retries from multiple clients. Without jitter, clients that started at the same time will retry at the same time, creating synchronized bursts.

Rate limit headers — many APIs include informational headers that help clients avoid hitting limits:

X-RateLimit-Limit: 100        # Total allowed requests per window
X-RateLimit-Remaining: 23     # Requests remaining in current window
X-RateLimit-Reset: 1698825600 # Unix timestamp when window resets

Well-designed clients track these headers and proactively slow down as they approach the limit, rather than waiting for a 429 response.