CDN Load Balancing: Round Robin, Least Connections, and Beyond
Load balancing is the mechanism that distributes incoming requests across multiple servers within a CDN so that no single server becomes a bottleneck. A load balancing algorithm determines which server handles each request based on criteria ranging from simple rotation to real-time server performance metrics. Choosing the right algorithm directly impacts response latency, cache efficiency, fault tolerance, and the overall capacity of the CDN infrastructure.
CDN load balancing operates at two distinct levels: global server load balancing (GSLB) distributes traffic across geographically dispersed data centers, while local load balancing distributes requests among servers within a single data center. Understanding both levels and the algorithms available at each is essential for building resilient, high-performance content delivery architectures.
How Does Round Robin Load Balancing Work?
Round robin is the simplest load balancing algorithm. It assigns each incoming request to the next server in a circular list. Server A gets request 1, server B gets request 2, server C gets request 3, then back to server A for request 4. The algorithm maintains a single counter and requires no knowledge of server state, making it extremely fast and easy to implement.
The advantage of round robin is its predictability and zero overhead. Because there is no need to query server health or connection counts, the routing decision takes constant time regardless of pool size. This makes round robin suitable for homogeneous server pools where all machines have identical hardware and handle similar workloads.
The significant limitation is that round robin ignores server capacity and current load. If one server is handling long-running video transcoding requests while another serves small static files, both receive the same number of new connections despite vastly different load levels. This can lead to uneven resource utilization and degraded performance on overloaded servers. For CDN workloads where request processing times vary widely, round robin is rarely used as the sole algorithm.
What Is Weighted Round Robin and When Should You Use It?
Weighted round robin extends the basic algorithm by assigning a weight to each server based on its capacity. A server with weight 3 receives three times as many requests as a server with weight 1. This addresses the heterogeneous hardware problem: a powerful 64-core server can be assigned a higher weight than a smaller 16-core instance.
| Metric | Edge Processing | Origin Processing | Improvement |
|---|---|---|---|
| Latency (p50) | 8–15 ms | 120–300 ms | 15–20x faster |
| Cache Hit Ratio | 85–95% | N/A | Reduces origin load |
| TLS Handshake | 1–5 ms (nearby PoP) | 50–150 ms | 10–30x faster |
| Bandwidth Cost | $0.01–0.04/GB | $0.05–0.12/GB | 3–5x cheaper |
| Availability | 99.99% (multi-PoP) | 99.9% (single DC) | 10x fewer outages |
# Weighted round robin configuration example
server edge-1.example.com weight=5; # 40-core, 128GB RAM
server edge-2.example.com weight=3; # 16-core, 64GB RAM
server edge-3.example.com weight=2; # 8-core, 32GB RAMWeights are typically assigned based on CPU cores, memory, or benchmark throughput. Some CDN platforms adjust weights dynamically based on real-time metrics, blurring the line between weighted round robin and more sophisticated algorithms. Static weights work well when server capacity is predictable and workloads are relatively uniform.
How Does the Least Connections Algorithm Improve Distribution?
The least connections algorithm routes each new request to the server with the fewest active connections. Unlike round robin, it accounts for actual server load in real time. If server A has 150 active connections and server B has 80, the next request goes to server B regardless of their position in any rotation.
This algorithm excels when request processing times vary significantly. In a CDN serving both small HTML files (processed in milliseconds) and large video chunks (held for seconds), least connections naturally sends more short-lived requests to servers that finish faster, while servers handling slow requests accumulate connections and receive fewer new ones.
Weighted least connections combines both approaches: the algorithm selects the server with the lowest ratio of active connections to assigned weight. A server with weight 4 and 200 connections (ratio 50) is preferred over a server with weight 2 and 120 connections (ratio 60). This is the most common algorithm in production CDN edge server deployments because it balances both capacity and real-time load.
What Is IP Hash and Why Does Session Persistence Matter?
IP hash load balancing computes a hash of the client's IP address and uses the result to deterministically select a server. The same client IP always maps to the same server, providing session persistence (also called session affinity or sticky sessions) without requiring shared session state between servers.
Session persistence matters for applications that store state locally on the server handling the request. If a user's shopping cart exists only in server A's memory, routing their next request to server B would lose the cart. IP hash ensures all requests from that client consistently reach server A.
In CDN contexts, IP hash has a secondary benefit: cache affinity. When the same users consistently hit the same edge server, that server's cache warms with the content those users request, improving cache hit ratios. However, this advantage is modest compared to URL-based hashing approaches like consistent hashing.
The major drawback of IP hash is uneven distribution. Corporate networks may funnel thousands of employees through a single NAT IP address, overloading the server that IP hashes to. Mobile carrier-grade NAT amplifies this problem further. IP hash also breaks session persistence when users switch networks (WiFi to mobile data), changing their IP and consequently their assigned server.
How Does Geographic Load Balancing Route by Location?
Geographic load balancing uses the client's geographic location to select a server or data center. At the GSLB level, this means mapping the client's IP address to a country or region using GeoIP databases and returning the nearest data center's address. Within a data center, geographic factors may influence which server cluster handles the request.
Geographic routing is the default GSLB strategy for most CDNs because it minimizes physical network distance, which generally correlates with lower latency. The DNS resolution process is the most common implementation point: the CDN's authoritative DNS server returns different IP addresses based on where the DNS query originates.
Geographic load balancing has accuracy limitations. GeoIP databases are imperfect, especially for mobile and satellite connections. More importantly, geographic proximity does not guarantee network proximity. A user in a city on the border of two regions may have better network paths to a data center in the farther region due to ISP peering arrangements. Latency-based routing addresses this limitation.
What Is Latency-Based Load Balancing?
Latency-based load balancing measures actual network round-trip times between CDN nodes and routes users to the server that provides the lowest latency, regardless of geographic distance. This approach uses active probing and passive measurement to build a continuously updated latency map.
Active probing sends synthetic requests between CDN PoPs and origins or between PoPs at regular intervals. The measured latencies feed into the routing decision. Passive measurement analyzes actual user traffic to compute latency statistics per region, per ISP, or per resolver IP. Combining both provides the most accurate latency picture.
AWS CloudFront and Route 53 offer latency-based routing as a standard feature. Cloudflare implements a similar approach through their Argo Smart Routing, which finds the fastest network path across their backbone rather than relying on public internet routing.
How Does Consistent Hashing Maximize Cache Efficiency?
Consistent hashing is a specialized algorithm designed for cache-heavy workloads. It maps both servers and content keys (typically URLs) onto a hash ring. Each request is routed to the next server clockwise on the ring from the content key's position. This ensures the same URL always routes to the same server, maximizing cache hit rates.
The critical advantage over simple IP hash or round robin is behavior during server changes. When a server is added or removed from the pool, only the keys that were mapped to that server need to be remapped. In a pool of 10 servers, adding an 11th server redistributes approximately 1/11 of the cached content rather than invalidating the entire cache. This property is essential for maintaining high cache hit ratios during scaling events and server maintenance.
Virtual nodes (vnodes) improve distribution evenness. Instead of placing each physical server at one point on the hash ring, each server occupies 100-200 virtual points. This spreads the key space more evenly across servers, preventing the hot-spot problem where one server is responsible for a disproportionately large arc of the ring.
CDN providers use consistent hashing extensively at the cache hierarchy level. When an L1 edge cache misses, the request is forwarded to a specific L2 cache server selected by consistent hashing on the URL. This concentrates cache fills at the L2 tier, reducing origin load and improving cache hit ratios at every level.
How Do These Algorithms Compare?
| Algorithm | Complexity | Load awareness | Session persistence | Cache efficiency | Best for |
|---|---|---|---|---|---|
| Round robin | O(1) | None | None | Low | Homogeneous pools, stateless services |
| Weighted round robin | O(1) | Static capacity | None | Low | Mixed hardware, predictable workloads |
| Least connections | O(log n) | Real-time | None | Low | Variable request duration |
| Weighted least connections | O(log n) | Real-time + capacity | None | Low | Production CDN edge pools |
| IP hash | O(1) | None | Yes (IP-based) | Moderate | Stateful applications |
| Geographic | O(1) | None | Regional | Regional | GSLB, multi-region |
| Latency-based | O(1) | Network-aware | None | Low | Latency-sensitive applications |
| Consistent hashing | O(log n) | None (basic) | Content-based | High | Cache-heavy workloads |
What Is the Difference Between GSLB and Local Load Balancing?
Global Server Load Balancing (GSLB) and local load balancing serve complementary roles in CDN architecture. GSLB operates at the DNS level to select which data center or PoP handles a request. It considers geographic proximity, data center health, and capacity to route users to the best location. GSLB decisions happen infrequently (once per DNS TTL period) and affect all subsequent requests from that client.
Local load balancing operates within a data center after GSLB has directed the user there. It distributes individual requests across the pool of servers in that location using algorithms like weighted least connections or consistent hashing. Local load balancing decisions happen per-request and can react to server-level changes in milliseconds.
In a typical CDN request flow, GSLB first routes the user to the nearest healthy data center via DNS with appropriate TTL settings. The user's TCP connection arrives at a local load balancer (hardware appliance or software like NGINX, HAProxy, or Envoy) which selects a specific edge server. If the edge server does not have the content cached, it may consult another load balancing layer to select the optimal L2 cache or origin shield server.
How Do Health Checks Integrate with Load Balancing?
Health checks are the feedback mechanism that keeps load balancing algorithms operating on accurate data. Without health checks, a load balancer might route traffic to a server that has crashed, is overloaded, or is returning errors.
Active health checks probe each server at regular intervals (every 5-30 seconds) with TCP connections, HTTP requests, or custom health endpoints. The load balancer tracks the results and removes servers that fail a configurable number of consecutive checks. When a failed server recovers and passes checks again, it is reintroduced to the pool gradually, often starting with a reduced weight to allow its cache to warm before receiving full traffic.
Passive health checks (also called real-time or inline checks) monitor actual user traffic for errors. If a server returns five consecutive 502 errors within a 10-second window, the load balancer marks it as unhealthy without waiting for the next active probe cycle. This provides faster failure detection than periodic active checks alone.
The combination of intelligent load balancing algorithms and robust health check and failover systems is what enables CDNs to deliver the high availability numbers (99.99% and above) that modern applications demand. No algorithm alone can guarantee uptime without the feedback loop that health checks provide.