Round Robin Load Balancing: Network Engineering Guide

Introduction and Core Concepts

Round robin load balancing is one of the most fundamental and widely deployed algorithms for distributing network traffic and computational workloads across multiple backend resources. At its core, it is a simple, deterministic scheduling discipline that cycles through a pool of servers, nodes, or service instances in a fixed circular order, assigning each new incoming request to the next available resource in sequence. Once the end of the list is reached, the distributor wraps around to the beginning and continues the cycle indefinitely.

The importance of round robin load balancing stems from its elegant balance of simplicity, fairness, and efficiency. In modern distributed systems, a single server is rarely capable of handling the volume, concurrency, or availability requirements of production workloads. Whether serving a high-traffic website, a global API, or an internal microservice mesh, organizations must horizontally scale across multiple machines. Without an intelligent distribution mechanism, that scaling is ineffective. Round robin provides the foundational strategy for ensuring no single node is overwhelmed while others sit idle, maximizing resource utilization and providing a baseline for high availability.

This algorithm belongs to the family of static, non-adaptive load balancing techniques. Unlike dynamic algorithms that make decisions based on real-time server metrics like CPU load, active connections, or response latency, classic round robin makes no assumptions about the current state or capacity of the backend. It treats all servers as equal participants and distributes requests with perfect mathematical evenness over time. This stateless nature is both its greatest strength and its primary limitation, making it ideal for homogeneous environments where servers have similar capabilities and requests have relatively uniform processing costs.

Why Even Distribution Matters

Even distribution is critical for three reasons: performance, resilience, and scalability. From a performance standpoint, queuing theory demonstrates that response time degrades non-linearly as a single server's utilization approaches 100%. By spreading load evenly, round robin keeps average utilization lower across the fleet, reducing tail latency and preventing hot spots. For resilience, the cyclical nature combined with health checks allows the system to automatically route around failed nodes, maintaining service continuity.

For scalability, round robin enables linear horizontal scaling. Adding a new server to the pool immediately increases total capacity without requiring complex reconfiguration or state migration. This predictability makes capacity planning straightforward: if one server handles 1,000 requests per second, ten servers behind a round robin load balancer can theoretically handle close to 10,000 requests per second, minus minimal overhead.

Core Terminology

To understand round robin load balancing in depth, several key terms are essential. The load balancer (or distributor/dispatcher) is the component that receives client requests and forwards them. The backend pool or server farm is the set of identical or similar servers that actually process requests. The rotation pointer or index is the internal counter that tracks which server is next. A health check is a periodic probe that determines if a backend is alive and eligible to receive traffic. Understanding these components clarifies how the simple circular logic translates into a robust network service.

How It Works

The mechanism of round robin load balancing can be visualized as a circular queue or a clock face. Imagine five backend servers labeled A, B, C, D, and E arranged in a ring. The load balancer maintains a pointer that initially points to server A. When the first client request arrives, it is forwarded to A and the pointer advances to B. The second request goes to B, pointer moves to C, and so on. After server E handles the fifth request, the pointer wraps around to A for the sixth request, creating an endless A-B-C-D-E-A-B-C loop.

This process is entirely deterministic and requires no inter-server communication or centralized state beyond the single integer pointer. In its purest implementation at the DNS or network layer, the load balancer does not even need to track which client sent which request; it simply increments a counter modulo the number of servers (next_server = (current_index + 1) % N). This makes the algorithm exceptionally fast and easy to implement in hardware appliances, operating system kernels, and high-performance proxies like NGINX, HAProxy, and cloud load balancers.

At the transport layer (Layer 4), round robin typically operates on TCP connections. Each new TCP connection is assigned to the next backend. At the application layer (Layer 7), it operates on HTTP requests, which allows for more granular distribution, especially when combined with keep-alive connections. In both cases, the fundamental principle remains the same: strict sequential rotation without regard for payload size, connection duration, or server load.

The Role of Health Checks and Failure Handling

A naive round robin implementation would continue sending traffic to a failed server, resulting in dropped requests. Production-grade implementations therefore integrate active and passive health checks. An active health check periodically sends a synthetic probe (e.g., an HTTP GET to /health) to each backend. If a server fails to respond within a timeout or returns an error, it is temporarily removed from the rotation pool, and the modulo arithmetic is adjusted to N-1 servers. Once the server recovers and passes consecutive health checks, it is re-inserted into the ring.

Passive health checking complements this by monitoring real client traffic. If a forwarded request results in a connection error or timeout, the load balancer can immediately mark that node as unhealthy and retry the request on the next server in the sequence. This combination ensures that the simplicity of round robin does not compromise availability.

State Management and Persistence

Because round robin is stateless, it does not inherently provide session persistence (also called stickiness). A single client's successive requests may land on different backends. For stateless applications (REST APIs, static content delivery), this is ideal. For stateful applications that store session data in local memory, this can break functionality. In such cases, round robin is often augmented with persistence mechanisms like cookie-based affinity, but the core distribution for the *first* request from a new client still follows the round robin order.

Implementation

Implementing round robin load balancing in software is straightforward, yet there are important considerations around concurrency, thread safety, and dynamic membership. Below are several Python implementations that progress from a basic model to production-ready variants. Note that these examples simulate the load balancer's decision logic; in a real deployment, this logic would be integrated with a network proxy or asyncio server.

Basic Round Robin Load Balancer

This first implementation demonstrates the core circular counter with thread safety for concurrent request handling. It uses a lock to ensure the index increment is atomic, preventing two threads from receiving the same backend.

import threading
from typing import List

class RoundRobinLoadBalancer:
    """Thread-safe basic round robin load balancer."""
    def __init__(self, servers: List[str]):
        if not servers:
            raise ValueError("Server pool cannot be empty")
        self.servers = servers
        self._index = 0
        self._lock = threading.Lock()

    def get_next_server(self) -> str:
        """Return the next server in round robin order."""
        with self._lock:
            server = self.servers[self._index]
            # Advance pointer with wrap-around using modulo
            self._index = (self._index + 1) % len(self.servers)
            return server

    def add_server(self, server: str):
        with self._lock:
            self.servers.append(server)

    def remove_server(self, server: str):
        with self._lock:
            if server in self.servers:
                # Adjust index if removal affects pointer position
                remove_idx = self.servers.index(server)
                if remove_idx < self._index:
                    self._index -= 1
                self.servers.remove(server)
                if self.servers:
                    self._index %= len(self.servers)
                else:
                    self._index = 0

# Example usage
pool = ["10.0.0.1:8000", "10.0.0.2:8000", "10.0.0.3:8000"]
lb = RoundRobinLoadBalancer(pool)
for i in range(7):
    print(f"Request {i+1} -> {lb.get_next_server()}")

Weighted Round Robin Implementation

Standard round robin assumes homogeneous servers. When servers have different capacities (e.g., 8-core vs. 2-core instances), weighted round robin assigns a weight to each server and distributes requests proportionally. A server with weight 3 should receive three times the traffic of a server with weight 1.

from collections import deque

class WeightedRoundRobinLoadBalancer:
    """Weighted round robin using expanded circular list."""
    def __init__(self, weighted_servers: dict):
        """
        weighted_servers: dict like {"serverA": 3, "serverB": 1, "serverC": 2}
        """
        self.servers = []
        for server, weight in weighted_servers.items():
            if weight <= 0:
                raise ValueError("Weight must be > 0")
            self.servers.extend([server] * weight)
        
        if not self.servers:
            raise ValueError("No servers provided")
            
        self._index = 0
        self._lock = threading.Lock()

    def get_next_server(self) -> str:
        with self._lock:
            server = self.servers[self._index]
            self._index = (self._index + 1) % len(self.servers)
            return server

# Example: A(3), B(1), C(2) creates sequence [A,A,A,B,C,C]
wrr = WeightedRoundRobinLoadBalancer({"10.0.0.1": 3, "10.0.0.2": 1, "10.0.0.3": 2})
# Over 6 requests, distribution will be A:3, B:1, C:2
for i in range(6):
    print(f"Request {i+1} -> {wrr.get_next_server()}")

Smooth Weighted Round Robin (Nginx Algorithm)

The naive weighted expansion above can cause bursts (e.g., three requests to A in a row). Nginx uses a smooth weighted algorithm that interleaves servers to distribute load more evenly over short windows, preventing temporary overload of high-weight servers.

class SmoothWeightedRoundRobin:
    """Smooth weighted round robin as used in Nginx."""
    def __init__(self, weighted_servers: dict):
        self.servers = []
        for name, weight in weighted_servers.items():
            self.servers.append({
                "name": name,
                "weight": weight,
                "current_weight": 0
            })
        self.total_weight = sum(s["weight"] for s in self.servers)

    def get_next_server(self) -> str:
        best = None
        for server in self.servers:
            server["current_weight"] += server["weight"]
            if best is None or server["current_weight"] > best["current_weight"]:
                best = server
        
        if best is None:
            return None
            
        best["current_weight"] -= self.total_weight
        return best["name"]

# Smooth distribution: A(3), B(1), C(2) -> interleaved, not bursty
smooth = SmoothWeightedRoundRobin({"A": 3, "B": 1, "C": 2})
for i in range(6):
    print(f"Request {i+1} -> {smooth.get_next_server()}")
# Output sequence is typically A, C, A, B, A, C - much smoother

Round Robin with Health Checks and Failure Detection

A production-ready balancer must handle dynamic failures. This example adds health status tracking and automatically skips unhealthy nodes while maintaining the round robin order for healthy ones.

import time
import threading

class HealthAwareRoundRobin:
    """Round robin that skips unhealthy servers."""
    def __init__(self, servers: List[str]):
        self.servers = {s: {"healthy": True, "fail_count": 0} for s in servers}
        self.server_list = servers
        self._index = 0
        self._lock = threading.Lock()

    def mark_unhealthy(self, server: str):
        with self._lock:
            if server in self.servers:
                self.servers[server]["healthy"] = False

    def mark_healthy(self, server: str):
        with self._lock:
            if server in self.servers:
                self.servers[server]["healthy"] = True
                self.servers[server]["fail_count"] = 0

    def get_next_server(self) -> str:
        with self._lock:
            if not self.server_list:
                raise RuntimeError("No servers available")
            
            # Try at most N times to find a healthy server
            for _ in range(len(self.server_list)):
                candidate = self.server_list[self._index]
                self._index = (self._index + 1) % len(self.server_list)
                if self.servers[candidate]["healthy"]:
                    return candidate
            
            raise RuntimeError("All backend servers are unhealthy")

# Simulate failure scenario
lb = HealthAwareRoundRobin(["srv1", "srv2", "srv3"])
print(lb.get_next_server())  # srv1
lb.mark_unhealthy("srv2")
print(lb.get_next_server())  # srv3 (skips srv2)
print(lb.get_next_server())  # srv1
print(lb.get_next_server())  # srv3 again

Step-by-Step Example

To concretize the theory, let us walk through a complete lifecycle of round robin load balancing with a realistic scenario. Consider an e-commerce platform running three identical web servers behind a Layer 7 load balancer. The servers are S1 (192.168.1.10), S2 (192.168.1.11), and S3 (192.168.1.12). The load balancer's pointer starts at index 0 (S1). A health check interval is set to 10 seconds.

At T+0 seconds, eight client requests arrive in rapid succession. The distribution proceeds as follows: Request 1 from Client A is routed to S1, pointer moves to 1. Request 2 from Client B goes to S2, pointer to 2. Request 3 from Client C goes to S3, pointer wraps to 0. Request 4 from Client A (second request) goes to S1 again, pointer to 1. This continues until Request 8, which lands on S2. After eight requests, the final distribution is S1: 3 requests (1,4,7), S2: 3 requests (2,5,8), S3: 2 requests (3,6). Over any number of requests divisible by three, the distribution is perfectly equal.

Now consider a failure event. At T+15 seconds, S2 crashes due to an out-of-memory error. The next active health check at T+20 seconds detects that S2 fails its HTTP probe. The load balancer immediately removes S2 from the active pool, shrinking N from 3 to 2. The server list is now [S1, S3] and the pointer is adjusted to maintain correct circular order. The next incoming request, Request 9, would have gone to S3 anyway, so it proceeds to S3. Request 10 goes to S1, Request 11 to S3, and so on, now alternating evenly between the two remaining healthy nodes. No client requests are sent to the failed S2 during this period, preventing errors.

Recovery and Rebalancing

At T+120 seconds, an automated remediation script restarts S2, and it begins passing health checks again. After two consecutive successful probes, the load balancer re-adds S2 to the pool, restoring N to 3. The re-insertion point is typically at the end of the list or at its original position to preserve ordering. The pointer continues from its current position, ensuring the distribution remains balanced without a thundering herd to the newly recovered node. Over the next six requests, each server again receives exactly two requests, demonstrating the algorithm's self-healing property.

This walkthrough highlights a key operational characteristic: round robin provides fairness over time, not necessarily at any single instant. If requests have vastly different processing times, a snapshot might show S1 handling three long-running video transcoding jobs while S3 is idle, even though request counts are equal. This is why understanding workload homogeneity is crucial when choosing this algorithm.

Complexity Analysis

One of the primary reasons round robin remains dominant after decades is its exceptional computational efficiency. The algorithm's time and space complexity are optimal for a high-throughput data plane component that may need to make millions of routing decisions per second with minimal latency overhead.

The core operation — selecting the next backend — involves only an array lookup, an integer increment, and a modulo operation. All of these are constant-time CPU instructions. There is no searching, sorting, or iteration over the server list in the basic implementation. Even with health checks, the selection process in the worst case iterates over N servers only when nearly all are unhealthy, which is an exceptional failure mode rather than the common path.

Space complexity is equally minimal. The load balancer needs to store only the list of backend addresses (N entries) and a single integer pointer, plus optional health status flags. No per-connection or per-client state is required for the basic algorithm, making its memory footprint O(N) where N is the number of backends, which is typically small (tens or hundreds), not the number of concurrent connections (which can be millions).

The table above clarifies an important nuance: while basic round robin is strictly O(1), the smooth weighted variant trades a small amount of CPU time (O(N) per selection) for better distribution smoothness. For typical pool sizes of < 100 servers, this O(N) scan is negligible (a few microseconds) and is still far cheaper than dynamic algorithms that must query system metrics or maintain connection counters. This is why even weighted round robin is considered highly performant for most network engineering contexts.

Impact on Latency and Throughput

In practical terms, the O(1) selection time means the load balancer itself adds almost zero latency to the request path — often less than 0.1 milliseconds. This allows a single load balancer instance to handle hundreds of thousands of requests per second on modest hardware. The bottleneck is almost always network I/O or backend processing, not the scheduling algorithm. This predictability is vital for providing strict Service Level Agreements (SLAs) on p99 latency.

Variations and Optimizations

While classic round robin is powerful in homogeneous environments, real-world infrastructure is rarely perfectly uniform. Over the years, several important variations have evolved to address heterogeneity, statefulness, and improved fairness without sacrificing the core simplicity.

Weighted Round Robin (WRR)

Weighted round robin is the most common variant. Each server is assigned a static weight proportional to its capacity. For example, in a pool with two servers where Server A has 4 vCPUs and Server B has 2 vCPUs, you might assign weights of 2 and 1 respectively. The load balancer will then send two requests to A for every one request to B, aligning load with capacity. Weights can be based on CPU, memory, network bandwidth, or a composite benchmark score. This is widely supported in NGINX (weight parameter), HAProxy, and AWS Application Load Balancer target groups.

The naive implementation simply repeats servers in the rotation list according to weight, but as shown earlier, this creates bursts. The smooth weighted round robin optimization solves this by using a current_weight accumulator to interleave selections, ensuring that over any small window, the distribution matches the desired ratio as closely as possible.

Weighted Round Robin with Dynamic Adjustment

Some advanced implementations adjust weights dynamically based on observed response times or error rates, blurring the line between static and dynamic algorithms. For instance, if a high-weight server starts showing elevated latency, its effective weight can be temporarily reduced. While powerful, this requires careful tuning to avoid oscillation and is often better served by a fully dynamic algorithm like least connections or latency-aware routing.

Round Robin with Sticky Sessions and Consistent Hashing Fallback

To support stateful applications, round robin is often combined with session affinity. The first request from a new client follows round robin, but the load balancer then sets a cookie (e.g., SERVERID) that pins subsequent requests from that client to the same backend. This hybrid approach provides the even initial distribution of round robin while preserving session locality. Another optimization is to use round robin as a fallback when a consistent hash ring is being rebalanced due to node changes, ensuring availability during transitions.

Real-World Applications

Round robin load balancing is ubiquitous across the networking stack, from low-level DNS to high-level application delivery. Its versatility and low overhead make it the default choice in many systems where simplicity and predictability are valued over fine-grained adaptivity.

Web Servers and Reverse Proxies

The most visible application is in front of web server fleets. NGINX, HAProxy, Apache Traffic Server, and Envoy all offer round robin as their default or simplest load balancing method. A typical deployment might have an NGINX reverse proxy in front of ten Node.js or Python Gunicorn application servers. Every incoming HTTP request is cycled through the app servers, allowing the site to handle an order of magnitude more traffic than a single server could. Cloud providers abstract this further: AWS Elastic Load Balancing, Google Cloud Load Balancing, and Azure Load Balancer all provide managed round robin distribution across availability zones.

DNS Load Balancing

DNS round robin is one of the oldest techniques on the internet. When a client queries the DNS for a domain like example.com, the authoritative DNS server responds with a list of IP addresses (A records) in a rotating order. The client typically attempts the first IP in the list. By rotating the order of IPs for each query, DNS spreads clients across multiple data centers or edge nodes. While crude (it is affected by DNS caching and does not handle failures quickly), it remains a critical first layer of global load distribution and is used by Content Delivery Networks (CDNs) for geographic traffic steering.

Databases and Caching Layers

In database architectures, round robin is used to distribute read queries across read replicas. A primary database handles writes, while multiple read replicas serve read-heavy workloads like reporting and user profile lookups. A database proxy such as ProxySQL or Amazon RDS Proxy can use round robin to evenly spread SELECT statements across replicas, scaling read throughput linearly. Similarly, application clients for distributed caches like Redis Cluster or Memcached often use client-side round robin to distribute cache gets and sets across shards.

Microservices and Container Orchestration

In Kubernetes, the default Service type kube-proxy uses a round robin-like iptables/IPVS rule to distribute traffic to pods backing a Service. When a microservice makes a call to http://user-service, Kubernetes automatically load balances that call across all healthy user-service pods in a round robin fashion. Service meshes like Istio and Linkerd also implement round robin as a baseline load balancing policy for inter-service communication, providing resilience and scalability for cloud-native architectures without requiring application-level logic.

Comparison with Alternatives

Choosing the right load balancing algorithm requires understanding the trade-offs between simplicity, fairness, and adaptivity. Round robin excels at simplicity and fairness in homogeneous environments, but other algorithms may be superior when workloads are heterogeneous or server states vary dramatically.

Least Connections is a dynamic algorithm that forwards each new request to the server with the fewest active connections. It is superior when requests have variable processing times (e.g., long-lived WebSocket connections mixed with short API calls), as it naturally avoids overloading a server stuck with heavy jobs. IP Hash (or consistent hashing) hashes the client's IP address to deterministically map a client to the same backend, providing session persistence without cookies — useful for stateful services but can lead to imbalance if client distribution is skewed.

Least Response Time and Resource-Based algorithms go further by using real-time latency or CPU/memory metrics to make decisions, offering the best adaptivity at the cost of complexity and monitoring overhead. Random and Power of Two Choices are probabilistic alternatives that can achieve surprisingly good balance with less coordination, especially in very large fleets.

In practice, many organizations start with round robin due to its predictability and low operational burden, then graduate to weighted round robin or least connections only when monitoring reveals persistent imbalance. For example, if dashboards show one server consistently at 80% CPU while others are at 30% despite equal request counts, it is a clear signal that requests are not uniform and a dynamic algorithm is warranted. The key is to measure before optimizing.

Common Mistakes and Best Practices

Despite its simplicity, round robin load balancing is frequently misconfigured in ways that negate its benefits or introduce subtle failures. Awareness of these pitfalls is essential for network engineers and DevOps practitioners.

Common Mistakes

1. Using Round Robin for Heterogeneous or Variable Workloads Without Weights: The most common error is deploying classic round robin in front of servers with different CPU/memory capacities or for requests with wildly different costs. This leads to the more powerful servers being underutilized while weaker servers become bottlenecks. If a video transcoding request takes 10 seconds and an API health check takes 10 milliseconds, equal request counts do not mean equal load.

2. Ignoring Health Checks and Draining: Failing to configure robust active health checks means failed nodes continue to receive traffic until manually removed, causing user-visible errors. Equally important is connection draining: when removing a server for maintenance, the load balancer should stop sending new requests but allow in-flight requests to complete, rather than abruptly terminating them.

3. Not Accounting for DNS Caching in DNS Round Robin: Engineers often assume DNS round robin provides even distribution, but intermediate DNS resolvers and client operating systems cache DNS responses for the duration of the TTL. With a TTL of 3600 seconds, a client may be pinned to a single IP for an hour, defeating the rotation. Best practice is to use a very low TTL (30-60 seconds) for DNS round robin or use it only as a coarse first layer in front of proper proxy-based load balancing.

4. Lack of Thread Safety: In custom implementations, forgetting to protect the rotation pointer with a lock or atomic operation in a multi-threaded or async environment leads to race conditions where two requests are sent to the same server and another server is skipped, breaking fairness and potentially causing duplicate processing.

Best Practices

1. Prefer Weighted Round Robin When in Doubt: Even in seemingly homogeneous environments, small differences in noisy neighbor effects or instance performance can accumulate. Starting with weighted round robin and setting all weights equal gives you the flexibility to adjust without changing algorithms later. Automate weight assignment based on instance type where possible.

2. Implement Comprehensive Health Checking: Configure both active checks (HTTP 200 on /health, TCP connect) and passive checks (monitor 5xx errors). Use an exponential backoff for marking nodes unhealthy to avoid flapping, and require multiple consecutive successes before re-adding a node. Log health transitions for alerting.

3. Monitor Distribution and Backend Metrics: Do not assume balance; measure it. Export metrics for requests per backend, active connections per backend, CPU, and latency. Set alerts for imbalance (e.g., if any backend receives > 20% more traffic than the average over 5 minutes). This data informs whether round robin remains appropriate.

4. Combine with Autoscaling and Graceful Degradation: Integrate the load balancer with an autoscaler so the pool size N adjusts to demand while round robin continues to distribute evenly. Ensure the system degrades gracefully when N=1 or N=0 by returning a meaningful 503 with Retry-After rather than crashing. Always test failure scenarios with chaos engineering tools.

5. Use Keep-Alive and Connection Pooling Wisely: For Layer 7 HTTP load balancers, enable keep-alive between the load balancer and backends to reduce connection overhead, but be aware that very long keep-alive connections can temporarily pin traffic. Periodically recycle connections to allow the round robin rotation to re-balance.

Interview Questions

Round robin load balancing is a frequent topic in system design and networking interviews because it tests understanding of distributed systems fundamentals, trade-offs, and practical operations. Below are common questions with guidance on strong answers.

1. Explain how round robin load balancing works and what its time complexity is. When would you choose it over least connections?

A strong answer should describe the circular pointer and modulo arithmetic, state that selection is O(1) time and O(N) space for the pool, and emphasize that it is ideal for stateless, homogeneous workloads with uniform request costs (e.g., serving static assets or stateless REST APIs). Contrast with least connections, which is O(N) or O(log N) and better for variable-cost or long-lived connections like WebSockets or database queries where active connection count is a better proxy for load. Mention that round robin's simplicity and lack of state make it more performant and predictable.

2. What happens in round robin if one server goes down? How would you design the system to handle failures without dropping user requests?

Interviewers look for discussion of health checks. Explain that naive round robin would black-hole requests to the failed node. A robust design uses active health probes (e.g., every 5 seconds) to remove unhealthy nodes from the rotation and passive checks to detect failures on live traffic. Upon failure, the load balancer should skip the node and optionally retry the request on the next healthy node (with idempotency considerations). Also mention connection draining, alerting, and automatic re-addition after recovery. For DNS round robin, note its poor failure handling due to caching and the need for low TTLs or a proxy layer.

3. How does weighted round robin differ from classic round robin, and how would you implement smooth weighted round robin?

Answer should define weights as capacity ratios and explain that weighted round robin sends proportionally more traffic to higher-weight servers. Describe the naive expansion method (repeating servers in a list) and its burstiness problem. Then explain the smooth algorithm (Nginx's method) where each server maintains a current_weight that is incremented by its weight each round and the server with the highest current_weight is selected, then its current_weight is decremented by total_weight. This interleaves selections smoothly. Provide complexity trade-off: naive is O(1) but bursty, smooth is O(N) but fairer over short windows.

4. You have a microservice that stores user session data in local memory. Can you use round robin load balancing? What are the alternatives?

This tests understanding of statefulness. Pure round robin would break sessions because consecutive requests from the same user would hit different servers and miss local session data. Solutions include: (a) making the service stateless by externalizing session to Redis or a database, which then allows round robin; (b) adding sticky sessions via cookies so the load balancer pins a user to one backend after the first round robin assignment; or (c) using IP hash / consistent hashing to deterministically map users to servers. Discuss the trade-off that stickiness can cause imbalance and reduces resilience if a server fails (sessions are lost), so externalizing state is generally preferred for cloud-native design.

5. Compare DNS round robin and proxy-based (NGINX/HAProxy) round robin. What are the pros and cons of each?

A comprehensive answer contrasts layers. DNS round robin operates at the DNS layer, requires no proxy hop, is simple and scalable, but suffers from caching (uneven distribution), slow failover (TTL dependent), no health awareness, and coarse granularity (per-client, not per-request). Proxy-based round robin operates at Layer 4/7, provides per-request distribution, instant health checks, weighted support, and observability, but introduces an extra network hop, is a potential single point of failure (mitigated by HA pairs), and has throughput limits. In practice, they are often layered: DNS round robin for global distribution across regions, and proxy round robin within each region for precise balancing.