I’ll never forget the 3 AM pager that taught me everything about GSLB the hard way. We’d built a beautiful multi-region CDN footprint across Virginia, Frankfurt, and Singapore, and a single fiber cut in Ashburn sent 60% of our global traffic into a black hole for 11 minutes. Our DNS TTL was set to 300 seconds and our health checks were polling the wrong endpoint — we’d optimized for cost instead of resilience. That night is why I don’t trust any global load balancing design I can’t break on purpose first.
Why DNS-Based GSLB Still Breaks (And How We Fix It)
Most teams start their GSLB journey thinking DNS is simple: point your authoritative nameserver at a few A records and let round-robin do the work. It isn’t. Traditional DNS has no concept of load, latency, or server health — it just hands out answers. I’ve audited deployments where a “global” setup was literally just three static A records with a 3600-second TTL, which means clients and recursive resolvers cache a dead PoP for an hour. You can’t do effective global load balancing without controlling both the resolution path and the health signal.
The pattern that actually works in production combines short TTLs, EDNS Client Subnet (ECS) awareness, and active health checks at the edge. We typically run TTLs between 20-60 seconds for GSLB records — that’s the sweet spot we found balancing resolver load and failover speed after testing with real ISP resolvers. Anything shorter and you’ll get hammered by query volume; anything longer and you’re lying to yourself about your RTO. And if your authoritative DNS doesn’t support ECS, you’re geolocating the resolver, not the user, which is useless for a CDN. Google Public DNS and Cloudflare both support ECS, but many enterprise resolvers strip it, so you need a fallback.
For comparison with local PoP balancing, don’t confuse this with data-center-local algorithms. Inside a single region you might use Weighted Round-Robin for CDN: Proportional Traffic Distribution Across Heterogeneous Servers or Least-Connections Load Balancing for CDN: Dynamic Distribution Based on Server Load to spread load across caches — GSLB sits one layer above that and decides which region should handle the request at all. The specs that govern how clients actually follow those decisions, from cache-control to retry semantics, are well documented in the HTTP Working Group Specifications.
# Production GSLB pool with ECS-aware routing + active health checks
# Example using PowerDNS / Route 53 style config
gslb_pool "cdn-edge-global" {
ttl = 30 # seconds - failover vs. query load tradeoff
edns_client_subnet = true
fallback_geo = "us-east-1"
pool_member "iad-edge" {
ip = 198.51.100.10
weight = 100
geo = "NA-East"
health_check {
type = "https"
path = "/__health?check=deep"
interval = 5s
timeout = 2s
threshold_healthy = 2
threshold_unhealthy = 3
expected_status = 200
# Check origin reachability, not just nginx alive
expected_body_regex = "\"origin_reachable\":true"
}
}
pool_member "fra-edge" {
ip = 203.0.113.45
weight = 100
geo = "EU-Central"
health_check { interval = 5s; path = "/__health?check=deep" }
}
pool_member "sin-edge" {
ip = 192.0.2.78
weight = 80 # smaller capacity PoP
geo = "APAC"
}
}
Active-Active vs. Active-Passive: Choosing Your GSLB Pattern
When I’m designing for a new client, the first question isn’t “which vendor” — it’s “can you run active-active?” Active-passive is comfortable: one primary region serves 100% of traffic and the secondary sits warm, waiting for a failover. We used that pattern for a financial services CDN in 2021 because their origin was strongly consistent and couldn’t handle multi-writer conflicts. It works, but you’re paying for idle capacity and your failover is always a cold event. We measured 45-90 seconds for DNS propagation alone during our chaos tests, even with a 30-second TTL, because of resolver non-compliance.
Active-active is where you get real value from GSLB, but it demands more from your application. Every region serves traffic concurrently, and GSLB steers users to the closest healthy PoP. That means you need eventual consistency or CRDTs at the origin, cache purging that’s globally coherent, and health checks that understand “degraded” not just “up/down.” I’ve seen teams fail here because their health check was a simple TCP connect to port 443 — the edge was up, but the cache fill path to origin was timing out, so users got 500s from the “healthy” region. Now we always implement layered checks: L4, HTTP edge, and origin reachability, and we expose that state to the GSLB control plane.
There’s also a hybrid you’ll see a lot: active-active for static assets with active-passive for dynamic APIs. For APIs, you might need session affinity, which is where IP Hash for CDN Session Persistence: Consistent User Routing Without Cookies can keep a user pinned to a region during a session. But be careful — IP hash at the global layer breaks when carrier-grade NAT puts 10,000 mobile users behind one IP. We’ve moved most dynamic routing to latency-based steering with application-level redirects (HTTP 302 with Alt-Svc) instead of pure DNS. Guidance on how browsers handle those redirects and connection reuse is covered well in web.dev Articles, and it’s worth testing before you rely on it.
// Application-level GSLB failover using latency + health score
// Runs at edge worker / L7 load balancer
async function selectRegion(clientSubnet, regions) {
const scores = await Promise.all(regions.map(async (r) => {
const health = await getHealthScore(r.id); // 0.0 - 1.0
const latency = await getP95Latency(r.id, clientSubnet);
const load = await getCpuLoad(r.id);
// Don't route to a region that's degraded, even if it's close
if (health < 0.6 || load > 0.85) return { region: r, score: Infinity };
// Weighted score: 70% latency, 30% load
const score = (latency * 0.7) + (load * 100 * 0.3);
return { region: r, score, health, latency };
}));
scores.sort((a, b) => a.score - b.score);
// Set Alt-Svc + cache-control for retry behavior per httpwg specs
const best = scores[0];
if (best.health < 1.0) {
// Tell client to retry other region quickly
setHeader("Cache-Control", "max-age=10, stale-while-revalidate=5");
}
return best.region;
}
Health Checks That Prevent Failover Flapping
Health checks are where GSLB patterns live or die, and I’ve got the scars to prove it. One e-commerce client had a “healthy” threshold of 1 success and an “unhealthy” threshold of 1 failure, polling every 2 seconds. A single packet loss blip in Frankfurt caused a global flap that sent 40k RPS to Virginia in under 5 seconds, which then overloaded Virginia — a classic cascade. We fixed it by requiring 3 consecutive failures to mark down and 2 successes to mark up, with a 5-second interval and a separate “observation” state that bleeds traffic back slowly.
What you monitor matters more than how often. Don’t just check if nginx is responding 200 OK. I always push teams to add a synthetic cache-fill check, a check for origin latency, and a check for error rate. If your edge is returning 5xx for 2% of requests because disks are failing, it’s not healthy even if the health endpoint is green. We export those metrics to our GSLB controller and use them to adjust weights automatically — a region at 1.5% error rate gets its weight halved before it’s marked down entirely. That graceful degradation has saved us more times than hard failovers have.
| GSLB Health Strategy | Failover Time (p95) | False Positive Rate | Query Overhead |
|---|---|---|---|
| DNS TTL 300s + TCP Check Only | 320 - 420 seconds | ~8.5% during network jitter | Low (~1.2k QPS per PoP) |
| DNS TTL 30s + HTTP + Origin Check | 38 - 55 seconds | ~1.1% with 3/2 thresholds | Medium (~8k QPS per PoP) |
| Anycast + Active Health + L7 Steering | 4 - 12 seconds | ~0.4% with graceful bleed | High (requires BGP + edge workers) |
| Latency-Based (RUM) + Passive Health | 60 - 90 seconds | ~2.3% without hysteresis | Very High (RUM beacon volume) |
From Theory to Practice: Deploying EDNS-Aware GSLB with RUM Feedback
Knowing why DNS GSLB breaks is one thing — actually shipping a setup that survives a regional brownout without melting your origin is another. We've found the sweet spot isn't pure latency-based routing or pure weighted round-robin, it's a hybrid. You start with GeoIP/EDNS Client Subnet (ECS) for initial steering, then adjust weights dynamically using Real User Measurements (RUM). Don't rely on synthetic probes alone; your lab in Ashburn doesn't look anything like a user on a congested Jio network in Mumbai.
Here's how we implement it in production for a 12-PoP CDN handling ~4.5 Tbps peak. Each PoP advertises its load and p95 TTFB via a central director every 5 seconds. The authoritative GSLB controller — we use PowerDNS with a custom Lua backend, but Route 53, NS1, and Avi Vantage work the same — recalculates weights every 15 seconds. TTLs are set brutally low: 30 seconds for A/AAAA records, 15 seconds during a failover event. Yes, it'll increase your query volume by 3-4x, but it's the only way to drain a failing region in under 45 seconds. If you're still using 300s TTLs in 2026, you're not doing GSLB, you're doing hope.
The trick is not to flip 100% of traffic at once. We use a consistent hashing layer behind the GSLB so existing connections don't all rehash to a new edge and cause a thundering herd on cache. If you've not nailed this part, read up on Consistent Hash Ring Architecture for CDN: Minimizing Redistribution on Server Changes — without it, a single PoP failure can drive your origin fetch rate from 8% to 65% in seconds. This is the glue between your DNS steering and your cache efficiency.
# PowerDNS Lua GSLB - RUM-weighted latency routing (simplified)
-- recalculates every 15s from RUM + health data
function getGSLBWeights(regionStats)
local weights = {}
for pop, stats in pairs(regionStats) do
if stats.health == "ok" and stats.p95_ttfb < 250 then
-- weight = capacity * (1 / latency_penalty) * health_factor
local latencyPenalty = 1 + (stats.p95_ttfb / 1000)
weights[pop] = stats.available_capacity * (1 / latencyPenalty)
else
weights[pop] = 0 -- blackhole failing PoP
end
end
return normalize(weights)
end
-- DNS response with ECS awareness and 30s TTL
-- e.g., US-East user -> iad:60, ord:30, lhr:10 (backup)
-- APAC user -> sin:70, nrt:25, iad:5
Hybrid Anycast + DNS Patterns and Troubleshooting Failover That Won't Fail Back
At scale, you'll almost never run pure DNS GSLB. The pattern that's won for us for multi-region video and large file delivery is hybrid: Anycast for the first hop (fast, BGP-driven) with DNS GSLB as the brain. Anycast gets you that 15-25ms TCP handshake win, while DNS does the cold-potato steering — we call it "controlled Anycast." Your Anycast announces all PoPs from the same prefix, but your GSLB tells users in Berlin to prefer the Frankfurt Anycast node via EDNS steering, rather than letting BGP decide and potentially dragging them to New York. It gives you deterministic routing without giving up Anycast's DDoS absorption.
Now, the problem everyone hits: failover works, failback doesn't. You'll drain Virginia, everything looks green, but Virginia never reclaims traffic because your health checks are lying to you. This usually comes down to two mistakes. First, you're only doing shallow TCP checks. A PoP can return 200 OK on /health while its cache tier is at 98% CPU and serving 4-second TTFBs. You need layered checks, and we've standardized on the model from Health Check Design Patterns for CDN: Shallow, Deep, and Synthetic Checks — shallow (TCP/HTTP 200) every 5s, deep (cache fetch + origin validation) every 15s, and synthetic transaction (full object pull) every 60s. If deep fails twice, we halve the weight; if synthetic fails, we zero it. Second, you're missing passive observation. Don't just probe, watch.
That second point is critical for preventing flapping. Active probing told us Ashburn was "healthy" last March, but passive metrics showed a 12% increase in 5xx and retransmits for 30 seconds before probes even noticed. That's why we run both in parallel, as detailed in Active vs Passive Health Checks for CDN: Probing vs Observation-Based Monitoring. Our rule now: active checks gate whether we *add* traffic back, passive checks (real request error rate > 1.5% or p95 > 400ms over 30s window) gate whether we *remove* it. And for failback, never go 0-to-100. Use linear ramp-up: 10% -> 25% -> 50% -> 100% over 4 minutes with automatic rollback if error rate spikes. It feels slow, but it prevents that yo-yo failover that makes your NOC hate you.
Frequently Asked Questions
What TTL should I actually set for GSLB records?
For most CDNs, 30 seconds is the practical default and 15 seconds during an active failover. Anything under 15s will get ignored or rate-limited by public resolvers like Google DNS and Cloudflare 1.1.1.1, so you don't gain faster drain but you do pay in query costs. Anything over 60s means you're waiting 2-3 minutes to drain a burning PoP, which violates most SLAs. We keep SOA TTL at 30s and drop to 15s via API when our director declares a PoP unhealthy.
Can I do GSLB without EDNS Client Subnet support?
You can, but you'll be steering based on the resolver's location, not the user's — and that can be off by thousands of miles. Without ECS, a user in Sydney using 8.8.8.8 (which may egress from the US) could get steered to Virginia instead of Singapore, adding 140ms+ of latency. About 78% of public resolvers now forward ECS. If your authoritative DNS doesn't support it, you're flying blind for nearly all ISP traffic. Always enable and test ECS.
Why does health check flapping still happen even with 3x retries?
Because you're retrying the wrong check. Flapping usually isn't network jitter — it's your check being too shallow and toggling on transient app issues. Three TCP successes don't mean the cache is healthy. Implement hysteresis: require 3 consecutive failures over 15 seconds to mark down, but 5 consecutive successes over 45-60 seconds to mark up. And combine it with passive monitoring — if active probes are green but real user error rates are elevated, trust the users.
Should I choose Anycast or DNS GSLB for my CDN?
Don't choose — combine them if you're over ~5 PoPs or 500 Gbps. Use Anycast for resilience and DDoS mitigation at the edge, and use DNS GSLB for intelligent, load-aware steering on top of it. Pure DNS gives you control but slower failover (30-45s). Pure Anycast gives you fast failover (<3s BGP reconvergence) but zero control over load or cold-potato routing. The hybrid gives you the best of both, at the cost of more complex automation.