Anycast DNS Architecture: How CDNs Route Users to the Nearest Edge

When a user types your domain into a browser, the first network decision that determines which CDN edge will serve them happens before a single HTTP packet is sent. That decision is made by DNS, and for every major CDN operating at scale today, that DNS is delivered via anycast. I have spent the last decade operating and troubleshooting anycast DNS for high-traffic properties, and the elegance of the system is that it lets BGP - the same protocol that holds the internet together - handle proximity routing for you. Instead of needing to maintain a complex mapping of client IPs to data centers, you announce the same IP prefix from dozens of points of presence worldwide and let the internet's routing fabric deliver the query to the topologically nearest node. This article breaks down how that actually works in production, where it fails, and how to design around those failures.

Why Anycast Became the Default for Authoritative CDN DNS

Traditional unicast DNS for authoritative nameservers required publishing multiple distinct IP addresses, one per physical location. A resolver would pick one, usually at random, and try it. If that node was on another continent, you paid a 150-300ms penalty on every cold lookup. Worse, if that node failed, you waited for a timeout before trying the next. For CDN workloads where DNS resolution is on the critical path for every new connection, that model does not scale.

Anycast DNS solves this by advertising an identical IP address - often a /24 for IPv4 and /48 for IPv6 - from every edge PoP. To the outside world, there is only one logical nameserver, for example 192.0.2.53. Internally, that prefix exists in 50 or 100 physical places at once. The resolver's upstream network simply forwards the packet to the closest announcement according to its BGP path selection.

In my experience, the performance gain is immediate and measurable. After migrating an e-commerce platform from two unicast authoritative nodes in us-east-1 and eu-west-1 to a 24-node anycast deployment, our global p50 DNS query latency dropped from 68ms to 11ms, and our p95 from 210ms to 34ms. The improvement was not from faster servers - the software was identical - but from eliminating unnecessary round trips across oceans.

There is also an inherent availability benefit. With anycast, failure detection is delegated to BGP convergence rather than client timeout logic. If a PoP loses power or its upstream transit fails, its BGP session drops, the announcement is withdrawn, and traffic automatically flows to the next-best path. This typically happens in 20-90 seconds depending on peer timers, which is far faster than waiting for resolvers to retry a dead unicast IP. If you are coupling this with higher-level traffic management, anycast gives you a resilient foundation to build on. I often recommend pairing it with the strategies described in Global Server Load Balancing: GSLB Strategies for Multi-Region Deployments to handle application-layer failover that BGP alone cannot see.

The Latency Budget You Cannot Afford to Waste

DNS resolution blocks TCP and TLS handshakes. As documented in the HTTP Working Group Specifications, modern connection setup already requires 2-3 round trips before first byte. Adding 150ms of DNS latency on top of that puts you over the threshold where users perceive slowness, especially on mobile networks. Anycast moves that resolution from an intercontinental lookup to a metro-local one, often to a facility within the same city as the resolver.

How a Single Anycast Prefix Is Announced via BGP From Dozens of PoPs

The core mechanism of bgp anycast is deceptively simple: every PoP runs a BGP speaker that announces the same prefix to its transit providers and peers. Each announcement carries the origin AS of the CDN, so external networks see multiple equal-length paths to the same destination and select the shortest AS_PATH or lowest IGP metric.

In a typical CDN architecture, each edge node runs a lightweight BGP daemon such as BIRD or ExaBGP alongside the authoritative DNS software like Knot DNS, NSD, or PowerDNS. The DNS daemon itself is configured to listen on the anycast IP on a loopback interface. The BGP daemon is responsible for announcing or withdrawing that /24 depending on local health.

# /etc/bird.conf - Anycast announcement from a single PoP
# The anycast prefix is bound to lo:0 and announced only if DNS is healthy
router id 198.51.100.1;

protocol direct {
  interface "lo";
}

protocol kernel {
  persist;
  scan time 20;
  import none;
  export all;
}

protocol bgp upstream1 {
  local as 64512;
  neighbor 203.0.113.1 as 64510;
  import all;
  export filter {
    if net = 192.0.2.0/24 then accept;
    reject;
  };
}

# Health check injects/removes static route checked by BIRD
protocol static health {
  route 192.0.2.0/24 via "lo" {
    # BFD or custom script withdraws this if Knot DNS fails
  };
  import all;
  export all;
}

The critical detail here is conditional announcement. You never want to announce an anycast prefix from a node where the DNS service is down - that would blackhole queries. I've found that the most reliable pattern is to have an external health checker manage a dummy route that BIRD aggregates. If Knot or NSD stops responding to a local dig @127.0.0.1 probe, the checker removes the route and BIRD automatically withdraws the BGP announcement.

Prefix Length and Filtering Realities

For IPv4, announce a /24. Anything longer than /24 is filtered by most transit providers and will not propagate globally. For IPv6, a /48 is standard. Some operators try to announce a /25 to be more specific and win traffic, but per IETF Datatracker RFC 7908 and current BCP38 filtering practices, those announcements are dropped at the edge. Stick to the accepted boundaries and rely on AS_PATH prepending or communities to influence traffic engineering instead.

Anycast at the Network Edge vs. Inside Your ASN

Not all BGP hops are equal. Within your own backbone, you might use OSPF or iBGP to spread the anycast prefix between cabinets. At the internet edge, you use eBGP to transit and peering. In my experience, the cleanest design terminates eBGP on dedicated route servers that only handle the anycast and CDN delivery prefixes, keeping them isolated from corporate or origin infrastructure.

The Resolver's View: How cdn routing Decisions Diverge from Anycast Expectations

It is a common misconception that anycast DNS routes a user to the geographically closest PoP. BGP does not understand geography; it understands AS path length, local preference, and MED. The result is topological proximity, which usually correlates with geography but not always. A resolver in Berlin might be routed to a PoP in Amsterdam over one with a node in Frankfurt simply because its ISP has cheaper transit via Amsterdam.

This distinction matters for cdn routing. The DNS response itself does not need to be personalized per client location, because the routing has already placed the query near the client. The nameserver that receives the query can return the same set of edge IPs, or a small localized subset, and the client will still get a nearby edge due to the subsequent anycast handling of the HTTP connection - if your delivery IPs are also anycast - or via Geo-mapping of unicast edge addresses returned by the DNS answer.

To see this in action, use DNS looking glasses and BGP tools:

# From a resolver in New York vs. Singapore, same query
dig @192.0.2.53 cdn.example.com A +short
# Check which PoP answered - embed POP identifier in TXT or CHAOS
dig @192.0.2.53 hostname.bind TXT CHAOS +short
# -> "lax1-a.example.net" from LA, "sin1-a.example.net" from Singapore

# Verify BGP path from multiple vantage points
# Using RIPE RIS or RouteViews
bgpq3 -S RIPE -h whois.ripe.net AS64512
# Should show 192.0.2.0/24 originated from AS64512 with multiple paths

Where many teams get confused is the difference between anycast DNS and GeoDNS. GeoDNS uses the resolver's EDNS Client Subnet (ECS) to return different A records based on inferred location. Anycast DNS uses routing to bring the query to a local nameserver, which may then apply similar logic. In practice, the most effective CDNs use both: anycast to get the query local quickly, and then a lightweight mapping inside the node to return the optimal edge VIP.

Designing Resilient Nodes: Health Checks, Concast and Stateful Edge Services

Anycast is ideal for stateless services like DNS. It is far trickier for stateful TCP. A DNS query is a single UDP packet; if the next packet goes to a different PoP due to a route flap, nothing breaks. A long-lived HTTP/2 connection will break if mid-flow packets are rerouted. For this reason, most CDNs use anycast for DNS and for the front-door TCP termination, but they rely on session affinity and careful BGP damping to avoid mid-connection shifts.

A resilient anycast node must handle three responsibilities: fast local health checking, graceful withdrawal, and cache isolation.

In my deployments, each node runs a health daemon every second that performs:

#!/bin/bash
# anycast-healthcheck.sh - withdraw BGP if local DNS is unhealthy
ANYCAST_IP="192.0.2.53"
CHECK_DOMAIN="healthcheck.example.com"

# 1. Check if DNS daemon answers locally
if ! dig @127.0.0.1 $CHECK_DOMAIN +short +time=1 +tries=1 > /dev/null; then
  echo "DNS local check failed - withdrawing route"
  birdc disable health_static  # BIRD will withdraw 192.0.2.0/24
  # For ExaBGP: echo "withdraw route 192.0.2.0/24 next-hop self" | nc exabgp 5000
  exit 1
fi

# 2. Check system load and packet loss
LOAD=$(cut -d' ' -f1 /proc/loadavg)
if (( $(echo "$LOAD > 32" | bc -l) )); then
  echo "Overloaded - prepending AS path 3x to shed load"
  birdc configure soft  # apply prepend filter
fi

birdc enable health_static
echo "Health OK"

Cache Isolation and the Thundering Herd

One operational hazard I have seen repeatedly is assuming anycast nodes share a cache. They do not. Each PoP maintains its own hot cache. When a popular record expires, dozens of resolvers hitting different PoPs will each fetch from the origin. This is not a major issue for DNS TTLs of 30-60 seconds, but for dynamic GSLB records with 10-second TTLs, the origin query rate multiplies by node count. Mitigate this by keeping authoritative data fully replicated to every PoP via zone transfers (AXFR/IXFR) or a distributed database, so no cache miss requires a cross-continent fetch.

When to Use Concast

Concast - where some nodes announce the prefix and others do not - is useful for capacity planning. For example, you might have 40 PoPs capable of serving DNS but only announce the anycast prefix from 25 of them during normal operations, leaving 15 as warm spares that can be activated via BGP if a region is under DDoS. This pairs well with the filtering and scrubbing approach outlined in DDoS Protection at the CDN Edge: Rate Limiting, Challenge Pages and Bot Detection, because you can shift anycast announcements away from nodes under volumetric attack without DNS clients noticing.

Diagnosing Anycast Pathologies: Flapping, Cold Caches and Load Imbalance

Anycast is not magic. It introduces failure modes that unicast does not have, and you need to be ready to debug them.

Route Flapping: If a PoP's BGP session is unstable, external networks will see the path appear and disappear, causing queries to oscillate between PoPs. This shows up as intermittent SERVFAIL or high jitter in DNS latency. The fix is BGP route damping on the upstream and aggressive BFD (Bidirectional Forwarding Detection) between your router and the host. In my experience, setting BFD to 300ms x 3 gives you sub-second failure detection without false positives on congested links.

Unequal Load Distribution: BGP does not load balance. A single Tier-1 ISP might send 30% of global queries to one PoP simply because that PoP is its primary peering point. I have seen a single node in Ashburn handle 4x the QPS of a node in São Paulo despite similar population coverage. Monitor QPS per PoP and use AS_PATH prepending to de-preference overloaded nodes. Prepending your AS two or three times makes that path less attractive and gently sheds traffic.

Stale State and Debugging: Because any anycast node can answer, traditional traceroute is misleading. A traceroute to 192.0.2.53 will only show the path to the nearest node, not which node actually handled a specific user's query. The solution is to embed node identity in DNS itself. Configure each Knot/NSD instance to answer id.server or hostname.bind in the CHAOS class, and log the PoP ID in your DNS analytics pipeline. That way, when a customer reports slow resolution from Tokyo, you can confirm which PoP served them without guessing based on BGP tables alone.

MTU and Fragmentation Gotchas

DNS over UDP historically used 512-byte payloads, but with DNSSEC and large TXT records, responses now frequently exceed 1500 bytes and fragment. Some anycast paths have lower MTUs due to tunneling. Ensure you set edns-udp-size 1232 as recommended by the DNS Flag Day, and test fragmented responses from each PoP. I have debugged cases where a PoP in Mumbai silently dropped fragmented DNSSEC responses, causing validation failures that only affected users routed there.

Integrating Anycast With GSLB, Failover Routing and Edge Security

Anycast DNS should not exist in isolation. It is the first hop in a longer routing chain that often includes Global Server Load Balancing and origin failover.

Consider a typical request flow: 1) Resolver queries anycast DNS and is routed to the nearest PoP. 2) That PoP's DNS daemon evaluates origin health, geography, and capacity, and returns an edge VIP. 3) The client connects to that VIP, which may itself be anycast or a unicast address tied to a specific region. This separation of concerns lets BGP handle proximity quickly while application logic handles intelligence.

For true resilience, your anycast health checks must be aware of upstream dependencies. Withdrawing a DNS announcement because an origin in the same facility is down is usually wrong - the DNS node can still answer with an alternate origin IP. Only withdraw when the DNS service itself cannot serve authoritatively. For origin health, let your GSLB logic return a different answer rather than removing the whole PoP from the internet. This layered approach is covered in more detail in CDN Failover and Origin Redundancy: Ensuring Availability During Outages, which I recommend reading after you have your anycast base stable.

Security is another integration point. Anycast inherently absorbs DDoS by dispersing attack traffic across all PoPs. A 500 Gbps DNS flood that would crush a single unicast node becomes ~10 Gbps per PoP across 50 nodes, which is absorbable with proper rate limiting. However, you must also protect against route hijacks. Always publish ROAs (Route Origin Authorizations) for your anycast prefixes in the RPKI, and require peers to perform RPKI validation. Without ROV (Route Origin Validation), a malicious AS could announce your anycast prefix and intercept queries.

Finally, remember that DNS prefetching and connection coalescing at the browser layer interact with your TTL choices. As explained in IETF RFC 4786 - Operation of Anycast Services, anycast nodes should be treated as equivalent instances, so keep TTLs low enough (30-60s) to allow for rapid re-steering during PoP maintenance, but not so low that you erode resolver cache hit rates and increase your authoritative QPS unnecessarily.

Frequently Asked Questions