CDN Regional Edge Cache Design: Balancing Latency and Cache Efficiency

Edge PoPs deliver sub-20ms latency but suffer from cache fragmentation. Regional edge caches fix that by aggregating demand — if you size, place, and tune them correctly. Here’s how to design the middle tier without sacrificing speed.

A well-designed CDN looks simple from the outside: users hit the nearest Point of Presence (PoP), get a cached response, and move on. Inside, the challenge is cache efficiency. With 300+ edge PoPs each seeing a thin slice of the global working set, the same object is fetched from origin dozens of times. Origin egress spikes, hit ratio stalls at 85-90%, and long-tail content never stays warm.

The regional edge cache — sometimes called mid-tier, shield, or L2 cache — solves this by inserting a larger, highly available aggregation layer between the edge and origin. Done right, it lifts overall hit ratio to 96-98%, cuts origin requests by 60-80%, and actually improves average Time to First Byte (TTFB). Done wrong, it adds latency, complexity, and a single point of failure. This guide covers the architecture, tradeoffs, and operational patterns we use to balance both goals.

If you’re new to tiered caching, start with our CDN architecture overview before diving into regional design.

1. Why Edge-Only Architectures Fail at Scale

In a flat architecture, every edge PoP fetches directly from origin on a miss. For a site with 500K unique cacheable URLs and 400 PoPs, the math is brutal. Even with consistent hashing, each PoP independently caches what its local users request. A video segment popular in Berlin may be hot in FRA and AMS but cold in 390 other PoPs. When traffic shifts, those cold PoPs all miss at once — a thundering herd.

Three problems compound:

• Fragmentation: Cache capacity is sharded across hundreds of small SSD pools. The effective working set is far smaller than the sum of disks because duplicates dominate.
• Origin pressure: A 92% edge hit ratio sounds good, but at 100K RPS that’s still 8,000 origin fetches per second. For dynamic or personalized content, it’s worse.
• Poor tail performance: Long-tail objects (product images, API responses, HLS segments beyond the first few) have eviction rates of minutes at the edge, so they are almost always misses.

Adding a regional tier consolidates those misses. Instead of 400 PoPs calling origin, 400 PoPs call 8-12 regional caches, which then coalesce requests to origin. This is known as request collapsing or origin shielding. The regional cache holds an order of magnitude more objects for longer, serving as the system’s memory.

2. Anatomy of a Regional Edge Tier

A regional edge is not just a bigger PoP. It differs in hardware, software, placement, and consistency model.

Key design choices:

• Storage: Regional nodes use NVMe + HDD hybrid tiers with 10-100x the capacity of an edge. Use large keys_zone and tiered eviction (LRU + LFU) to retain long-tail objects.
• Compute: Fewer but denser machines with high network throughput (100-200 Gbps) to handle fan-in from 30-50 edge PoPs each.
• Consistency: Regional caches are authoritative for revalidation. Edge PoPs send If-None-Match to regional, not origin, reducing 304 churn.
• Routing: Edges are statically mapped to a primary and secondary regional. Avoid dynamic nearest-regional selection — it fragments the cache again. Anycast can work if the regional set is small and well-connected.

Security and transport matter here too. Terminate TLS at the edge, but maintain persistent, optimized connections from edge to regional and regional to origin. For protocol choices between tiers, see our guide to HTTP/2, HTTP/3 and QUIC, and ensure your SSL/TLS configuration supports session reuse across tiers to avoid handshake overhead.

3. The Latency vs. Hit Ratio Tradeoff

The central tension is simple: every extra hop adds latency on a miss, but dramatically reduces how often you miss. The goal is to minimize weighted latency, not just best-case latency.

Consider a request path with these typical RTTs: User→Edge 15ms, Edge→Regional 25ms, Regional→Origin 60ms. A hit at edge costs ~15ms. A hit at regional costs ~40ms. A miss to origin via regional costs ~100ms, versus ~75ms direct to origin. The regional path is slower when it misses, but misses far less often.

Weighted average TTFB tells the story: With 90% edge hit ratio and 75ms origin miss, average = 0.9×15 + 0.1×75 = 21ms. With regional (90% edge hit, 70% regional hit on the remaining 10%, 100ms full miss), average = 0.9×15 + 0.07×40 + 0.03×100 = 19.3ms — faster overall despite the extra hop.

To keep that math favorable, you must optimize two things: regional hit ratio (via larger cache and smarter keys) and regional latency (via placement and persistent connections). Track this with the metrics in our CDN performance metrics guide — especially P50/P95 TTFB segmented by cache status (HIT, HIT-Regional, MISS).

4. Design Patterns: Sizing, Placement & Routing

Placement strategy

Place regional edges near your origin and near major traffic basins, not equidistant. For a US-centric origin in Virginia, ideal regions are US-East (Ashburn), US-West (Oregon), EU-West (Dublin), and APAC-Southeast (Singapore). Each edge PoP maps to the nearest regional with a failover to the next-nearest. This keeps Edge→Regional RTT under 40ms for 95% of users while keeping Regional→Origin RTT low for revalidation.

Avoid placing a regional in every edge city — you’ll recreate fragmentation. And avoid a single global regional — you’ll add 100ms+ for distant edges. The sweet spot is 1 regional per 30-50 edge PoPs.

Sizing for working set

Size regional storage to hold 1.5–2× your 7-day cacheable working set. Estimate this from origin logs: sum of unique URL + Vary combinations with Cache-Control: public, max-age>0 over 7 days, multiplied by average object size. For many media sites, that’s 10–30 TB. Provision 20% headroom for burst and 2× for replication if you run active-active pairs.

Memory matters more than disk at the regional tier. Allocate at least 64–128 GB for keys and metadata — a 20 TB cache with 250 KB average object size holds 80M keys.

Cache key normalization and TTL layering

The biggest efficiency win is key normalization. Edges may vary on Accept-Language or User-Agent, but the regional should collapse where safe. Use tiered TTLs: short at edge, long at regional.

# Regional edge - nginx example: large, long-lived cache
proxy_cache_path /data/regional levels=1:2 keys_zone=REGIONAL:256m
                 inactive=7d max_size=30t use_temp_path=off
                 manager_files=500 manager_sleep=50ms;

# Edge PoP - smaller, shorter TTL, collapse to regional
proxy_cache_path /data/edge levels=1:2 keys_zone=EDGE:32m
                 inactive=6h max_size=400g;

# Normalized key at regional: strip edge-specific vary
map $http_accept_encoding $cache_key_encoding {
    "~*br"  "br";
    "~*gzip" "gzip";
    default "";
}
proxy_cache_key "$scheme$host$request_uri|$cache_key_encoding";

# Tiered TTLs + stale serving
proxy_cache_valid 200 302 1h;           # edge: 1 hour
proxy_cache_valid 404 1m;
proxy_cache_use_stale error timeout updating
                      http_500 http_502 http_503 http_504;
proxy_cache_background_update on;
proxy_cache_lock on;                   # collapsed forwarding
add_header X-Cache-Status $upstream_cache_status always;

# Origin response headers for tiered behavior
HTTP/1.1 200 OK
Cache-Control: public, max-age=60, s-maxage=86400
              stale-while-revalidate=300, stale-if-error=86400
CDN-Cache-Control: max-age=3600  # edge honors this if present
Surrogate-Key: product/12345 collection/shoes
Vary: Accept-Encoding

This pattern lets the edge revalidate every hour but the regional serves stale for up to a day while revalidating in background, so users never wait on a synchronous origin fetch. Purge by Surrogate-Key propagates regional→edge in seconds.

5. Measuring and Tuning for Efficiency

You can’t balance what you don’t measure. Segment these metrics by PoP, region, and cache status:

• Hit ratio by tier: Edge HIT %, Regional HIT %, Origin MISS %. Alert if regional hit ratio drops below 85% on the miss stream (i.e., 8.5% of total).
• Weighted TTFB: p50_TTFB = hit_edge * TTFB_edge + hit_regional * TTFB_regional + miss * TTFB_origin. Optimize for P95, not just P50.
• Origin shielding factor: requests_edge_miss / requests_origin. Target 5–10× reduction.
• Byte hit ratio: More important than request hit ratio for egress cost. A 90% request hit ratio can be 70% byte hit ratio if large objects miss.
• Revalidation ratio: 304s / 200s from origin. High 304 rate means TTLs are too short; increase s-maxage.

Tune iteratively: First, increase regional inactive and max_size until byte hit ratio plateaus. Second, normalize keys — audit Vary headers and query string ordering. Third, enable tiered stale-while-revalidate to hide revalidation latency. Finally, review placement — if an edge’s Regional RTT exceeds 60ms, remap it to a closer regional.

For a full observability checklist, pair this with your performance metrics dashboard and synthetic tests from each major geography.

Regional edge design is not set-and-forget. Traffic shifts, catalogs grow, and origins move. Re-evaluate working set size quarterly and failover mapping after any PoP expansion. When done well, the regional tier becomes invisible — users see lower latency, origins see less load, and your cache efficiency finally scales with your footprint.