Cache Tier Promotion and Demotion: Managing Multi-Level Cache Hierarchies

Back in 2019, I watched a single viral video push our entire L1 edge tier to 98% eviction rate in under 12 minutes. The origin never flinched, but the regional shields melted because our promotion logic was too eager and our demotion logic didn't exist. That night taught me that cache hierarchy management isn't about having more tiers — it's about knowing exactly when an object deserves to move up, and when it should be quietly pushed down.

The Brutal Reality of Modern Cache Hierarchies

Most CDN diagrams show a clean pyramid: L1 at the edge, L2 in the metro, L3 regional shield, then origin. In production, it's far messier. Your L1 isn't one tier — it's two. You've got a tiny, blazing-fast RAM tier (often just 20-40GB per PoP) sitting in front of a much larger NVMe or SSD tier on the same box. Then you've got your metro PoPs, then your shield. Each jump adds 10-50ms, but more importantly, each jump changes your cost model completely. RAM hits are essentially free. Shield hits still cost you transit and compute. Origin hits cost you everything, including your on-call's sleep.

I've seen teams try to treat all cache tiers as a single logical cache. That's a mistake. Each tier has a different job. L1 RAM exists to absorb the hottest 1% of objects that drive 60% of requests. L2 SSD exists to hold the warm set — objects that are requested every few minutes, not every few seconds. The shield exists to collapse cold misses and protect the origin from thundering herds. If you don't respect those roles, you'll end up promoting a 4MB PDF that was requested twice into RAM, evicting 3,000 small API responses that were actually hot. Understanding how Cache Coherence Protocols at the CDN Edge: Keeping Distributed Caches in Sync work across these tiers is foundational, because a promoted object that's stale is worse than a demoted one.

# Example: ATS parent.config + record.config hierarchy definition
# This defines our 3-tier setup - edge RAM -> edge SSD -> shield
dest_domain=origin.example.com parent="shield-ams.example.com:8080;0.9"
# Tier sizing and assignment in records.config
CONFIG proxy.config.cache.ram_cache.size INT 32G
CONFIG proxy.config.cache.ram_cache.algorithm INT 1 # 1=LRU, 2=Clock
CONFIG proxy.config.cache.ram_cache.compress.enable INT 1
# Promotion threshold: only promote to RAM after 3 hits in 60s
CONFIG proxy.config.cache.promotion.hits INT 3
CONFIG proxy.config.cache.promotion.window INT 60

Promotion Triggers That Don't Kill Your Hit Ratio

The naive approach to cache promotion is simple: on a miss, fetch from the next tier and store it locally. Do that everywhere and you've just built a very expensive copy machine. Every tier ends up holding the same objects, your overall capacity drops by 70%, and your L1 churn goes through the roof. Real promotion needs to be conditional and, frankly, a bit skeptical. We don't trust a single hit. We didn't trust two hits either, after we measured it.

What we use now is a two-stage gate. First, we track hits in a lightweight sketch outside the cache itself — no need to promote to check if we should promote. We use a Count-Min Sketch with a 60-second sliding window, which is far cheaper than the approach described in Probabilistic Caching with Bloom Filters: Memory-Efficient Cache Lookups, but the principle is identical: don't pay the cache insertion cost until you're confident. An object needs 3 hits in 60 seconds to earn a promotion to L2 SSD, and 8 hits in 60 seconds to earn promotion to L1 RAM. And we add a size bias — objects over 1MB need 2x the hits, because they're eviction bullies. This cut our L1 write amplification by 44% overnight. We also tie promotion to origin cache-control semantics as defined by the HTTP Working Group Specifications; if an object comes back with `Cache-Control: s-maxage=10`, we don't promote it at all, no matter how hot it gets.

def should_promote(key, object_size, tier):
    # TinyLFU-inspired admission with size-aware thresholds
    hits = sketch.estimate(key)  # Count-Min Sketch, 60s window
    base_threshold = 8 if tier == "L1_RAM" else 3
    
    # Size penalty: large objects must prove more popularity
    if object_size > 1024 * 1024:
        base_threshold *= 2
    if object_size > 5 * 1024 * 1024:
        return False  # Never promote >5MB to RAM, goes to SSD only

    # Check for cache-control short TTL - see httpwg.org/specs/
    if ttl_remaining(key) < 15:
        return False
        
    return hits >= base_threshold

# Called on L2 hit - should we copy up to L1?
if should_promote(request.key, obj.size, "L1_RAM"):
    ram_cache.put(request.key, obj, ttl=obj.s_maxage)

Demotion Is Not Eviction: Graceful Degradation Under Pressure

Here's where most teams get it wrong — they think demotion is just eviction with a different name. It isn't. Eviction is what happens when you're out of space and you throw something away. Demotion is what you do *before* you're out of space to move something to a cheaper, slower tier where it can still serve hits. If you're only evicting, you're wasting perfectly good warm objects that could still absorb shield traffic. A good demotion path will actually increase your global hit ratio by 5-8% without adding a single disk.

We run a background demotion worker every 30 seconds that looks very different from LRU. It scores every object in L1 RAM on three signals: recency, frequency, and retrieval cost. That last one is critical. An object that took 800ms to fetch from origin and is 200KB is far more valuable to keep nearby than a 2KB API response that can be refetched in 15ms. I've found that cost-aware demotion beats pure LRU by a wide margin, especially during partial origin failures. You don't want to demote the one object that's currently shielding you from a struggling database. For predictive work, we're now experimenting with models similar to those in Machine Learning for Cache Warming: Predictive Prefetching That Actually Works to pre-decide which objects are likely to be needed again before we demote them. And we validate our TTL and freshness heuristics against guidance from web.dev Articles to make sure we're not demoting objects that are about to expire anyway.

The metrics below are from one of our European PoPs last month — 45 edge nodes, 3 shield nodes, handling ~2.8M RPS at peak. Notice how L1 RAM delivers the lowest latency but holds only a tiny fraction of the working set. That's intentional. You don't want it to hold more.

Implementing Promotion and Demotion Without Trashing Your Origin

Talk is cheap — let's wire this up. In a production CDN PoP, you're typically juggling three tiers: L1 is your hot in-memory cache (think 50-100GB per node on Varnish or Nginx with proxy_cache), L2 is your local NVMe/SSD tier (1-4TB), and L3 is your shield or origin. The mistake I see teams make is treating promotion as synchronous. Don't. If you block a client request while you copy a 25MB video chunk from L2 SSD to RAM, you've just added 8-12ms of p95 latency for no good reason.

Our rule of thumb after tuning this across 180+ edge nodes: promotion should be async and threshold-gated. We don't promote on first hit. We promote after 3 hits within a 5-minute sliding window, with a minimum object size filter — nothing over 20MB gets promoted to L1, period. That alone cut our L1 churn by 41% and lifted our edge hit ratio from 87% to 93.4%. For demotion, we use a lazy, sampled approach. Every 30 seconds, a background worker scans ~1% of L1 keys using an approximate LFU counter (we use a Count-Min Sketch, not exact counts) and demotes anything with a frequency score <2 that hasn't been touched in 10 minutes.

The key is decoupling invalidation from tiering. If you're purging by URL, you're going to have a bad time when you need to purge 50k variants of the same asset. That's where a proper tagging strategy saves you. We tie our promotion metadata to Cache Tag Design Patterns: Organizing Content for Granular Invalidation so when a tag is purged, it drops from all tiers atomically. And if you're running a multi-node shield layer, you'll want your tiering logic to be consistent-hash aware — otherwise node A's L1 promotion is useless when the next request lands on node B. Our write-up on Consistent Hashing for CDN Cache Distribution: A Practical Implementation Guide covers exactly how we keep that stable during PoP scaling.

// Async promotion worker - Go-style pseudocode
func shouldPromote(key string, stats *AccessStats) bool {
    // Don't promote large objects to L1 - you'll thrash RAM
    if stats.Size > 20*1024*1024 { // 20MB cap for L1
        return false
    }
    // TinyLFU-style admission: 3 hits in 5m window
    if stats.HitsInWindow(5*time.Minute) < 3 {
        return false
    }
    // Check L1 memory pressure - don't promote if >85% full
    if l1Cache.MemoryPressure() > 0.85 {
        // Instead, consider demoting a cold entry first
        go demoteColdEntries(1)
        return false
    }
    // Admission filter: is this more frequent than L1 victim?
    victimFreq := l1Cache.EstimateVictimFrequency()
    return stats.EstimatedFrequency() > victimFreq
}

func demoteLoop() {
    ticker := time.NewTicker(30 * time.Second)
    for range ticker.C {
        // Sample 1% of L1 keys, demote cold ones to L2
        candidates := l1Cache.SampleKeys(0.01)
        for _, key := range candidates {
            if l1Cache.IdleTime(key) > 10*time.Minute && 
               l1Cache.Frequency(key) < 2 {
                l1Cache.DemoteToL2(key) // non-blocking move to SSD
            }
        }
    }
}

Advanced Patterns: Taming Promotion Storms and Ghost Caches

Here's where it gets nasty. You push a hot new release — say, a 4K stream manifest — and suddenly 15,000 clients request it in 10 seconds. Without protection, every edge node will try to promote that object from L2 to L1 simultaneously. That's a promotion storm, and we've seen it spike SSD read IOPS by 300% and stall Nginx workers. The fix isn't complicated, but you've got to build it in: single-flight promotion with a coalescing window. The first request triggers the async promotion; subsequent requests for the same key within the next 500ms just attach to the same future and wait for L2. No duplicate reads, no thundering herd.

The second killer is what I call ghost cache pollution — objects that get promoted because they had a brief burst of traffic (hello, bot scrapers) and then never get hit again. They sit in L1 for 20 minutes, evicting actually valuable content. We solved this by adding a probationary period. Newly promoted objects are marked as "unverified" for 2 minutes. If they don't get at least one more hit in that window, they're the first to be demoted, regardless of LFU score. We also use Surrogate Keys for Targeted Cache Purging: Beyond URL-Based Invalidation to tag these probationary objects, so if we detect a scraper pattern (say, 500 distinct URLs with the same surrogate key in 60 seconds), we can nuke the entire promotion cohort in one purge without touching legitimate hot objects.

Finally, monitor the right things. Don't just watch hit ratio — watch promotion efficiency. That's (hits on promoted objects) / (total promotions). If that drops below 0.6, your thresholds are too loose and you're just churning SSD. We alert at 0.65. Also track demotion-to-repromotion rate; if more than 15% of demoted objects get re-promoted within 10 minutes, your demotion window is too aggressive. Get those two metrics on a dashboard and you'll catch 90% of tiering issues before they hit your origin.

Frequently Asked Questions