Machine Learning for Cache Warming: Predictive Prefetching That Actually Works

Last Black Friday we watched our edge hit ratio collapse from 96% to 71% in under four minutes. Not because origin was slow, but because our "smart" prefetch warmed the wrong 40,000 objects. I've seen this pattern too many times in ten years building CDN caching layers: teams deploy naive prefetching, spike their origin bill, and then turn the whole thing off. Predictive caching only works when you stop guessing and start modeling actual user navigation.

Why Rule-Based Warming Falls Apart at Scale

For years, most of us relied on simple heuristics for cache warming — prefetch the top 100 URLs from yesterday's logs, warm everything linked from the homepage, or follow a blind `Link: ; rel=prefetch` header. They feel safe because they're deterministic, but they don't understand intent. A user who lands on `/product/18371` from a Google ad behaves nothing like a user who arrives from your internal search. Rule-based systems treat them identically, so you end up pushing gigabytes of assets to edges that'll never be requested before eviction. We learned this the hard way on a media site where warming "related articles" actually doubled our cold writes without moving hit ratio more than 0.8%.

The math gets ugly when you factor in distribution. If you're running Consistent Hashing for CDN Cache Distribution: A Practical Implementation Guide correctly, each edge PoP only owns a slice of the keyspace. A centralized warmer that doesn't know about your hash ring will happily warm `edge-ord-01` with objects that should live on `edge-ord-03`, creating duplicate cached copies and still missing on the next request. And if you haven't thought through Cache Coherence Protocols at the CDN Edge: Keeping Distributed Caches in Sync, a stale prefetch can be worse than no prefetch — you're serving a pre-warmed but invalid version while the revalidation storm hits origin anyway.

What you really need is a model that predicts *which* object will be requested *where* and *when*. That means moving from static lists to sequence prediction. Instead of warming what was popular, we started warming what is likely to be requested next given the last three requests, device class, and time-of-day. The shift from popularity-based to predictive caching cut our wasted bytes by a factor of four on the same traffic.

# Feature extraction for ML cache warming - per-request sequence builder
import pandas as pd
from collections import deque

def build_prefetch_features(request_log, window=3):
    features = []
    # request_log sorted by user_id, timestamp
    for user_id, group in request_log.groupby('user_id'):
        history = deque(maxlen=window)
        for _, row in group.sort_values('timestamp').iterrows():
            if len(history) == window:
                # Don't warm blindly - predict next path probability
                feat = {
                    'prev_paths': '|'.join([h['path'] for h in history]),
                    'time_delta_avg': sum(history[i+1]['ts'] - history[i]['ts'] 
                                         for i in range(window-1)) / (window-1),
                    'device_class': row['device_class'],
                    'edge_pop': row['pop'],
                    'hour': row['timestamp'].hour,
                    'label': row['path']  # next path to predict
                }
                features.append(feat)
            history.append({'path': row['path'], 'ts': row['timestamp']})
    
    df = pd.DataFrame(features)
    # Filter low-confidence labels - critical for prefetch optimization
    return df[df.groupby('prev_paths')['label'].transform('count') > 50]

Designing a Prefetch Model That Respects Origin Budgets

I don't recommend throwing a giant transformer at your CDN logs. We tried it. Inference latency was 40ms per decision, which is an eternity at the edge, and the model wanted to prefetch half the catalog. For machine learning CDN workloads, you want something you can run inside the request path without blocking. Our current stack uses gradient-boosted trees for most properties and a small 2-layer LSTM only for high-value video manifest prefetching. Both run as sidecar microservices that answer in under 2ms, or as WASM compiled directly into ATS/Varnish where we can afford it.

The trick isn't the algorithm — it's your training objective. Most teams optimize for recall: "did we prefetch the next URL?" That's how you bankrupt your origin shield. You must optimize for precision under a cost constraint. We frame it as a binary classification for each candidate URL: will this object be requested at this PoP within the next TTL window? And we penalize false positives 10x more than false negatives because a wasted prefetch costs origin egress, cache churn, and TLS. For inspiration on keeping those lookups cheap, we've borrowed ideas from Probabilistic Caching with Bloom Filters: Memory-Efficient Cache Lookups to gate predictions — we only run the model if a bloom filter shows the candidate has been seen recently at that edge.

You also have to be ruthless about cache-control semantics. Your model should never prefetch an object that isn't cacheable. We parse `Cache-Control`, `CDN-Cache-Control`, and `Vary` according to the HTTP Working Group Specifications before we even consider warming, and we check `stale-while-revalidate` windows described in web.dev Articles to avoid warming content the browser can tolerate stale. Here's the kind of edge logic we now run before firing any predictive prefetch — it looks simple, but it stopped 23% of our wasted origin fetches overnight.

// Edge prefetch gate - runs in ATS plugin / Varnish VCL context
func shouldPrefetch(candidate string, score float64, resp *http.Response) bool {
    if score < 0.82 { // tuned per PoP; 0.82 gave best F0.5 on our traffic
        return false
    }
    cc := parseCacheControl(resp.Header.Get("Cache-Control"))
    if cc.NoStore || cc.NoCache || cc.MaxAge < 30 {
        return false // don't warm uncacheable or ultra-short-lived objects
    }
    // Respect Vary: high cardinality Vary headers kill hit ratio
    if hasHighCardinalityVary(resp.Header.Get("Vary")) {
        return false
    }
    // Origin shield check - don't prefetch if origin is already hot
    if originLoadEWMA() > 0.85 {
        return score > 0.95 // only ultra-confident prefetches when shielded
    }
    return true
}

What We Measure Before We Trust Any Predictive System

If you let a data science team ship a prefetch model with only a hit ratio chart, you're going to have a bad quarter. Hit ratio alone lies. I've seen models that pushed edge hit ratio from 93% to 96% while tripling origin egress because they prefetched four objects for every one that was actually used. You have to track a basket of metrics together, and you need to track them per PoP, not globally. A model that works beautifully in `iad` can be pure noise in `sin` where traffic patterns and catalog subsets are completely different.

The numbers that finally made our ML cache warming trustworthy were waste ratio and byte precision. Waste ratio tells you how many prefetched bytes were evicted without ever being served, while byte precision tells you what percentage of prefetched bytes actually resulted in a hit. We also watch p95 origin latency during warming windows — a good prefetch optimization should flatten your tail latency without creating a new spike when predictions are wrong. Until you can see all of these on one dashboard, don't enable predictive caching in production.

We ran a controlled 14-day A/B on a 12 Tbps e-commerce property to compare approaches head-to-head. Same hash ring, same TTLs, same shield topology. Only the warming logic changed. The results killed our appetite for heuristic warming for good.

Putting Predictive Prefetch Into Production Without Melting Your Origin

We've seen too many teams train a decent model offline, then point it directly at the origin and wonder why their database falls over at 9am. A predictive warmer isn't just a model — it's a budget-aware scheduler sitting between your prediction service and your edge. The pattern that actually holds up in production is an async scoring loop paired with a token-bucket throttle at the edge. Your model scores URLs every 60-90 seconds, pushes anything above your confidence threshold into a prefetch queue (we use Redis Streams), and edge workers drain that queue only if they have origin budget left.

The trick is not to chase 99% recall. We'd much rather have 87-92% precision on the top 15% of predicted URLs than try to prefetch everything. Our current production setup targets a 4-5% origin overhead budget — so if your origin comfortably handles 2,000 RPS, you allocate 80-100 RPS for warming. Anything that would exceed that gets deferred or dropped. Feature-wise, don't overcomplicate it at first. Request recency, inter-arrival time variance, time-of-day cohorts, and referrer clusters will get you 80% of the lift. You'll want to log the full feature vector with your Cache Tag Design Patterns: Organizing Content for Granular Invalidation so you can later correlate prefetch accuracy with content groups instead of single URLs. That grouping is what lets you invalidate and re-warm intelligently later.

// edge-worker/prefetch-gate.js - Runs on a 5s tick, not per-request
import { getPrefetchQueue, scoreThreshold } from './scoring';

const ORIGIN_BUDGET_RPS = 80; // 4% of 2000 RPS origin capacity
const tokenBucket = new TokenBucket(ORIGIN_BUDGET_RPS, 200);

export async function onSchedule(event) {
  if (!tokenBucket.tryConsume(1)) return; // hard stop, no queuing

  const candidates = await getPrefetchQueue(25); // batch pull
  for (const item of candidates) {
    if (item.score < 0.82 || item.confidence < 0.75) continue;
    if (await isAlreadyHot(item.url)) continue; // check edge LRU + 2-min dedupe

    // Use low-priority fetch that won't contend with user traffic
    await fetch(item.url, {
      cf: { cacheTtl: 3600, cacheEverything: true },
      headers: { 'X-Prefetch': 'ml-warming', 'X-Priority': 'low' }
    });
    tokenBucket.consume(1);
    if (tokenBucket.empty()) break;
  }
}

async function isAlreadyHot(url) {
  const cached = await caches.default.match(url, { ignoreMethod: true });
  return cached && cached.headers.get('Age') < 120;
}

When Good Models Go Bad: Throttling, Feedback Loops, and Debugging at the Edge

Once you're live, the first failure mode you'll hit isn't model accuracy — it's a feedback loop. Your warmer prefetches URL A, which creates a cache hit, which your model then logs as "popular," which makes it prefetch URL A again with even higher confidence. Within 48 hours your hit ratio looks amazing and your origin is still getting hammered for the long tail you actually needed. You have to explicitly label traffic. We tag every warming request with X-Prefetch: ml-warming and exclude it from both training data and your hit-ratio SLI. If you don't, you're just teaching the model to memorize its own outputs. We also hard-separate that traffic in our analytics so we don't accidentally apply Surrogate Keys for Targeted Cache Purging: Beyond URL-Based Invalidation purges to warmed content and then immediately re-warm it in a tight loop.

The second gotcha is thundering herds on grouped content. Say you have a product collection with 1,200 variants sharing the same surrogate key. Your model correctly predicts a 3pm spike for the collection, but then fires 1,200 individual prefetches in 10 seconds. That's where coupling your prefetch keys to your invalidation strategy saves you. Instead of warming each URL, warm the top 20-30 representative URLs per tag and rely on stale-while-revalidate for the rest. And know when *not* to warm at all. We maintain a denylist driven by our Cache Bypass Patterns: When and How to Skip the CDN Cache rules — personalized carts, authenticated API responses, and anything with a Cache-Control: private or Vary: Cookie should never enter the prefetch queue, no matter how high it scores. Prefetching it just pollutes the edge and wastes budget.

If your precision suddenly drops from 88% to under 60%, don't retrain immediately. Check three things first: 1) Cold start after a purge — did marketing just flush collection:shoes and now your feature store is stale for 15 minutes? 2) Origin latency — if p95 jumps from 45ms to 300ms, your workers will timeout and your "isAlreadyHot" check will return false negatives. 3) Seasonality drift — Monday's model hates Saturday's traffic. We run two models (weekday/weekend) and shadow-score both, flipping via a simple cron. A proper setup should lift your edge hit ratio from ~82% to 93-95% within two weeks while keeping origin 5xx flat. If your hit ratio climbs but your origin CPU does too, your throttle is leaking. If your origin is quiet but hit ratio is flat, your threshold is too conservative. Tune the threshold, not the model.

Frequently Asked Questions