CDN Purge Strategies: Selective Invalidation, Tag-Based and Surrogate Keys

Cache invalidation is one of the few problems in edge delivery where theory and operational reality diverge sharply. On paper, setting a short TTL solves staleness. In production, it destroys your hit ratio and origin costs. I have managed purge pipelines for e-commerce catalogs with over 200,000 SKUs and media sites serving 50M+ requests per day, and the lesson is consistent: your purge strategy directly determines your availability during traffic spikes. A blunt Purge Everything might fix stale content in seconds, but it also guarantees an origin stampede. This article breaks down the progression from simple URL invalidation to a mature surrogate key architecture, with the operational details you need to implement it correctly without taking your origin down.

The Stale Cache Tax: Why Purge-All Is an Expensive Anti-Pattern

In my experience, the first purge implementation teams reach for is the wildcard purge — /* or a full service invalidation. It is easy to understand, supported by every provider, and seems reliable. The problem is that it invalidates the entire working set of your cache, both stale and fresh content, in one operation.

When you purge everything, every subsequent request until the cache rewarms is a cache miss. For a site with a 92% hit ratio, that means your origin load jumps by roughly 12x in the seconds following the purge. I have seen origins that handle 800 RPS comfortably collapse under 9,000 RPS after a global purge during a product launch. The edge did its job - it invalidated — but it created a self-inflicted DDoS.

This is why global purges should be treated as an emergency tool, not a workflow. It also has a propagation problem. A global purge is eventually consistent across a 300+ PoP network. Fastly, Cloudflare, Akamai, and CloudFront all acknowledge purges in under 150ms, but full global convergence can take 2-5 seconds, and up to 10-15 seconds for large objects. During that window, users get inconsistent responses depending on which PoP they hit.

If you rely on frequent global purges to keep content fresh, you are paying the stale cache tax twice: once in origin compute and egress, and again in higher p95 latency for your users. A better approach is to keep your TTLs long — I typically recommend 1 hour to 1 year for immutable assets and 1-5 minutes for semi-dynamic content paired with background revalidation — and invest in precise invalidation. The strategies below let you do exactly that. For a deeper look at how TTLs interact with invalidation, see CDN Caching Strategies: TTL, Cache-Control Headers and Invalidation Patterns.

How CDN Invalidation Actually Propagates

It helps to understand what happens when you issue a purge. You are not deleting files from disks globally. You are updating a metadata index that marks cached objects as invalid. The next request for that object either revalidates or fetches fresh from origin. Most providers use a two-phase system: a control plane message is fanned out to all PoPs via their internal pub/sub, and each edge node updates an in-memory index. The object itself may remain on SSD until evicted, but it will not be served as fresh. This is why a "purge" and a "soft purge" can have dramatically different performance characteristics.

URL-Based Purging: Surgical Precision for a Limited Toolset

URL purge, also called exact-match or single-file purging, is the most basic form of selective invalidation. You send the exact URL path, including query string if it is part of the cache key, and the CDN marks that single cache entry as stale.

This works perfectly for 1:1 assets. Purging /assets/app.3a4f1c.css after a deploy, or /images/hero-1920.jpg after an edit, is precise and has no collateral damage. I have found that for Jamstack-style deployments with content-hashed filenames, you rarely need to URL-purge static assets at all, but it remains essential for HTML documents.

Where it fails is with 1:many relationships. Consider a product update. Product ID 8472 appears on:

/products/8472
/category/shoes?sort=popular
/search?q=running+shoes
/api/products/8472.json
/sitemap-products.xml

To fully invalidate that product via URL purging, your backend must know every URL where that product appears. That mapping is fragile and explodes combinatorially once you add pagination, personalization parameters, and faceted navigation. I have seen teams attempt to maintain this URL graph in Redis, only to find they were making 400-1200 purge calls for a single price change, hitting provider rate limits immediately.

Most CDNs limit URL purges to 500-3000 URLs per API call and throttle you to a few thousand purges per minute. Exceed that and purges queue, delaying freshness by minutes. URL purging is best used as a complement to key-based purging, not the core strategy.

# URL purge - precise but operationally brittle at scale
# Vary and query string handling matters: include full cache key

curl -X POST "https://api.fastly.com/service/SU1Z.../purge" \
  -H "Fastly-Key: $FASTLY_API_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "purge_response": {
      "urls": [
        "https://www.example.com/products/8472",
        "https://www.example.com/api/products/8472.json"
      ]
    }
  }'

# CloudFront - requires invalidation path, billed after first 1,000/month
aws cloudfront create-invalidation \
  --distribution-id EDFDVBD6EXAMPLE \
  --paths "/products/8472" "/api/products/8472.json"

Surrogate Keys Under the Hood: Decoupling Cache Identity From the URL

Surrogate keys, also called cache tags or edge tags depending on your provider (Fastly calls them Surrogate-Keys, Cloudflare Cache-Tag, Akamai Cache Tag, Varnish xkey), solve the 1:many problem by decoupling what to purge from where it is cached.

Instead of addressing cached objects by their URL, you attach one or more logical tags to the response at origin. The CDN indexes the cached object under both its cache key (URL) and all of its tags. Purging a tag then invalidates every object that was tagged with it, regardless of URL.

For product 8472, your origin would respond like this:

HTTP/1.1 200 OK
Content-Type: text/html
Cache-Control: public, max-age=86400, stale-while-revalidate=60
Surrogate-Key: product-8472 category-shoes brand-nike collection-summer24
Surrogate-Control: max-age=86400
Vary: Accept-Encoding

Notice two headers. Surrogate-Control tells the edge how long to cache, independent of the browser's Cache-Control. This is critical. You can tell browsers max-age=0 while telling the edge to keep it for a day. More importantly, Surrogate-Key attaches metadata that never reaches the browser - the CDN strips it before delivery.

When the price of product 8472 changes, you issue a single purge: purge by key: product-8472. In one API call, you invalidate the product page, every category page it appeared on, the search results, and the API response. No URL graph to maintain.

I have implemented this pattern for a retailer where a single inventory update previously required ~300 URL purges. After moving to surrogate keys, the same update required 1 key purge. Their purge volume dropped by 94%, and they stopped hitting Akamai's rate limits during flash sales.

Implementation Constraints You Need to Plan For

Every provider has limits on keys that affect your taxonomy design:

First, cardinality per object. Fastly allows up to 1024 keys per object (but header size limited to 16KB), Cloudflare Enterprise allows up to 1024 cache-tags per response with a 16KB total header limit, Akamai allows 128 tags per object. If you add a key for every related entity, you will truncate.

Second, key format. Most providers restrict keys to alphanumeric characters, hyphens, and underscores, with a max length of 1024 characters combined. Spaces are not allowed. Stick to a normalized pattern like type-id.

Third, purge granularity. You purge by single keys, not boolean expressions. You can purge product-8472 OR category-shoes, but you cannot atomically purge "product-8472 AND category-shoes" in one call on most platforms. Some support bulk purge with AND/OR logic, but treat that as provider-specific.

Reference the HTTP Working Group Specifications for how Cache-Control and Surrogate-Control interact, and how Vary affects key generation. The surrogate headers themselves are non-standard but de facto.

Designing a Tag Taxonomy That Survives Catalog Churn

The hardest part of surrogate keys is not the purge call; it is designing a taxonomy that is both granular enough to be precise and stable enough not to create cache fragmentation. A poorly designed taxonomy is worse than no taxonomy.

I've found that a three-tier hierarchy works consistently well:

1. Entity Keys: The finest granularity. product-8472, article-59281, user-11492. Purge this when the entity itself changes. Every cacheable response that renders that entity gets this tag.

2. Collection Keys: Logical groupings. category-shoes, collection-summer24, author-jdoe. This lets you purge an entire category when you reorder products or update a landing page template. Be careful not to make this too broad - purging category-all is just a global purge in disguise.

3. Template/Layout Keys: Structural dependencies. template-product-card, header-nav, footer-promo. When you deploy a frontend change to the product card component, you purge template-product-card and every page containing that component refreshes. This is incredibly powerful for headless CMS setups.

A practical example from an origin application (Node.js/Express):

// Middleware that composes surrogate keys dynamically
function setSurrogateKeys(req, res, next) {
  const keys = new Set();
  
  // Entity keys - from data fetched for this response
  res.locals.products.forEach(p => keys.add(`product-${p.id}`));
  res.locals.products.forEach(p => keys.add(`brand-${p.brandSlug}`));
  
  // Collection keys - derived from request context
  if (res.locals.category) {
    keys.add(`category-${res.locals.category.slug}`);
  }
  
  // Template keys - what components rendered this page
  keys.add('template-product-grid');
  if (req.query.sort) keys.add('template-sorted-view');

  // Enforce provider limits: cap at 128 keys, 16KB header
  const surrogateKey = Array.from(keys).slice(0, 128).join(' ');
  res.set('Surrogate-Key', surrogateKey);
  res.set('Surrogate-Control', 'max-age=604800, stale-while-revalidate=86400');
  res.set('Cache-Control', 'public, max-age=60, stale-while-revalidate=300');
  next();
}

// On product update webhook:
async function onProductUpdate(productId) {
  // Purge product and its brand - covers PDP, PLP, search, API
  await cdn.purgeKeys([`product-${productId}`, `brand-${product.brandSlug}`]);
  
  // For price changes, also purge sorted views if price sort is cached
  // This is where collection keys save you from URL enumeration
}

One mistake I see repeatedly is tagging every response with a global key like site-all for "easy" global purges. Do not do this. It couples every purge to every object and defeats selective invalidation. If you need a true global purge, use the provider's dedicated purge_all endpoint which is optimized for that path. Keep your taxonomy pure.

Also, consider cardinality in reverse. A popular product that appears on 10,000 category and search pages will create a key with 10,000 object references. Purging that key is O(n) on the edge index. Providers handle this well, but Akamai and Fastly both document that high-cardinality keys (100k+ objects per key) can take longer to converge. If a single key maps to more than 50k URLs, consider sharding it, for example category-shoes-p1, category-shoes-p2 by pagination shard.

Soft Purge vs Hard Purge: Controlling Origin Load During Invalidation

The difference between a hard purge and a soft purge determines whether your origin experiences a thundering herd.

A hard purge (the default) marks the object as invalid immediately. The next request is a cache miss, goes to origin, and blocks until origin responds. If 800 edge nodes receive 1,000 requests for that object during revalidation, you get request coalescing at best, but still a significant origin spike.

A soft purge marks the object as stale but still servable. The CDN continues to serve the stale object while it fetches a fresh copy in the background. This is the edge equivalent of stale-while-revalidate but triggered explicitly by your purge.

In my experience, soft purge should be the default for any non-critical update. Price changes, description edits, CMS content updates - all soft purges. Hard purge is reserved for legal takedowns, security issues, or when serving stale content for even 5 seconds is unacceptable. The performance difference is stark.

Pair this with origin shielding to compound the benefit. A shield PoP acts as a mid-tier cache; even if 100 edge PoPs revalidate, only the shield makes a single origin request. For more on this architecture, see Origin Shield and Multi-Tier Caching: Reducing Origin Load at Scale and Stale-While-Revalidate: The CDN Pattern That Eliminates Cache Misses.

The web.dev Articles on stale-while-revalidate provide an excellent baseline for how this pattern translates to both browser and edge caches. The core insight applies directly: serve what you have while you fetch what you need.

Orchestrating Purges at Scale: Automation, Idempotency, and Eventual Consistency

A purge system is a distributed system, and it needs to be treated as one. Issuing a purge from your CMS webhook is easy. Making it reliable at 200 purges per minute across multiple data centers is where teams struggle.

First, make purges idempotent and retriable. CDN purge APIs are generally at-least-once. If your webhook fires twice, purging the same key twice is harmless but counts against your rate limit. I recommend a deduplication queue. When a product update event fires, push the surrogate key into a queue with a 5-second debounce. If product 8472 is updated 12 times in 5 seconds during a bulk import, you coalesce that into one purge, not 12. I have seen this single optimization cut purge volume by 60% during bulk operations.

Second, handle rate limits gracefully. Fastly allows ~1000 soft purges per second per service on enterprise contracts, but Cloudflare caps at 5000 cache-tag purges per second and Akamai at ~500 tag invalidations per minute depending on contract. Implement exponential backoff with jitter. Never fail open by ignoring a 429 - a missed purge means stale content served for the full TTL, which could be hours.

// Purge orchestrator with debounce and backoff
import { Queue } from 'bullmq';

const purgeQueue = new Queue('cdn-purge', { connection: redis });

// Coalesce rapid updates: debounce by surrogate key
async function enqueuePurge(keys) {
  const deduped = [...new Set(keys)];
  for (const key of deduped) {
    await purgeQueue.add('purge-key', { key }, {
      jobId: `purge:${key}`,          // deduplication ID
      delay: 5000,                    // coalesce bursts for 5s
      attempts: 5,
      backoff: { type: 'exponential', delay: 1000 }
    });
  }
}

// Worker - single key purge with soft-purge flag
purgeQueue.process(async job => {
  const { key } = job.data;
  const res = await fetch(`https://api.fastly.com/service/${SERVICE_ID}/purge/${key}`, {
    method: 'POST',
    headers: { 'Fastly-Key': FASTLY_TOKEN, 'Fastly-Soft-Purge': '1' }
  });
  if (res.status === 429) throw new Error('Rate limited - retry');
  if (!res.ok) throw new Error(`Purge failed ${res.status}`);
  return res.json();
});

Third, accept eventual consistency and design your UX around it. After you receive a 200 OK from a purge API, the content is not yet guaranteed fresh globally. Poll for convergence if you need to confirm. One pattern I use is to append a ?purgeCheck={timestamp} or use a Surrogate-Key audit endpoint to verify freshness before sending a "publish succeeded" callback to the editor. Otherwise, an editor will hit refresh and still see stale content for 3-5 seconds, assume the purge failed, and spam the purge button.

Fourth, monitor your hit ratio after purges. A healthy key-based purge strategy should barely register on your overall cache hit ratio. If you see hit ratio drop from 94% to 78% after a routine catalog update, your tags are too coarse. I track purge volume vs origin request volume on the same dashboard. Ideally, one purge invalidates many objects but triggers only one origin revalidation thanks to shielding and request coalescing. That relationship is also central to CDN Cache Hit Ratio Optimization: From 60% to 95% Hit Rate.

Finally, version your purge logic. Log every purge with key, actor, timestamp, and provider request ID. When stale content is reported, you need to answer: Was the purge sent? Was it accepted? Did origin return the new content on revalidation? Without that audit trail, you are guessing. Surrogate keys give you surgical precision, but only if the operational layer around them is equally precise.

Frequently Asked Questions