Cache Invalidation: The Hardest Problem in CDN

"There are only two hard things in Computer Science: cache invalidation and naming things." Phil Karlton's famous observation resonates with every engineer who has deployed content through a CDN and then needed to update it immediately. Cache invalidation is the process of removing or replacing stale content from distributed caches so that users see the latest version. It is inherently difficult because CDNs distribute content across hundreds of edge servers worldwide, and ensuring every copy is updated simultaneously without disrupting active users requires careful strategy. This guide covers every invalidation method available, from brute-force purges to elegant cache-busting patterns that eliminate the problem entirely. For a comprehensive overview, see our guide on CDN Caching Strategies: Pull, Push, and Hybrid — FlyFish NWA.

Why Is Cache Invalidation So Difficult?

Caching works by storing copies of content closer to users, reducing origin load and latency. The fundamental tension is that the longer you cache content, the better your performance, but the harder it becomes to update that content when it changes. Several factors make invalidation particularly challenging in CDN environments:

• Geographic distribution — Content is cached at dozens to hundreds of Points of Presence (PoPs) worldwide. A purge must reach every one of them.
• Propagation delay — Even the fastest purge systems take 2-5 seconds to propagate globally. During that window, some users receive old content while others see the new version.
• Cache hierarchy depth — With L1, L2, and origin shield layers, content may be cached at three or more tiers, each requiring separate invalidation.
• Browser caches — CDN purges do not clear browser caches. If a user's browser has cached the old version, they continue seeing it until the browser TTL expires.
• Consistency requirements — Purging a CSS file without simultaneously purging the HTML that references it can cause visual breakage during the transition window.

What Are the Different Purge Methods?

CDN providers offer several purge mechanisms, each with different precision, speed, and cost characteristics. Choosing the right method depends on how much content needs invalidation and how quickly it must propagate.

Single URL Purge

The most precise method invalidates one specific URL. You provide the exact URL including query parameters, and the CDN removes that object from all edge caches.

# Cloudflare API: purge single URL
curl -X POST "https://api.cloudflare.com/client/v4/zones/{zone_id}/purge_cache" \
  -H "Authorization: Bearer {token}" \
  -d '{"files":["https://example.com/images/hero.webp"]}'

Single URL purges are fast (typically under 5 seconds globally) and precise, but they require knowing every URL that needs invalidation. For a site with thousands of pages, purging URLs one by one is impractical.

Tag-Based Purge (Surrogate Keys)

Tag-based purging is the most powerful invalidation strategy for complex sites. You attach custom tags to cached responses using headers like Surrogate-Key (Fastly) or Cache-Tag (Cloudflare), then purge all objects sharing a tag with a single API call.

# Origin response headers:
Surrogate-Key: product-123 category-electronics homepage-featured
Cache-Tag: product-123, category-electronics, homepage-featured

# Purge everything tagged "product-123":
curl -X POST "https://api.fastly.com/service/{id}/purge/product-123"

When product 123 changes, one purge request invalidates the product page, category listings, search results, and any page that displays that product. This approach scales to millions of objects because you reason about content relationships rather than individual URLs. Tag-based purging integrates naturally with cache key design patterns.

Wildcard Purge

Wildcard purges invalidate all URLs matching a pattern. For example, purging /images/* removes every cached object under the images path. Not all providers support wildcards, and those that do may have different pattern syntax and performance characteristics.

Purge All (Full Cache Flush)

A full cache flush removes every cached object across all edge servers. This is the nuclear option: effective but expensive. After a full flush, every request becomes a cache miss, creating a cache stampede as all edge servers simultaneously fetch from the origin.

# CloudFront: create invalidation for everything
aws cloudfront create-invalidation \
  --distribution-id EDFDVBD6EXAMPLE \
  --paths "/*"

Reserve full flushes for emergencies such as accidentally deploying sensitive data or major site restructures. For routine updates, use targeted purge methods.

What Is the Difference Between Soft Purge and Hard Purge?

Hard purge and soft purge represent two fundamentally different philosophies toward invalidation.

Hard Purge

A hard purge immediately removes the cached object. The next request for that resource results in a cache miss and a full origin fetch. This guarantees freshness but creates a brief performance penalty as edges refill their caches.

Soft Purge

A soft purge marks the object as stale without removing it. The cache continues serving the stale version while asynchronously revalidating with the origin in the background. This behavior mirrors stale-while-revalidate and provides several advantages:

• Zero user-facing latency increase — Users receive an immediate response (the stale version) while the cache updates.
• Origin protection — Revalidation happens gradually as requests arrive, rather than all at once.
• Graceful degradation — If the origin is temporarily down, users still get the stale version instead of an error.

Soft purge is the preferred default for most invalidation scenarios. Use hard purge only when stale content poses a security or compliance risk.

How Does Cache Busting Eliminate the Invalidation Problem?

Cache busting sidesteps invalidation entirely by changing the URL when content changes. If the URL is different, there is nothing to invalidate because the old URL is simply never requested again.

Content Hash in Filename

Modern build tools (webpack, Vite, esbuild) generate filenames that include a hash of the file contents:

# Before change:
/assets/app.3f8a2b.js
/assets/style.9c4d1e.css

# After change:
/assets/app.7e2f1a.js    (new hash = new URL)
/assets/style.9c4d1e.css (unchanged = same URL, served from cache)

Because the hash changes only when the content changes, you can set Cache-Control: public, max-age=31536000, immutable on these assets. They are cached for one year and never need purging. The HTML page that references them uses a short TTL or no-cache, so it always fetches the latest version with the new filename references.

This pattern works seamlessly with Cache-Control header strategies and is the recommended approach for all static assets.

Query String Versioning

An alternative approach appends a version query parameter:

/assets/app.js?v=3f8a2b
/assets/app.js?v=7e2f1a

However, query string versioning is less reliable. Some CDNs ignore query strings in cache keys by default, and some proxies do not cache URLs with query strings at all. Content-hashed filenames are universally supported and the preferred approach.

How Do Major CDN Providers Compare on Purge Features?

Tag-based purging is the most architecturally elegant approach, but it requires planning your tag taxonomy upfront and may require higher-tier CDN plans. If your provider does not support tags, combine single URL purges for critical pages with content-hashed filenames for static assets.

How Should You Integrate Purging into CI/CD Pipelines?

Manual purging does not scale. Integrate invalidation into your deployment pipeline so that every deployment automatically handles cache updates:

1. Build phase — Generate content-hashed filenames for all static assets. No purging needed for these.
2. Deploy phase — Upload new assets to the origin. Old filenames remain accessible for users still loading the previous version.
3. Invalidation phase — Purge HTML pages and API endpoints that reference the new asset filenames. Use tag-based purging if available, or batch URL purging.
4. Verification phase — Confirm purge propagation by checking Age and X-Cache headers from multiple edge locations.

# Example CI/CD invalidation step
deploy:
  - run: npm run build       # generates hashed filenames
  - run: aws s3 sync dist/ s3://origin-bucket/
  - run: aws cloudfront create-invalidation \
      --distribution-id $CDN_ID \
      --paths "/index.html" "/api/config"
  - run: ./scripts/verify-purge.sh

For TTL optimization, combine short HTML TTLs with long static asset TTLs so that only HTML pages need explicit purging. This reduces the number of purge operations per deployment from hundreds to a handful.

What Are Best Practices for Cache Invalidation?

After years of production experience across high-traffic CDN deployments, these practices consistently deliver reliable invalidation:

• Prefer cache busting over purging — Use content-hashed filenames for static assets. Purge only what you cannot cache-bust (HTML, API responses).
• Use soft purge when available — Serve stale content during revalidation rather than forcing cache misses.
• Implement surrogate keys early — Retrofitting tag-based purging onto an existing system is harder than designing it from the start.
• Never purge all in production unless necessary — A full flush creates a stampede that can overwhelm your origin. If you must, pre-warm the cache immediately after.
• Account for browser cache — CDN purges do not reach browsers. If you need users to see changes immediately, ensure HTML pages use short TTLs or no-cache so browsers check the CDN on every visit.
• Monitor after purging — Check your cache hit ratio after invalidation to confirm that caches refill properly and the hit rate recovers.
• Version your APIs — Use path-based API versioning (/v2/products) so that API changes do not require cache purging of existing endpoints.

Cache invalidation will always involve tradeoffs between freshness and performance. The best strategy combines multiple approaches: content-hashed filenames for static assets, short TTLs with stale-while-revalidate for HTML, tag-based purging for complex content relationships, and automated pipeline integration that removes the human error factor from the equation.