CDN Cache Validation Testing: Verifying Correct Caching Behavior Across Edges

Q: What's the difference between Age and Cache-Status?

Cache-Status (or X-Cache) tells you HIT or MISS at this moment. Age tells you how long that object has been in cache, in seconds. You need both. A HIT with Age: 0 is suspicious - it likely just got revalidated from origin. A HIT with Age: 5000 when your s-maxage is 3600 means you're serving stale. Always assert that 0 < Age < TTL on your second request.

Q: Can I validate cache behavior without access to every edge POP?

You can get 90% confidence with just 4-5 geographically diverse test nodes - we use clients in US East, EU West, AP Southeast, and South America. Use tools like Catchpoint, ThousandEyes, or even distributed GitHub Actions runners. You won't catch a single broken POP in a minor market, but you'll catch systemic key generation, shielding, and header propagation errors, which are 95% of the bugs we see in production.

A couple of years ago we pushed a one-line VCL change that was supposed to extend image TTLs from one hour to one day. It passed every check in our lab, but within an hour of production deploy, our origin traffic doubled and users in Singapore were getting 3-day-old price data. No alerts fired because the CDN was returning 200s. That’s the trap with cache validation — incorrect caching behavior rarely looks like an error, it looks like success until you check the headers. I’ve learned to distrust any deploy that hasn’t had an explicit CDN Integration Testing Framework Design: Validating Edge Behavior Programmatically style check for cache correctness.

Why "200 OK" Doesn't Mean Your Cache Is Correct

When folks new to CDN work run their first CDN cache test, they usually fetch a URL twice and look for a HIT on the second request. If they see X-Cache: HIT or age > 0, they mark it as passed. I did that myself back in 2014, and it burned me. The problem is that a HIT only tells you *something* was cached, not that the *right* thing was cached for the right duration, with the right key, and with the right revalidation rules. I've seen caches happily store a 500 error page for 24 hours because Cache-Control was missing, and caches that ignored Vary: Accept-Encoding and served brotli to clients that couldn't decode it. Both returned HITs.

Real cache correctness means verifying the full contract between origin, edge, and client as defined in the HTTP Working Group Specifications. That contract lives in headers: Cache-Control, Surrogate-Control, Expires, ETag, Last-Modified, Vary, and CDN-specific ones like CDN-Cache-Control or Surrogate-Key. A proper caching behavior test has to assert on all of them, not just the hit/miss flag. You also need to account for how different CDNs normalize headers — some lower-case everything, some strip stale-while-revalidate from the client response but keep it at the edge. If you don’t know your provider’s behavior, you’ll write tests that flake or miss real bugs.

We now treat cache validation as a state machine test. You start with a cold cache, prime it with a known response, then make a series of conditional and unconditional requests to verify transitions: fresh HIT, stale revalidation, bypass, and purge. Each transition should be checked against the RFC semantics, and you should run the same sequence from more than one POP if you can. That’s where a lot of teams realize they need End-to-End CDN Testing: Building Production-Like Test Environments instead of a single curl from their laptop, because edge behavior isn’t global until you’ve proved it is.

#!/bin/bash
# cold -> prime -> hit validation with header assertions
set -euo pipefail
URL="https://cdn.example.com/static/app.7d4a.js"
PURGE_TOKEN="your-purge-token"

# 1. Force cold state
curl -X PURGE -H "Authorization: $PURGE_TOKEN" "$URL" -s -o /dev/null

# 2. Prime - expect MISS, cacheable response
echo "=== Prime request (expect MISS) ==="
PRIME=$(curl -s -D - "$URL?primer=$(date +%s)" -o /tmp/body1)
echo "$PRIME" | grep -i "x-cache:.*MISS"
echo "$PRIME" | grep -i "cache-control:.*max-age=86400"
echo "$PRIME" | grep -i "etag:"

ETAG=$(echo "$PRIME" | grep -i "^etag:" | cut -d' ' -f2 | tr -d '\r')

# 3. Fresh hit - expect HIT and Age > 0
echo "=== Fresh hit (expect HIT) ==="
HIT=$(curl -s -D - "$URL?primer=$(date +%s)" -o /dev/null)
# Note: reuse same URL without primer change to actually hit
HIT=$(curl -s -D - "$URL" -o /dev/null)
echo "$HIT" | grep -i "x-cache:.*HIT"
echo "$HIT" | grep -i "age: [1-9]"

# 4. Conditional request - expect 304 if ETag matches
echo "=== Revalidation (expect 304) ==="
curl -s -D - -H "If-None-Match: $ETAG" "$URL" -o /dev/null | grep "304 Not Modified"

Designing a Repeatable Caching Behavior Test Across Edges

The script above is fine for one edge, but it doesn’t prove behavior *across* edges, which is where most cache bugs hide. In 2019 we had a configuration where US and EU POPs honored Surrogate-Control, but our APAC POPs fell back to Cache-Control because of a staged rollout that never finished. Our single-region tests were green for weeks while APAC customers saw 10-minute TTLs instead of 1-hour. Since then I don’t trust any cache validation that doesn’t run from at least three geographically dispersed points and compare results.

To make that repeatable, you need two things: cache-busted URLs and deterministic origin responses. I generate a unique path or query string per test run (like /__cache_test__/<uuid>/asset.js) and have the origin return fixed headers for that prefix via a test route. That way you don’t pollute real cache and you eliminate origin variance. The test harness then fans out requests to the same test URL through different POPs — either via the CDN’s test edge hostnames, via synthetic monitors like Catchpoint or ThousandEyes, or by using Workers/Lambda@Edge to request from inside each region. As web.dev Articles on caching best practices point out, you must also test with realistic request headers (Accept-Encoding, Accept-Language, Cookie) because Vary and key normalization bugs only show up when those differ.

Don’t forget negative cases. A solid CDN cache test suite should include: an uncacheable response (Cache-Control: private, no-store) that must always be MISS, a response with Vary: Authorization that must not be shared, and a 404/500 that you’ve explicitly said not to cache. We also add a purge test — prime, purge, then assert the next fetch is MISS and fetches a new ETag. If your framework can’t automate purge, you’ll never catch stale-on-purge bugs before customers do. For teams building this at scale, I usually point them to CDN Performance Testing Methodology: Load Generation, Measurement, and Analysis for the measurement plumbing, even though performance isn’t the goal here — the load generators and header loggers are reusable for correctness.

import requests, uuid, time

CDN_URL = "https://cdn.example.com/__cache_test__/{id}/data.json"
EDGES = ["https://us-east.cdn.example.com", "https://eu-west.cdn.example.com", "https://ap-southeast.cdn.example.com"]
ORIGIN_HEADERS = {"Cache-Control": "public, max-age=300, stale-while-revalidate=60", "ETag": '"abc123"'}

test_id = str(uuid.uuid4())
results = {}

for edge in EDGES:
    url = CDN_URL.format(id=test_id).replace("https://cdn.example.com", edge)
    # Prime
    r1 = requests.get(url, headers={"Pragma": "no-cache"})
    assert "MISS" in r1.headers.get("X-Cache", ""), f"Prime failed at {edge}: {r1.headers}"
    assert r1.headers.get("Cache-Control") == ORIGIN_HEADERS["Cache-Control"]
    
    time.sleep(0.5)
    # Hit - should be HIT with Age
    r2 = requests.get(url)
    assert "HIT" in r2.headers.get("X-Cache", ""), f"Hit failed at {edge}: {r2.headers}"
    assert int(r2.headers.get("Age", "0")) >= 1, "Age not incremented"
    assert r2.headers.get("ETag") == '"abc123"'
    
    # Conditional revalidation
    r3 = requests.get(url, headers={"If-None-Match": '"abc123"'})
    assert r3.status_code == 304, f"Expected 304 at {edge}, got {r3.status_code}"
    results[edge] = {"age": r2.headers.get("Age"), "cache": r2.headers.get("X-Cache")}

print(results)  # Compare across edges - all should match

How to Read Cache Validation Results Without Fooling Yourself

Headers lie if you don’t know how to read them. Age is the most misinterpreted. Age: 0 doesn’t always mean MISS — some CDNs reset Age on a revalidated HIT, others keep counting. And X-Cache: HIT, HIT vs. HIT, MISS means your request hit the edge but missed the shield layer. I’ve had junior engineers chase a “miss” that was actually a shield miss but an edge hit, which is perfectly correct. You need to map your provider’s header vocabulary before you write assertions. Document it. Put it in your test utils as constants, not string literals scattered through tests.

Another gotcha is clock skew and TTL truncation. If your origin says max-age=86400 but you consistently see Age max out at 3600, your CDN has a global max TTL or an edge TTL cap you didn’t know about. We caught one of those by logging Age every 10 seconds for an hour and plotting it — the curve flattened at 3600 exactly. Same with stale-while-revalidate: the CDN may serve stale and revalidate in background, so your client sees HIT and Age > max-age. That’s correct behavior, but your test will flag it as failed if you assert Age < max-age. Check the spec semantics for stale-while-revalidate before you set thresholds.

Below is the checklist we run in CI for every cache rule change. It’s not exhaustive, but if these four checks pass from three regions, we’ve caught about 90% of cache correctness regressions before they reach production. We store the actual header values in our test artifacts so we can diff against the last green run — that diff is often more useful than a simple pass/fail.

Validation Check Signal / Header to Assert Pass Threshold Common Failure Mode

Fresh HIT consistency

X-Cache + Age

HIT with Age 1-5s on 2nd request, identical ETag across 3 POPs

Validation Check	Signal / Header to Assert	Pass Threshold	Common Failure Mode
Fresh HIT consistency	X-Cache + Age	HIT with Age 1-5s on 2nd request, identical ETag across 3 POPs	Automating Validation: Building a Repeatable Edge Harness Reading headers manually is fine for a one-off debug, but it doesn't scale. If you're validating caching behavior properly, you need an automated harness that hits every edge you care about, with cache-aware logic, not just a looped curl. We've seen teams run "validation" from a single Jenkins agent in us-east-1 and call it global - that's how you miss a misconfigured POP in Singapore that’s serving MISS on 40% of your image traffic. Your harness needs to do three things reliably: generate unique-but-repeatable test URLs, force requests through specific edges or geographies, and assert on more than just the status code. Don't just check for `HIT`. Assert that `Age` increments on the second request, that `Cache-Control` matches your origin intent, and that `Age` never exceeds your `max-age`. A response with `Age: 86400` when your TTL is 300s isn't a HIT, it's a stale bug. We typically run 50 sequential requests per POP with a 500ms delay - enough to warm the cache and catch that first MISS -> HIT transition without tripping rate limits. #!/bin/bash # cdn-cache-validation.sh - Repeatable edge cache validation EDGE_POPS=("iad" "lhr" "nrt" "sin" "gru") TEST_PATH="/assets/test-object-$(date +%s).jpg" ORIGIN_TTL=300 for pop in "${EDGE_POPS[@]}"; do echo "=== Testing POP: $pop ===" # First request - expect MISS, warms the edge curl -s -D - -o /dev/null \ -H "X-Debug-POP: $pop" \ -H "Cache-Control: no-cache" \ "https://cdn.example.com${TEST_PATH}" \| grep -E "^(HTTP\|X-Cache\|Age\|Cache-Control\|Cache-Status):" sleep 1 # Second request - MUST be HIT, Age should be >0 and < TTL RESPONSE=$(curl -s -D - -o /dev/null "https://cdn.example.com${TEST_PATH}") CACHE_STATUS=$(echo "$RESPONSE" \| grep -i "Cache-Status:" \| cut -d' ' -f2) AGE=$(echo "$RESPONSE" \| grep -i "^Age:" \| awk '{print $2}' \| tr -d '\r') if [[ "$CACHE_STATUS" == "HIT" ]] && [[ "$AGE" -gt 0 ]] && [[ "$AGE" -lt "$ORIGIN_TTL" ]]; then echo "✓ PASS: $pop HIT with Age: $AGE" else echo "✗ FAIL: $pop expected HIT, got $CACHE_STATUS Age:$AGE" exit 1 fi done # For true distributed validation, pair this with # distributed traffic generation - see our guide on # Distributed Load Testing for CDN Edge: Generating Traffic from Multiple Geographies Advanced Patterns: When Edges Disagree and Caches Lie If your validation passes in Ashburn but fails in Tokyo, don't immediately blame replication lag. The most common culprit we see is inconsistent Vary handling. One edge correctly caches separate variants for `Accept-Encoding: gzip, br` while another collapses them because someone disabled `Vary` normalization at the shield. You'll see a HIT, but your users get garbled content. Your harness should explicitly test variants - send the same URL with different `Accept-Encoding` and `Accept-Language` headers and ensure your hit/miss pattern is predictable. If you get a HIT when you should get a MISS for a new variant, your cache key is too broad. Tiered caching and Origin Shield make this trickier. A MISS at the edge that’s a HIT at the shield will still look fast - maybe 45ms TTFB instead of 15ms for a pure edge HIT, but nowhere near the 250ms origin fetch. If you’re only asserting HIT vs. MISS, you’ll miss shield misconfigurations entirely. Look for the full chain: `Cache-Status: fwd=shield HIT, edge MISS` or whatever your provider exposes. We also use Chaos Testing at CDN Edge: Injecting Faults to Validate Resilience techniques here - intentionally taking a shield POP offline during validation to prove edges gracefully fall back to origin instead of serving 3-hour-old stale content. Finally, watch out for clock skew and soft purging. If you purge and immediately validate, you might get a HIT with an `Age: 2` because the purge was soft and the edge is serving stale-while-revalidate in the background. That’s not a failure, but it is a lie if your test expects a MISS. Always validate with a hard purge or better yet, a unique surrogate key per test run. And if validation is part of your release pipeline, link it directly to your CDN Failover Testing Playbook: Scripted Scenarios for Disaster Recovery Validation - a cache validation failure should block a deploy just as fast as a failover test failure. When you need to prove this at scale, don’t run it from one region; use Distributed Load Testing for CDN Edge: Generating Traffic from Multiple Geographies to generate real concurrent pressure and verify your 95%+ hit ratio holds under load, not just with two sequential curls. Frequently Asked Questions Should I expect 100% HIT ratio during validation? No, and you shouldn't want it. A healthy validation run should show exactly one MISS per edge/POP per unique URL, then 100% HITs for the next 49 requests within your TTL window. If you're seeing 100% HITs from the very first request, your test URLs aren't unique and you're reading a previous run's cache. If you're seeing 85-90% HITs after warmup, you've got a cache fragmentation problem - usually Vary or query string handling. How long should I wait after a purge before validating? For a hard purge, allow 5-10 seconds for global propagation, then validate. Most providers acknowledge purge in 150-300ms at the API level but take 2-8 seconds to evict from all 300+ edges. For soft purges or surrogate key purges, don't validate HIT/MISS at all - validate that the `ETag` or `Last-Modified` changed. We recommend using a brand new cache-busted path for automated tests instead of purging - it's deterministic and avoids the "did the purge propagate?" question entirely. What's the difference between Age and Cache-Status? `Cache-Status` (or `X-Cache`) tells you HIT or MISS at this moment. `Age` tells you how long that object has been in cache, in seconds. You need both. A HIT with `Age: 0` is suspicious - it likely just got revalidated from origin. A HIT with `Age: 5000` when your s-maxage is 3600 means you're serving stale. Always assert that 0 < Age < TTL on your second request. Can I validate cache behavior without access to every edge POP? You can get 90% confidence with just 4-5 geographically diverse test nodes - we use clients in US East, EU West, AP Southeast, and South America. Use tools like Catchpoint, ThousandEyes, or even distributed GitHub Actions runners. You won't catch a single broken POP in a minor market, but you'll catch systemic key generation, shielding, and header propagation errors, which are 95% of the bugs we see in production. External References MDN: HTTP Caching Guide AWS CloudFront Documentation RFC 9110: HTTP Semantics Related Articles CDN Integration Testing Framework Design: Validating Edge Behavior Programmatically End-to-End CDN Testing: Building Production-Like Test Environments CDN Performance Testing Methodology: Load Generation, Measurement, and Analysis Distributed Load Testing for CDN Edge: Generating Traffic from Multiple Geographies Chaos Testing at CDN Edge: Injecting Faults to Validate Resilience References & Standards: HTTP Working Group Specifications · web.dev Articles · IETF Datatracker

Automating Validation: Building a Repeatable Edge Harness

Reading headers manually is fine for a one-off debug, but it doesn't scale. If you're validating caching behavior properly, you need an automated harness that hits every edge you care about, with cache-aware logic, not just a looped curl. We've seen teams run "validation" from a single Jenkins agent in us-east-1 and call it global - that's how you miss a misconfigured POP in Singapore that’s serving MISS on 40% of your image traffic.

Your harness needs to do three things reliably: generate unique-but-repeatable test URLs, force requests through specific edges or geographies, and assert on more than just the status code. Don't just check for HIT. Assert that Age increments on the second request, that Cache-Control matches your origin intent, and that Age never exceeds your max-age. A response with Age: 86400 when your TTL is 300s isn't a HIT, it's a stale bug. We typically run 50 sequential requests per POP with a 500ms delay - enough to warm the cache and catch that first MISS -> HIT transition without tripping rate limits.

#!/bin/bash
# cdn-cache-validation.sh - Repeatable edge cache validation
EDGE_POPS=("iad" "lhr" "nrt" "sin" "gru")
TEST_PATH="/assets/test-object-$(date +%s).jpg"
ORIGIN_TTL=300

for pop in "${EDGE_POPS[@]}"; do
  echo "=== Testing POP: $pop ==="
  # First request - expect MISS, warms the edge
  curl -s -D - -o /dev/null \
    -H "X-Debug-POP: $pop" \
    -H "Cache-Control: no-cache" \
    "https://cdn.example.com${TEST_PATH}" | grep -E "^(HTTP|X-Cache|Age|Cache-Control|Cache-Status):"

  sleep 1

  # Second request - MUST be HIT, Age should be >0 and < TTL
  RESPONSE=$(curl -s -D - -o /dev/null "https://cdn.example.com${TEST_PATH}")
  CACHE_STATUS=$(echo "$RESPONSE" | grep -i "Cache-Status:" | cut -d' ' -f2)
  AGE=$(echo "$RESPONSE" | grep -i "^Age:" | awk '{print $2}' | tr -d '\r')

  if [[ "$CACHE_STATUS" == *"HIT"* ]] && [[ "$AGE" -gt 0 ]] && [[ "$AGE" -lt "$ORIGIN_TTL" ]]; then
    echo "✓ PASS: $pop HIT with Age: $AGE"
  else
    echo "✗ FAIL: $pop expected HIT, got $CACHE_STATUS Age:$AGE"
    exit 1
  fi
done

# For true distributed validation, pair this with
# distributed traffic generation - see our guide on
# Distributed Load Testing for CDN Edge: Generating Traffic from Multiple Geographies

Advanced Patterns: When Edges Disagree and Caches Lie

If your validation passes in Ashburn but fails in Tokyo, don't immediately blame replication lag. The most common culprit we see is inconsistent Vary handling. One edge correctly caches separate variants for Accept-Encoding: gzip, br while another collapses them because someone disabled Vary normalization at the shield. You'll see a HIT, but your users get garbled content. Your harness should explicitly test variants - send the same URL with different Accept-Encoding and Accept-Language headers and ensure your hit/miss pattern is predictable. If you get a HIT when you should get a MISS for a new variant, your cache key is too broad.

Tiered caching and Origin Shield make this trickier. A MISS at the edge that’s a HIT at the shield will still look fast - maybe 45ms TTFB instead of 15ms for a pure edge HIT, but nowhere near the 250ms origin fetch. If you’re only asserting HIT vs. MISS, you’ll miss shield misconfigurations entirely. Look for the full chain: Cache-Status: fwd=shield HIT, edge MISS or whatever your provider exposes. We also use Chaos Testing at CDN Edge: Injecting Faults to Validate Resilience techniques here - intentionally taking a shield POP offline during validation to prove edges gracefully fall back to origin instead of serving 3-hour-old stale content.

Finally, watch out for clock skew and soft purging. If you purge and immediately validate, you might get a HIT with an Age: 2 because the purge was soft and the edge is serving stale-while-revalidate in the background. That’s not a failure, but it is a lie if your test expects a MISS. Always validate with a hard purge or better yet, a unique surrogate key per test run. And if validation is part of your release pipeline, link it directly to your CDN Failover Testing Playbook: Scripted Scenarios for Disaster Recovery Validation - a cache validation failure should block a deploy just as fast as a failover test failure. When you need to prove this at scale, don’t run it from one region; use Distributed Load Testing for CDN Edge: Generating Traffic from Multiple Geographies to generate real concurrent pressure and verify your 95%+ hit ratio holds under load, not just with two sequential curls.

Frequently Asked Questions

Should I expect 100% HIT ratio during validation?

No, and you shouldn't want it. A healthy validation run should show exactly one MISS per edge/POP per unique URL, then 100% HITs for the next 49 requests within your TTL window. If you're seeing 100% HITs from the very first request, your test URLs aren't unique and you're reading a previous run's cache. If you're seeing 85-90% HITs after warmup, you've got a cache fragmentation problem - usually Vary or query string handling.

How long should I wait after a purge before validating?

For a hard purge, allow 5-10 seconds for global propagation, then validate. Most providers acknowledge purge in 150-300ms at the API level but take 2-8 seconds to evict from all 300+ edges. For soft purges or surrogate key purges, don't validate HIT/MISS at all - validate that the ETag or Last-Modified changed. We recommend using a brand new cache-busted path for automated tests instead of purging - it's deterministic and avoids the "did the purge propagate?" question entirely.

What's the difference between Age and Cache-Status?

Cache-Status (or X-Cache) tells you HIT or MISS at this moment. Age tells you how long that object has been in cache, in seconds. You need both. A HIT with Age: 0 is suspicious - it likely just got revalidated from origin. A HIT with Age: 5000 when your s-maxage is 3600 means you're serving stale. Always assert that 0 < Age < TTL on your second request.

Can I validate cache behavior without access to every edge POP?