CDN Proof of Concept Planning: Designing a PoC That Answers Real Questions

About six years ago I watched a $2M CDN deal drag on for nine months because the proof of concept never proved anything. We'd set up two vendors side-by-side, pushed 5% of traffic through each, and stared at dashboards for weeks without deciding anything. Nobody had defined what "success" actually looked like, so every result was arguable. If you're planning a CDN PoC now, you can't afford that kind of ambiguity.

Why Most CDN PoCs Answer Questions Nobody Asked

I've reviewed probably 40 different CDN proof of concept plans over the last decade, and the same mistake shows up in most of them: they test whether the CDN works, not whether it works for you. A generic PoC that just CNAMEs a static asset bucket and measures cache hits doesn't tell you how the platform will handle your authenticated API calls, your messy origin failover logic, or that legacy image transformation service nobody wants to touch. Vendors love these simple PoCs because they're impossible to fail.

A useful PoC starts with the problems you're actually trying to solve. Maybe your origin's getting hammered during flash sales, or your video startup time is killing you in Southeast Asia, or your current provider's purge times are breaking your content workflow. Your CDN RFP Template: Writing Requirements That Get Meaningful Vendor Responses should have already forced you to articulate those pain points. The PoC's job is to validate the claims vendors made in response to them. If you haven't nailed down those requirements, you'll end up comparing pretty graphs instead of business outcomes. I always tell teams to write the decision memo before the PoC starts - if you can't say "we'll choose vendor A if X is under 200ms and Y is above 95%", you're not ready to test.

That means getting uncomfortable with scope. A real PoC design should include your worst traffic, not your easiest. Don't just test your marketing site; test the checkout API that can't be cached, the personalized homepage that needs EdgeWorkers or equivalent, and the origin that occasionally returns 500s. Document your success criteria in a way anyone can run without interpretation:

# poc-success-criteria.yml - define pass/fail BEFORE traffic flows
# Based on evaluation planning workshop output
poc_objectives:
  primary_question: "Can Vendor X cut p95 TTFB by 30% for dynamic API in EU/APAC without increasing origin load?"

success_criteria:
  - metric: "p95_ttfb_dynamic_api"
    target: "< 180ms"
    measurement: "RUM + synthetic, 7 days, >1M requests/day"
    tooling: "Catchpoint + mPulse, filtered to /api/v2/checkout"
    pass_threshold: "95% of 5-min windows meet target"

  - metric: "origin_offload"
    target: "> 92%"
    measurement: "Origin request count vs edge requests"
    fail_if: "Origin RPS increases >5% vs baseline"

  - metric: "purge_propagation_p95"
    target: "< 2s global"
    measurement: "Automated purge test every 15 min"
    reference: "https://httpwg.org/specs/ - Cache-Control and CDN-Cache-Control semantics"

decision_rule: "Must pass 2 of 3 primary criteria AND no sev1 incidents during failover test"

Scoping Traffic, Origins, and Failure Modes Like Production

Once you've defined what you're proving, you have to decide how to prove it. The classic mistake is mirroring too little production reality. I worked with a retailer who ran their CDN PoC on a single, pristine origin in us-east-1 with no WAF rules. Of course it was fast. When they cut over for real, they discovered the vendor's WAF added 40ms per request and their origin shield in Frankfurt didn't peer well with their GCP origin. They'd never tested the actual path traffic would take.

Your PoC architecture should be a scaled-down but faithful replica of what you intend to run in production. That includes the boring parts: TLS configuration, origin connection pooling, stale-while-revalidate behavior, and how you handle range requests. For the HTTP semantics, don't just trust the vendor's defaults - validate against the HTTP Working Group Specifications and modern guidance from web.dev Articles on caching best practices. If you're testing multiple vendors, give each the same origin infrastructure and the same traffic split, and use a weighted DNS or load balancer setup you can control. Don't let vendors self-report - instrument at the client and at the origin so you've got an independent source of truth. And please include failure testing. Pull the origin down for 30 seconds. Expire the TLS cert on staging. See what actually serves stale.

Here's the kind of harness I typically have teams deploy for synthetic validation during a PoC. It's not fancy, but it catches 80% of integration surprises before real users do:

#!/bin/bash
# poc-validation-harness.sh - Run every 60s from 3 regions during PoC
set -euo pipefail

VENDORS=("vendor-a.poc.example.com" "vendor-b.poc.example.com")
ENDPOINTS=("/api/v2/checkout" "/static/hero-2000.webp" "/personalized/home")
ORIGIN="origin-staging.example.com"

for vendor in "${VENDORS[@]}"; do
  for path in "${ENDPOINTS[@]}"; do
    echo "--- Testing $vendor$path ---"
    # Check cache status, TTFB, and correctness
    curl -s -o /dev/null -w "%{time_starttransfer},%{http_code},%{size_download}\n" \
      -H "Host: www.example.com" \
      -H "Cache-Control: no-cache" \
      "https://${vendor}${path}" \
      --resolve "www.example.com:443:$(dig +short $vendor | head -1)"

    # Validate surrogate keys / cache tags are stripped correctly
    curl -s -I "https://${vendor}${path}" -H "Host: www.example.com" | grep -i "cache-status\|cdn-cache-control\|age"

    # Test stale-on-error - compare with origin down simulation
    curl -s -H "Host: www.example.com" "https://${vendor}/healthz?origin_test=1" | jq '.originReachable'
  done
done

# Purge propagation test - time to global invalidation
PURGE_START=$(date +%s%3N)
curl -X POST "https://api.vendor-a.com/purge" -H "Authorization: Bearer $TOKEN" -d '{"url":"/static/hero-2000.webp"}'
while true; do
  STATUS=$(curl -s -I "https://vendor-a.poc.example.com/static/hero-2000.webp" -H "Host: www.example.com" | grep -i "x-cache" || true)
  if [[ "$STATUS" == *"MISS"* ]]; then break; fi
  sleep 0.2
done
echo "Purge propagation: $(( $(date +%s%3N) - PURGE_START ))ms"

What to Measure When Marketing Claims Meet Real Traffic

Vendor dashboards will show you gorgeous numbers during a proof of concept. Ignore them, or at least don't rely on them alone. You need your own measurements from Real User Monitoring, synthetic probes, and origin logs. The goal isn't to get the highest hit ratio - it's to get the right hit ratio for your content mix while protecting the origin and keeping tail latency low. I've seen PoCs where a vendor hit 98% cache ratio by aggressively caching query-string variations that should have been unique. Looked great until users started seeing each other's carts.

This is where your CDN Vendor Evaluation Matrix: Scoring Providers on What Actually Matters becomes the scoring engine for the PoC. You're not just logging metrics; you're weighting them by business impact. A 20ms improvement in TTFB for your product detail page might be worth more than a 10% increase in offload if it moves conversion. Agree on those weights before you see the data. And build your Comprehensive CDN Migration Checklist: Every Step from Planning to Cutover around what the PoC teaches you - if purge takes 8 seconds instead of the promised 1 second, that changes your content publish runbook on day one.

Keep the metric set tight. You don't need 30 KPIs; you need five that directly map to user experience and operational risk. Track them consistently across vendors and baseline them against your current CDN or direct-to-origin for at least a full business cycle, including a peak:

Implementing a PoC That Mirrors Production Without Touching It

The fastest way to kill a PoC's credibility is to build it in a lab. We've seen teams stand up a pristine origin, point a test CDN at it, hammer it with 1,000 RPS of synthetic traffic from Loader.io, and declare victory when they hit 98% cache ratio. That tells you nothing about how the CDN will handle your actual traffic - with its 40% long-tail of uncacheable API calls, its 12 query-string variants for the same image, and its brutal 6pm traffic spike from the US East Coast. A useful PoC doesn't need to be production, but it has to *behave* like production.

Don't mirror all of production - isolate a slice you can control. The pattern that works best is header or weighted-DNS based shadowing. Route 5-10% of real user traffic to the PoC CDN using a weighted DNS record or an edge worker that duplicates requests, but keep your primary CDN as the authority. Use a dedicated PoC origin or, better, a read-replica of your production origin with identical autoscaling rules. Enable origin shield in a single region (IAD or FRA typically) and enforce your real cache-key logic from day one - including that `sort-query-string` and `strip-utm` normalization you've been meaning to fix for two years. Run it for at least 7-14 days so you capture a full weekly traffic cycle. Anything shorter and you'll miss weekend vs. weekday behavior. For a full breakdown of what to track and how long to run, use a proper CDN Pilot Testing Methodology: Metrics, Duration, and Success Criteria before you write a single line of config.

You'll want your PoC configuration version-controlled and diffable against production. Don't click-ops this in a vendor dashboard. Here's the Terraform pattern we use to keep PoC and prod in sync while allowing safe experimentation:

# poc-cdn.tf - Production-faithful PoC with 5% weighted traffic
resource "cloudflare_ruleset" "poc_cache_rules" {
  zone_id = var.zone_id
  name    = "poc-cache-optimization"
  phase   = "http_request_cache_settings"

  rules {
    action = "set_cache_settings"
    action_parameters {
      cache = true
      edge_ttl {
        mode    = "override_origin"
        default = 86400
        status_code_ttl = [
          { status_code = 200, value = 3600 },
          { status_code = 404, value = 60 }
        ]
      }
      serve_stale {
        disable_stale_while_updating = false
      }
    }
    expression = "(http.request.uri.path wildcard \"/api/*\" and not http.request.uri.query contains \"nocache\")"
  }
}

# Weighted DNS cutover - 5% to PoC, 95% to primary
resource "aws_route53_record" "www_weighted_poc" {
  zone_id = var.hosted_zone_id
  name    = "www"
  type    = "CNAME"
  ttl     = 60
  weighted_routing_policy { weight = 5 }
  set_identifier = "poc-cdn"
  records        = [var.poc_cdn_endpoint]
}

resource "aws_route53_record" "www_weighted_primary" {
  zone_id = var.hosted_zone_id
  name    = "www"
  type    = "CNAME"
  ttl     = 60
  weighted_routing_policy { weight = 95 }
  set_identifier = "primary-cdn"
  records        = [var.primary_cdn_endpoint]
}

Troubleshooting the PoC: When Your Numbers Don't Add Up

If your PoC's cache hit ratio comes back at 68% when the vendor promised 92%, don't assume the vendor lied - assume your cache keys are broken. About 70% of the "low hit ratio" issues we debug come down to key fragmentation. A single product image served as `/img/123.jpg?w=800&quality=80&v=2` and `/img/123.jpg?quality=80&w=800&v=2` is two separate cache objects unless you've normalized query-string ordering. Same with `Accept-Encoding` and `Vary` headers - if you don't collapse `br` vs `gzip` correctly, you'll double your origin load. Check your `Cache-Status` and `Age` headers on a sample of 10,000 requests; if you see a lot of `MISS` with `Age: 0` on objects that should be hot, you've got a key or `Vary` problem, not a PoP problem.

The other silent killer is origin shield misconfiguration leading to a thundering herd. Without shielding, a regional purge or a cold PoP can send 800-1,200 RPS directly to your origin - we've watched autoscaling groups take 3-4 minutes to catch up, during which p95 origin latency spikes from 45ms to 900ms+ and you start serving 502s. With shielding enabled correctly, that same event collapses to ~40-60 RPS at origin. If you see origin CPU peg at 85%+ during the PoC's first cache fill, your shield isn't working. Validate it by purging a top-100 URL and watching origin logs: you should see exactly one origin fetch per shield region, not one per edge PoP. That behavior is critical to validate before you plan your full CDN Traffic Cutover Strategy: Migrating Without Downtime or Data Loss, because a full cutover without shielding is how you create a self-inflicted DDoS.

Finally, don't ignore DNS. We had a client whose PoC looked perfect for 10 days - 89% hit ratio, 38ms improvement in p75 TTFB - then they flipped the CNAME and lost 18% of traffic for 47 minutes because their DNS TTL was still at 3600s and the old CDN was still serving stale DNS from ISP resolvers. Your PoC should include a DNS TTL burn-down plan from day one: drop your apex and `www` TTLs from 3600s to 60s at least 48 hours before any weighted shift, and monitor resolver propagation with tools like DNSPerf or ThousandEyes. If you're changing DNS providers as part of the migration, test that separately. Our checklist for CDN DNS Migration Planning: Changing DNS Providers Without Breaking Resolution covers the double-TTL and SOA serial pitfalls that most PoCs skip entirely.

Frequently Asked Questions