CDN Performance Benchmarking: Methodology and Tools
Why Benchmark CDNs
CDN performance varies by: geographic region (a CDN fast in the US may be slow in Southeast Asia), content type (static vs. dynamic), time of day (congestion patterns), and configuration (your specific cache rules and origin setup). Generic "fastest CDN" claims are meaningless without context. Benchmark with your own content, from your users' locations. For foundational context, read CDN Monitoring and Observability: Metrics That Matter — FlyFish NWA.
Benchmarking Methodology
Step 1: Define What You're Measuring
- TTFB — Time to first byte. Measures CDN's processing + network latency.
- Full page load — Complete page render time. Includes all assets.
- LCP — Largest Contentful Paint. Google's key Core Web Vitals metric.
- Cache hit latency vs. miss latency — How fast cache hits are vs. cache misses.
- Purge propagation time — How quickly cache invalidation takes effect globally.
Step 2: Choose Testing Locations
Test from locations where your users are. A CDN optimized for North America won't help if 40% of your users are in Asia.
| Metric | Edge Processing | Origin Processing | Improvement |
|---|---|---|---|
| Latency (p50) | 8–15 ms | 120–300 ms | 15–20x faster |
| Cache Hit Ratio | 85–95% | N/A | Reduces origin load |
| TLS Handshake | 1–5 ms (nearby PoP) | 50–150 ms | 10–30x faster |
| Bandwidth Cost | $0.01–0.04/GB | $0.05–0.12/GB | 3–5x cheaper |
| Availability | 99.99% (multi-PoP) | 99.9% (single DC) | 10x fewer outages |
If your audience is:
60% US → Test from 5 US cities (East + West + Central)
20% Europe → Test from London, Frankfurt, Paris
15% Asia → Test from Tokyo, Singapore, Mumbai
5% Other → Test from São Paulo, SydneyStep 3: Test Methodology
- Run tests for at least 24 hours (catches time-of-day variations)
- Test the same content across all CDN providers
- Include cache-hit AND cache-miss scenarios
- Test during normal traffic AND peak hours
- Run at least 100 requests per location per hour
Testing Tools
| Tool | Type | Best For |
|---|---|---|
| Catchpoint | Synthetic | Enterprise-grade, 800+ locations |
| WebPageTest | Synthetic | Free, detailed waterfall analysis |
| Datadog Synthetics | Synthetic | Integration with APM stack |
| curl + cron | DIY | Simple, free, customizable |
Simple DIY Benchmark Script
#!/bin/bash
# Test TTFB from current location
for cdn in cloudflare cloudfront fastly; do
echo "=== $cdn ==="
for i in $(seq 1 100); do
curl -o /dev/null -s -w "%{time_starttransfer}
" "https://$cdn-test.example.com/benchmark.jpg"
done | awk '{sum+=$1} END {print "Avg TTFB:", sum/NR, "s"}'
doneInterpreting Results
Look at percentiles, not averages. P50 shows typical experience; P95 shows tail latency; P99 shows worst-case. A CDN with 20ms P50 but 500ms P99 has an intermittent problem that averages hide.
Common Pitfalls
- Testing only from one location (not representative)
- Testing only cache hits (miss latency matters too)
- Short test duration (misses time-of-day patterns)
- Not matching real traffic patterns (test with representative content)