CDN Percentile Latency Analysis: What p50, p95, and p99 Actually Tell You

Last Black Friday, our dashboard showed a beautiful p50 of 42ms across North America. Everything looked green. Then support tickets started rolling in from users on mobile in the Midwest complaining about 3-second page loads. The average hid it, the p50 hid it, but the p99 was screaming at 2.8 seconds — we just weren’t looking at it closely enough. If you’ve run a CDN for any length of time, you’ve lived some version of that story.

Why Averages Lie to You at CDN Scale

When I started working on edge performance about a decade ago, we reported on mean latency. It felt intuitive. You add up all your response times, divide by the number of requests, and you’ve got your number. The problem is that CDN latency is never normally distributed. It’s almost always multimodal and heavily skewed by a long tail. You’ve got cache hits serving in 15ms sitting next to cache misses that have to go back to origin, TLS handshakes on cold connections, and the occasional packet loss recovery that blows a single request out to seconds.

A single 5-second origin fetch can drag your mean up by more than hundreds of fast hits can pull it down, and conversely, a million fast hits can mask thousands of terrible experiences if you’re only watching the average. I’ve seen a PoP where we improved the mean by 8ms and celebrated, only to realize we’d made the tail worse because we’d introduced a retry policy that helped the median but punished the slowest 5% of users. That’s the core danger of RTT Measurement Methodology for CDN: Getting Accurate Round-Trip Times without percentile context — you can’t tell if your measurements are telling you about the typical experience or just the noisy edges. Percentile latency fixes this by asking a different question: what was the latency for the request at a specific rank in your sorted distribution?

If you sort every request in a time window from fastest to slowest, the p50 is the request in the middle, p95 is the request where 95% of requests were faster, and p99 is where 99% were faster. It doesn’t average anything; it just picks a position. For CDN analytics, that’s far more useful because it lets you isolate what different groups of users actually experienced. Here’s a tiny Python snippet I still use to sanity-check logs when a dashboard feels off — it shows how dramatically mean and percentiles can diverge on real data:

import numpy as np

# latencies in milliseconds from a 1-minute edge log sample
latencies = np.array([18, 22, 19, 25, 21, 24, 20, 23, 1800, 28, 19, 350, 21, 22, 2900])

print(f"mean: {np.mean(latencies):.1f}ms")
print(f"p50:  {np.percentile(latencies, 50):.1f}ms")
print(f"p95:  {np.percentile(latencies, 95):.1f}ms")
print(f"p99:  {np.percentile(latencies, 99):.1f}ms")

# Output:
# mean: 346.1ms
# p50:  22.0ms
# p95:  2090.0ms
# p99:  2758.0ms
# The mean says 346ms, which describes almost no one.
# p50 says most users were fine, p95/p99 show the tail is broken.

What p50, p95, and p99 Are Actually Telling You

Think of these three percentiles as three different personas querying your edge. I’ve come to explain them to teams like this: p50 is your happy-path user. They hit a warm PoP, got a cache HIT, reused an HTTP/2 or HTTP/3 connection, and didn’t hit any congestion. If your p50 moves, something fundamental changed — like your cache hit ratio, your edge routing, or your payload size. It’s a great stability signal, but a terrible indicator of pain. You can have a perfect p50 of 30ms while 5% of your users can’t load the checkout page.

p95 is where you start seeing the real world intrude. This isn’t an outlier anymore; if you serve 10 million requests per minute, p95 represents 500,000 requests that were slower than that threshold. Every minute. These users are often on lossy mobile networks, geographically far from a PoP, hitting cache MISS or STALE, or negotiating TLS from scratch. When I’m tuning performance, p95 is usually my primary service level objective. It’s sensitive enough to catch regressions from a bad configuration push or an origin slowdown, but not so noisy that it jumps around every time a single user on satellite internet loads your site. If you’re trying to understand the relationship between throughput and latency here, it helps to consider Bandwidth-Delay Product and CDN Tuning: Filling the Pipe Without Overwhelming It, because p95 latency often degrades when you’re not filling the pipe efficiently for users with higher round-trip times.

p99 (and beyond, like p99.9) is your canary for systemic edge cases. These are the users who hit every worst-case at once: cold TCP connection + TLS handshake + cache miss + origin connection coalescing + packet loss. On a well-run CDN, p99 might be 10x to 20x your p50, and that’s normal. What isn’t normal is when p99 suddenly triples while p50 stays flat. I once chased a p99 spike that turned out to be a single origin shield node with a saturated disk — only 1% of global traffic went through it, so means and medians didn’t move, but for those users the Time To First Byte was over 4 seconds. You won’t find that without slicing your latency distribution by PoP, cache status, and ASN. A useful query for this in Prometheus with histogram data looks like this:

# p50, p95, p99 from a histogram metric like cdn_request_duration_seconds
# Works with histogram_quantile on Prometheus / Grafana

histogram_quantile(0.50, sum(rate(cdn_request_duration_seconds_bucket[5m])) by (le))
histogram_quantile(0.95, sum(rate(cdn_request_duration_seconds_bucket[5m])) by (le))
histogram_quantile(0.99, sum(rate(cdn_request_duration_seconds_bucket[5m])) by (le))

# To isolate the tail by cache status - this is where the truth often hides
histogram_quantile(0.99, sum(rate(cdn_request_duration_seconds_bucket{cache_status="MISS"}[5m])) by (le))
histogram_quantile(0.99, sum(rate(cdn_request_duration_seconds_bucket{cache_status="HIT"}[5m])) by (le))

Reading the Distribution Without Lying to Yourself

There’s a temptation to treat p95 and p99 as simple pass/fail thresholds — set an alert at 500ms and forget it. That misses the point of percentile latency analysis. The shape of the distribution between those points tells you what kind of problem you have. A healthy CDN distribution is steep early and flat late: p50 is low, p95 is maybe 2-4x p50, and p99 is perhaps 5-10x p50. When I see p50 at 35ms, p95 at 110ms, but p99 at 1900ms, I don’t think “we’re 99% good.” I think “something is catastrophically wrong for 1%,” which could be thousands of users per minute.

The other trap is aggregating too broadly. A global p95 that looks fine can hide a regional disaster. I always break down CDN analytics by PoP, by cache status (HIT vs MISS vs PASS), by HTTP version, and by content type. Your static assets might have a pristine p99 of 80ms while your API calls that must go to origin have a p99 of 3 seconds — blending them together gives you a meaningless middle number. The same applies to protocol effects. For users on high-latency links, default TCP settings can artificially inflate tail latency, which is why TCP Window Scaling and Its Impact on CDN Throughput: Why Defaults Fall Short is directly relevant to why your p95 and p99 don’t improve even when your edge is fast. And for methodology, we rely on standardized timing definitions from the HTTP Working Group Specifications and modern metrics guidance from web.dev Articles to ensure we’re measuring Time to First Byte and response completion consistently.

Below is a pattern I look for when reviewing a weekly performance report. These aren’t universal targets — they depend on your content and origin — but they illustrate how to interpret a p50 p95 p99 spread for a typical edge-cached asset mix:

Putting Percentiles Into Practice: From Logs to Alerts

You can't fix what you're not measuring correctly, and most CDN latency setups aren't. If you're still pulling averages from your edge logs or relying on your CDN provider's 5-minute aggregated dashboard, you're flying blind where it matters most. Averages will happily tell you everything's fine at 78ms while 5% of your users are waiting 800ms+ for cache misses to crawl back from origin.

At scale, you've got two jobs: capture the full distribution without drowning in data, and make it queryable. Don't log every request's latency as a raw gauge — you'll burn through cardinality and your Prometheus will hate you. Use histograms. On the edge, we use HDRHistogram or DDSketch in the data plane, or let Prometheus-style histograms do the bucketing. You want bucket boundaries that actually reflect user experience: 10ms, 25ms, 50ms, 100ms, 250ms, 500ms, 1s, 2s. Linear buckets are useless here. Your 95% lives between 100ms and 300ms, your 99% lives out past 500ms, so give those regions resolution.

Here's the pattern we run in production for per-PoP, per-status (HIT/MISS) percentile alerts. This PromQL works on standard histogram metrics from Nginx/Varnish or Envoy sidecars, and it's way more honest than an avg() query:

# Don't alert on average latency - alert on the tail
# p95 by PoP and cache status over 5m window
histogram_quantile(0.95,
  sum by (pop, cache_status, le) (
    rate(cdn_edge_request_duration_seconds_bucket[5m])
  )
)

# p99 for MISS path only - this is your origin + network truth
histogram_quantile(0.99,
  sum by (pop, le) (
    rate(cdn_edge_request_duration_seconds_bucket{cache_status="MISS"}[5m])
  )
)

# Alert when p95 degrades > 40% vs 1h ago (catches slow burns)
(
  histogram_quantile(0.95, sum by (le) (rate(cdn_edge_request_duration_seconds_bucket[5m])))
  /
  histogram_quantile(0.95, sum by (le) (rate(cdn_edge_request_duration_seconds_bucket[5m] offset 1h)))
) > 1.4

Pro tip: Split your percentiles by HIT vs MISS from day one. We've seen setups where p50 was a rock-solid 42ms, but p95 crept from 180ms to 340ms over two weeks. The culprit wasn't the edge at all — it was a 15% increase in MISS ratio after a cache-key change, and origin p99 had ballooned to 1.2s. If you'd mixed HIT and MISS into one histogram, you'd have chased ghosts at the edge for days.

Advanced Patterns: When p50 Is Flat but Users Are Still Suffering

Here's where most teams misread the distribution. If your p50 hasn't moved but your p95 and p99 are diverging — say, p50 at 45ms, p95 at 210ms, p99 at 890ms — you don't have a general capacity problem, you've got a tail problem. That tail is almost never random. It's usually one of three things: origin retrieval variance, congested interconnects on specific PoPs, or transport-layer weirdness that doesn't show up in averages.

First, correlate your tail with jitter, not just latency. A stable 150ms is playable for video; a stream that bounces between 40ms and 250ms is unwatchable, even if the average looks great. High p99-p95 spread is your early warning for jitter. If that spread doubles while p50 holds, look at CDN Jitter Measurement and Its Impact on User Experience before you start tuning caches — you'll often find bufferbloat or policers on a peering link. Similarly, don't confuse provisioned bandwidth with actual goodput. We've debugged "slow" PoPs where bandwidth graphs were at 40% but throughput collapsed due to loss and retransmits; that's why you need to understand CDN Throughput vs Bandwidth: Understanding the Difference and Why It Matters at the packet level.

Second, if your p99 is 4-6x your p95 consistently, suspect packet reordering or loss recovery. At 25 Gbps+ per cache node, even 0.02% reordering will hammer your tail because TCP has to stall and reorder. We once chased a p99 spike to 1.8s on a single PoP that turned out to be a faulty line card reordering packets in bursts every 90 seconds. No error counters fired, averages barely moved, but p99 lit up like a Christmas tree. If you're seeing throughput drop without bandwidth saturation, check out Packet Reordering Effects on CDN Performance: The Hidden Throughput Killer — it's the hidden killer that standard monitoring misses. Your fix isn't more cache, it's pinning that PoP out of rotation and getting the network team involved.

Frequently Asked Questions