When 10,000 users request the same uncached object within 200 milliseconds, does your CDN send 10,000 requests to origin? Without request collapsing, it will. That thundering herd can saturate origin connections, spike p95 latency, and turn a routine cache miss into an outage.
Request collapsing — also called request coalescing — solves this by merging simultaneous cache misses for an identical object into a single origin fetch. One request goes forward, the rest wait, and all receive the same response. It's one of the most effective yet under-configured origin protection mechanisms in modern how CDN works architectures.
What Request Collapsing Actually Does
At its core, request collapsing is a synchronization primitive for the edge cache. When a cache node receives multiple concurrent requests for an object that is not in cache, expired, or being revalidated, it elects a single “leader” request to contact the origin. All other requests for that same cache key are parked in a waiting queue.
Once the origin responds, the edge replicates that response to every waiter and populates the cache in one write. From the origin’s perspective, a flash crowd of 5,000 requests looks like one.
This matters most in three scenarios:
- Cold start / cache flush: After a purge or PoP restart, every object is a miss. Without collapsing, each PoP fans out N parallel origin fetches per object.
- High TTL churn: Small, popular objects with short TTLs (live scores, pricing JSON, HLS segments) expire simultaneously for thousands of clients.
- Cache stampede on revalidation: When
stale-while-revalidateis not used, expirations create a synchronized miss window.
Collapsing is not caching. It does not increase hit ratio directly. It protects origin availability and tail latency during miss storms. For a full primer on caching layers, see our what is CDN overview and CDN fundamentals guide.
/product/123.json with different Vary headers or query-string normalization will not collapse unless your cache key is configured to treat them as identical.How Coalescing Works at the Edge
Implementation varies by proxy, but the state machine is consistent:
| Metric | Edge Processing | Origin Processing | Improvement |
|---|---|---|---|
| Latency (p50) | 8–15 ms | 120–300 ms | 15–20x faster |
| Cache Hit Ratio | 85–95% | N/A | Reduces origin load |
| TLS Handshake | 1–5 ms (nearby PoP) | 50–150 ms | 10–30x faster |
| Bandwidth Cost | $0.01–0.04/GB | $0.05–0.12/GB | 3–5x cheaper |
| Availability | 99.99% (multi-PoP) | 99.9% (single DC) | 10x fewer outages |
- Lookup: Request arrives, cache key is computed (host + path + query + Vary).
- Miss detection: If no fresh entry exists and no in-flight fetch is registered for that key, the node creates a “fetch lock” and forwards the leader request.
- Queue: Subsequent requests finding the lock do not forward. They subscribe to the in-flight transaction.
- Fan-out: On origin response, the node writes to cache once, then duplicates the response buffers to all waiters.
- Timeout / error handling: If the leader fails or times out, one waiter is promoted to become the new leader (or all fail fast, depending on config).
Modern stacks implement this with lightweight in-memory locks. In Nginx it’s proxy_cache_lock; in Varnish it’s waitinglist; in ATS and Fastly it’s native request coalescing. The lock is held only for the duration of the origin transaction, typically 20–400ms.
For HTTP/2 and HTTP/3, coalescing interacts with multiplexing. A single QUIC connection may carry dozens of streams for the same asset (e.g., browsers retrying). Proper coalescing prevents those streams from multiplying origin load. See CDN HTTP/2, HTTP/3 and QUIC for multiplexing implications.
Collapsing vs. Deduplication vs. Origin Shield
These terms are often conflated, but they operate at different layers:
| Mechanism | Scope | What It Collapses | Best For |
|---|---|---|---|
| Request Collapsing | Single edge node, in-memory | Concurrent misses for same key within milliseconds | Thundering herd, flash crowds |
| Request Deduplication | Edge node or cluster | Identical in-flight requests (often includes POST de-dupe) | API idempotency, retry storms |
| Origin Shield | Regional mid-tier PoP | Cross-PoP misses (global fan-in) | Multi-PoP origin offload |
| Stale-While-Revalidate | Per object | Revalidation window (serve stale + async refresh) | Eliminating miss latency entirely |
The strongest architecture combines collapsing at the edge with a shield in the middle. Edge collapsing handles the micro-burst (thousands in milliseconds); shield handles the macro-burst (hundreds of PoPs missing at once after a purge). Without both, a global purge of a popular object can still generate one origin request per PoP — often 150+ simultaneous hits.
Load balancing also influences collapse efficiency. If your CDN load balancing algorithms spread identical requests across many nodes without consistent hashing, collapse hit rate drops. Use consistent hash or rendezvous hashing for cache keys.
Implementation Strategies & Configuration
Enable collapsing only for cacheable, idempotent requests. The cache key must be deterministic and normalized before the lock is checked.
1. Nginx (proxy_cache_lock)
proxy_cache_lock_age prevents a stalled leader from blocking waiters forever. After 5s, a new leader is elected. background_update lets stale responses be served while refresh happens asynchronously — reducing queue depth.
2. Varnish (built-in waitinglist + grace)
Varnish coalesces by default via its waitinglist. Tuning beresp.grace and keep is critical — grace allows stale delivery while collapsing handles the refresh.
3. Fastly / VCL Services
4. Cloudflare Workers (manual coalescing)
At the application layer, use single-flight patterns (Go singleflight, JS promise maps) for origin services that sit behind the CDN. Collapsing at the edge does not protect origin-internal fan-out.
Cookie, Authorization, or user-specific headers unless those headers are part of the cache key. See CDN security guide for cache poisoning considerations.Measuring Effectiveness
Collapsing is invisible if you only watch hit ratio. Track these instead:
- Origin request coalescing ratio:
(client misses - origin fetches) / client misses. Aim for >80% during bursts. - Collapsed request count: Exposed as
proxy_cache_lockwaits (Nginx$upstream_cache_status), VarnishMAIN.n_waitinglist, Fastlycollapsesmetric. - Origin concurrency: p99 concurrent connections to origin should flatten during bursts when collapsing is active.
- Wait time histogram: Time waiters spent queued. Should be < origin TTFB. Long tails indicate origin slowness, not collapse overhead.
Load test with a synchronized miss: purge an object, then fire 1,000 concurrent GETs from multiple regions. With collapsing off, origin sees ~1,000 fetches; with it on, origin should see 1–3 fetches (one per PoP/shield).
Pitfalls and Tuning Tradeoffs
Collapsing is not free. Misconfiguration causes worse behavior than no collapsing at all.
Head-of-line blocking: If the leader fetch stalls for 10s, 5,000 waiters also stall. Always set proxy_cache_lock_timeout or equivalent to 5–10s and serve stale if available. Pair collapsing with stale-while-revalidate and stale-if-error so waiters can be served immediately on timeout.
Cache key explosion: Overly granular keys (e.g., including User-Agent or random query params) prevent coalescing. Normalize keys: sort query strings, strip tracking params, and only Vary on what you must.
Large objects & Range requests: Do not collapse Range: bytes=0-1023 together with full-object requests. Use slice or range-aware caching, or disable collapsing for Range.
Thundering herd on error: If origin returns 500, do not cache the error for too long, but also do not immediately retry with all waiters. Use negative caching (e.g., proxy_cache_valid 500 5s) and exponential backoff for leader re-election.
When tuned correctly, collapsing reduces origin QPS by 10–100x during bursts, cuts origin bandwidth, and smooths p95 latency without any client-visible changes. It is a foundational control — enable it before adding more origin capacity.
FAQ
What is the difference between request collapsing and request coalescing?
They are the same mechanism with different vendor terminology. Both describe collapsing multiple simultaneous cache-miss requests for the same object into a single origin fetch, then fanning the response out to all waiting clients. Fastly and Akamai typically say “request collapsing” while Varnish and Nginx docs use “coalescing.”
Does request collapsing increase Time To First Byte for users?
Slightly for collapsed requests. The first request pays the full origin latency; subsequent coalesced requests wait instead of triggering parallel origin fetches. The wait is almost always shorter than a thundering herd would cause, and it prevents origin timeout cascades. Configure a waiting timeout (e.g., 5–10s) to fail fast if origin stalls.
Should I enable request collapsing for uncacheable or POST requests?
No. Collapsing should only apply to cacheable GET/HEAD requests with identical cache keys. Never collapse POST, PUT, or requests with Authorization or Cookie-dependent personalization, or Range headers unless your cache key explicitly isolates them. Collapsing uncacheable traffic can cause data leakage or incorrect responses.
How does request collapsing work with Origin Shield?
They are complementary. Collapsing coalesces concurrent requests at each edge PoP, while Origin Shield adds a mid-tier aggregation layer so only one PoP forwards to origin. Together they reduce origin requests from thousands per second to one per object per TTL, even during global cache flushes.
Next steps: audit your edge config for proxy_cache_lock / waitinglist status, enable stale-while-revalidate, and add an Origin Shield if you run more than 10 PoPs. Small changes here prevent the most common CDN-induced origin outages.