CDN Log Sampling Strategies: Reducing Storage Costs Without Losing Signal

Last winter a client called me in a panic after their monthly log storage bill hit $42,000. They were pushing about 80,000 requests per second through their CDN and logging every single one, 100% fidelity, because someone told them they'd need it for debugging. They didn't. What they needed was a smarter CDN Log Aggregation Architecture: Collecting Billions of Events Without Dropping Data and a sampling strategy that kept the signal while cutting 90% of the noise. I've made that same mistake early in my career, and it taught me that storing everything isn't thorough — it's just expensive.

The Math That Makes Full Logging Unsustainable

When you're running a small site, logging everything feels harmless. A few gigabytes a day, you can grep it, ship it to S3, and forget about it. At CDN scale, that math falls apart fast. A typical edge log line with request headers, timing, cache status, and TLS info runs 0.8 to 1.5 KB when serialized as JSON. Multiply that by 10,000 requests per second — which is modest for a mid-sized e-commerce property — and you're generating roughly 1.2 TB per day. Keep that for 90 days for compliance and you're staring at over 100 TB before replication, indexing, and query costs.

I learned this the hard way back in 2016 managing logs for a video streaming platform. We insisted on 100% sampling for the first six months because we were terrified of missing a cache miss pattern. By month three, our Elasticsearch cluster was spending more time rotating indices than answering queries. The team couldn't run a simple status code breakdown without waiting 20 minutes. We weren't preserving signal; we were burying it. That's when we started treating CDN Real-Time Log Streaming Architecture: From Edge Events to Analytics in Seconds as a cost-optimization problem, not just a data collection one. If you can't query your logs quickly and cheaply, you don't have observability.

The real cost isn't just storage on disk. It's the downstream pipeline: the Kinesis or Kafka throughput, the transformation Lambda functions, the BigQuery or Snowflake slots you burn scanning terabytes for a dashboard that only needs thousands of rows. Effective CDN log management means you decide what to keep before you pay to move it, store it, and index it. A straightforward way to model this is to calculate your daily ingest before you choose a rate:

# Estimate daily log volume and sampling impact
# Based on real numbers from a recent retail client

requests_per_second = 25000
avg_log_size_kb = 1.2
seconds_per_day = 86400

raw_gb_per_day = (requests_per_second * avg_log_size_kb * seconds_per_day) / (1024 * 1024)
print(f"Raw volume: {raw_gb_per_day:.1f} GB/day")  # ~2468 GB/day

# With 10% head sampling
sample_rate = 0.10
sampled_gb_per_day = raw_gb_per_day * sample_rate
monthly_cost_at_023_per_gb = sampled_gb_per_day * 30 * 0.023

print(f"Sampled volume: {sampled_gb_per_day:.1f} GB/day")
print(f"Est. S3 Standard cost: ${monthly_cost_at_023_per_gb:.2f}/month (before query/indexing)")

# Deterministic sampling keeps cardinality for distinct user counts
def should_sample(request_id, rate_percent):
    # Use hash of request ID for stable, deterministic decision
    import hashlib
    hash_val = int(hashlib.md5(request_id.encode()).hexdigest()[:8], 16)
    return (hash_val % 100) < rate_percent

Head-Based vs. Tail-Based Sampling: When You Decide Changes Everything

There are two places you can make the sampling decision, and they give you very different results. Head-based sampling decides at the start of the request — often using a hash of the URL, a cookie, or a random number — whether to log it at all. It's cheap, it's simple to implement in VCL or an edge worker, and it reduces load on your entire pipeline instantly. The downside is that you're blind. You might sample away the one 5xx error that explains an outage because you decided before you knew how the origin would respond.

Tail-based sampling flips that. You make the decision after you have the response. You have the status code, the cache state (HIT, MISS, PASS), the origin latency, and the full set of response headers defined in the HTTP Working Group Specifications. This lets you build rules like "keep 100% of 5xx, 10% of 4xx, and 1% of successful 2xx cache hits." I've found this is the only sane way to run a serious log sampling setup. You're still cutting 90%+ of your volume, but you're keeping the errors and the outliers that actually tell you something. The trade-off is you have to buffer and evaluate every request at the edge, which takes a bit more memory and logic.

For most teams, I recommend starting with tail-based rules implemented directly at the edge. You don't need a complex sidecar. A small worker that checks the response before emitting the log event does the job. Here's a simplified version of what we run in production on Cloudflare Workers and Fastly Compute@Edge — it keeps high-value traffic without drowning you in successful image requests:

// Tail-based sampling at the edge (Edge Worker / VCL-style logic)
export default {
  async fetch(request, event) {
    const response = await fetch(request);
    
    // Always log what matters for reliability
    const status = response.status;
    const cacheStatus = response.headers.get("x-cache") || "UNKNOWN";
    const contentType = response.headers.get("content-type") || "";

    let sampleRate = 0.01; // Default: 1% for 2xx HITs

    if (status >= 500) {
      sampleRate = 1.0;      // Keep every server error
    } else if (status >= 400) {
      sampleRate = 0.25;     // Keep 25% of client errors
    } else if (cacheStatus === "MISS") {
      sampleRate = 0.10;     // Keep 10% of origin fetches
    } else if (contentType.includes("text/html")) {
      sampleRate = 0.05;     // Keep 5% of HTML docs for performance checks
      // See https://web.dev/articles/ for why HTML timing matters more than assets
    }

    if (Math.random() < sampleRate) {
      event.waitUntil(sendToLogPipeline({
        url: request.url,
        status,
        cacheStatus,
        sampleRate,
        timestamp: Date.now()
      }));
    }
    return response;
  }
}

Tuning Sample Rates by Traffic Tier Without Losing the Story

You don't need one global sampling rate. In fact, you shouldn't have one. The best sampling strategy I've seen splits traffic into tiers based on business impact and debuggability. Static assets like .jpg, .css, and .woff files that return a 200 and a HIT from cache are almost never worth logging beyond 0.5% to 1%. They tell you very little about user experience once you've confirmed your cache hit ratio is stable. HTML documents, API calls, and anything that goes to origin are different — they carry latency, personalization, and error signals you need for Clickstream Analysis at CDN Edge: Understanding User Navigation Patterns.

We audited a media site last year that was sampling everything at a flat 5%. They were paying to store millions of identical 200s for thumbnails while discarding most of their checkout API errors, simply because errors were rare in absolute numbers. We moved them to a tiered model and their storage bill dropped from ~$18k to ~$2.1k per month with no loss in alerting quality. The trick is to set your tiers around the questions you actually ask your logs: "Are origins healthy?", "Are users seeing errors?", and "Is cache performance degrading?" If a sample doesn't help answer one of those, you can probably cut it.

Start conservative, then measure. After we roll out a new tier, we run a two-week comparison where we log at 100% for a single point-of-presence and compare the sampled metrics against the full-fidelity baseline. You want your sampled p95 latency, error rate, and cache hit ratio to stay within 2-3% of the true value. Below is a realistic baseline we use as a starting point for intermediate CDN deployments — adjust the rates based on your own risk tolerance, but don't go below 0.1% for any tier you care about or your percentiles will start to lie to you.

Implementing Tiered Sampling in Your Edge Config and Log Pipeline

Theory's great, but wiring this up without breaking your observability is where teams get burned. You don't want a single global sample rate — you want deterministic rules running right at the edge that decide what gets logged before it ever hits your collector. Most of us do this in VCL, Nginx Lua, or a Cloudflare/Fastly Compute Worker. The key is to make the decision early, attach a sampling flag as a header, and let your log shipper respect it. That way you keep 100% of what matters and shed the noise.

Here's the pattern we've used to cut log volume by 94% without losing signal on conversions. We keep 100% of 4xx/5xx, 100% of POSTs to /checkout and /api/cart, 100% of requests with our first-party attribution cookie, and 1% of successful cache HITs for static assets. Everything else — mostly anonymous 200s for images, JS, and video segments — gets sampled at 0.5%. That turns a typical 3.2 TB/day firehose (at ~85k rps average) into about 180 GB/day. At $0.50/GB for ingestion and 30-day retention in S3 + Athena, that's the difference between an $48k/month logging bill and about $2,700. And critically, we can still rebuild full User Journey Analytics from CDN Logs: Mapping the Path to Conversion because we never sample a session that touched a conversion path.

// Example: Fastly VCL + Vector (Remap) tiered sampling
sub vcl_log {
  // Default: sample 0.5% of cache hits
  if (resp.status >= 400) {
    set req.http.X-Sample-Keep = "1"; // Keep all errors
  } else if (req.url ~ "^/(checkout|api/cart|login)") {
    set req.http.X-Sample-Keep = "1"; // Keep all critical paths
  } else if (req.http.Cookie:__attrib_id) {
    set req.http.X-Sample-Keep = "1"; // Keep all attributed sessions for
                                      // Conversion Attribution from CDN Edge: First-Party Tracking Without Client-Side Scripts
  } else if (fastly_info.state ~ "HIT" && req.url ~ "\.(jpg|png|js|css|ts)$") {
    if (randomint(0, 1000) > 5) { // 0.5% sample
      set req.http.X-Sample-Keep = "0";
    } else {
      set req.http.X-Sample-Keep = "1";
    }
  } else {
    if (randomint(0, 100) > 1) { // 1% sample for other 200s
      set req.http.X-Sample-Keep = "0";
    }
  }
  // In Vector / Fluent Bit, drop if X-Sample-Keep == "0"
  // and add `sample_rate` field for extrapolation: 200 for 0.5%, 100 for 1%
}

Adaptive Sampling, Outlier Preservation, and Fixing Blind Spots

Static rates will lie to you the moment traffic shifts. A 1% sample is perfect at 50k rps but you'll drown during a 500k rps flash sale, and you'll starve for data at 2k rps at 3 AM. That's why we moved to adaptive sampling. We set a target — say, 2,000 logs/sec per PoP — and adjust the rate every 30 seconds based on an EWMA of request volume. If traffic spikes 10x, the sample rate auto-drops from 1% to 0.1% to hold the volume flat. You’ll need to emit the effective sample_rate with every log line so your downstream counts can be weighted back up (count * (1/rate)). Don't forget to do that, or your dashboards will show traffic collapsing when it's actually surging.

The other classic mistake is accidentally averaging away your outliers. If you head-sample a 1% uniform random slice, you'll almost certainly miss that single 5-second TTFB outlier that affected 12 users in Boise. For tail-based alerting, we keep a second, tiny pipeline: we don't sample *errors*, but we also buffer for 2-3 seconds in memory and make a tail decision. If any request in a trace exceeds p95 latency, has a cache MISS + origin 503, or is flagged by our Bot Traffic Analysis and Classification from CDN Logs: Separating Good from Bad model as a credential-stuffing attempt, we keep the entire session. Yes, it costs a bit more RAM at the edge, but we’ve caught bot bursts that a 1% head sample would have smoothed to invisibility.

When your numbers look off, check these three things first. One, are you sampling *before* you classify bots? If you sample 1% of all traffic and 60% of your traffic is bots, you’ve just sampled mostly bots. Filter and classify *before* you sample — keep 100% of verified bots in a separate low-retention bucket or drop them entirely. Two, is your extrapolation correct? A common gotcha is double-counting when you JOIN sampled logs with 100%-sampled conversion events — you'll need to use weighted counts (SUM(1/sample_rate)) not COUNT(*). Three, watch for header cardinality blowup. If you log every User-Agent or X-Forwarded-For at full fidelity on sampled data, you'll still break your ClickHouse strains. Hash or bucket high-cardinality fields at the edge.

Frequently Asked Questions