Edge Cache Compression: Storing More Content with Less Memory

Last Black Friday, one of our busiest edge PoPs in Frankfurt started shedding cache almost an hour before peak traffic. It wasn't disk or CPU — we simply ran out of edge memory. We were storing thousands of uncompressed JSON API responses because a single origin flag had disabled gzip. Hit ratio collapsed from 94% to 71% and origin traffic tripled in minutes. That's the moment compressed caching stopped being a nice optimization and became core infrastructure for us.

Why Edge RAM Is Always the Bottleneck

If you've run CDN edges for any length of time, you know the pattern. Edge memory is expensive, power-hungry, and brutally limited compared to what you'd like to cache. A typical edge server might have 128GB or 256GB of RAM, and after you reserve space for connections, TLS state, and OS overhead, you've got maybe 70% left for actual content. When you're trying to keep hot objects close to users, every megabyte counts, and storing objects uncompressed is like leaving half your warehouse empty.

I learned this the hard way back in 2016 when we were caching a lot of small but highly cacheable assets — SVG sprites, API fragments, personalized HTML shards. Our storage optimization at the time was purely about eviction policy. We'd tuned LRU for weeks and still saw poor byte hit ratios. The problem wasn't what we were evicting, it was how much space each object took up. An average 45KB JSON response could be 8KB with Brotli, but we were keeping the full version in RAM and compressing on the fly for every hit. We were burning CPU to save a bit of implementation complexity.

The fix wasn't complicated, but it required rethinking where compression happens. Instead of storing the origin response verbatim and compressing at serve time, we started storing the compressed representation directly in cache. That approach, which most folks call cache compression or compressed caching, means you pay the compression cost once on miss and then serve the same bytes repeatedly. It also plays nicely with Cache Coherence Protocols at the CDN Edge: Keeping Distributed Caches in Sync because you're not trying to keep multiple encodings coherent across nodes — you just store the canonical compressed variant and handle negotiation at the edge.

# Varnish-style logic for storing compressed objects directly
# We normalize Accept-Encoding and store only the compressed variant

sub vcl_backend_response {
    # Ask origin for compressed content if client supports it
    if (bereq.http.Accept-Encoding ~ "br") {
        set bereq.http.Accept-Encoding = "br";
    } elsif (bereq.http.Accept-Encoding ~ "gzip") {
        set bereq.http.Accept-Encoding = "gzip";
    }

    # Store what origin sends, don't decompress for caching
    if (beresp.http.Content-Encoding ~ "^(br|gzip|zstd)$") {
        # Keep it compressed in cache — don't gunzip
        set beresp.do_gunzip = false;
        set beresp.http.Vary = "Accept-Encoding";
        # Tag for debugging hit vs. miss compression state
        set beresp.http.X-Cache-Storage = "compressed";
    }
    set beresp.ttl = 1h;
}

sub vcl_deliver {
    # If client doesn't support stored encoding, decompress at edge
    if (obj.http.Content-Encoding == "br" && req.http.Accept-Encoding !~ "br") {
        set resp.http.Content-Encoding = "br";
        # Fallback decompression handled by edge filter
    }
}

Picking a Compression Algorithm That Won't Wreck Your TTFB

Not all compression is created equal for edge use. I've seen teams jump straight to the highest ratio algorithm and then wonder why P95 TTFB jumped 20ms. For edge memory, you care about three things together: ratio, compression speed, and especially decompression speed. Decompression happens on every cache hit where you need to transcode, so if it's slow, you've just added latency to your hottest path. That's why our early experiments with heavy gzip levels (-9) looked great for storage but terrible in production.

These days, we've mostly standardized on Zstandard for compressible text assets and let Brotli handle pre-compressed static assets from origin. Zstd at level 3 gives us roughly 85% of the ratio of Brotli level 11 but decompresses 4-5x faster on our Xeon edges, which matters when you're serving 30k requests per second per node. For tiny objects under about 1.5KB, though, I don't compress at all — the dictionary overhead and CPU cost often outweigh the few hundred bytes you save. We've also had good results using Probabilistic Caching with Bloom Filters: Memory-Efficient Cache Lookups alongside compression to decide whether an object is worth recompressing in the first place. If an object hasn't been seen more than twice, we skip the extra work.

import zstandard as zstd

# Edge cache write path: compress once, store compressed
cctx = zstd.ZstdCompressor(level=3, threads=0)
dctx = zstd.ZstdDecompressor()

def cache_store(key, body: bytes, content_type: str):
    # Don't bother compressing images, video, or tiny bodies
    if content_type.startswith(("image/", "video/")) or len(body) < 1536:
        cache.set(key, body, headers={"Content-Encoding": "identity"})
        return

    compressed = cctx.compress(body)
    # Only store if we actually saved >15%
    if len(compressed) < len(body) * 0.85:
        cache.set(key, compressed, headers={
            "Content-Encoding": "zstd",
            "X-Compression-Ratio": f"{len(body)/len(compressed):.2f}"
        })
    else:
        cache.set(key, body, headers={"Content-Encoding": "identity"})

def cache_serve(key, accept_encoding: str):
    blob, headers = cache.get(key)
    enc = headers.get("Content-Encoding")

    if enc == "zstd" and "zstd" not in accept_encoding:
        # Fast decompression for clients without zstd
        if "br" in accept_encoding or "gzip" in accept_encoding:
            # Transcode: decompress then recompress to client's format
            # In practice, cache both if this path is hot
            blob = dctx.decompress(blob)
            enc = "identity"
    return blob, enc

What We Actually Measure Before Calling It a Win

It's tempting to look at compression ratio alone and declare victory. Don't. The metric that pays your bills is origin offload and user-perceived latency, not just bytes saved in RAM. When we rolled out compressed caching to our European PoPs, we tracked effective cache capacity — how many objects we could hold in the same 96GB allocation — alongside hit ratio, byte hit ratio, and decompression CPU. The numbers told a more nuanced story than the marketing decks. We fit about 2.7x more compressible objects in memory, but our overall hit ratio only improved by 11 points because a long tail of incompressible video chunks still dominated evictions.

We also learned to watch for double-compression bugs. If your origin already sends Brotli and your edge recompresses with Zstd, you aren't saving anything — you're just wasting cycles. That's why we now normalize Content-Encoding at the cache key layer and explicitly store the Vary header. The HTTP Working Group Specifications are pretty clear on how Vary and Content-Encoding should interact, and the web.dev Articles on compression negotiation are worth revisiting whenever you change this logic. Pair this with a solid predictive layer like Machine Learning for Cache Warming: Predictive Prefetching That Actually Works and you can prioritize compression for the objects your model knows will be hot tomorrow, not just today.

Below is a snapshot from one of our production edge nodes (Intel Ice Lake, 64 cores, 128GB RAM, ~18k RPS) after a week of A/B testing different storage encodings on text-heavy workloads — HTML, JSON, JavaScript, and CSS. Your numbers will vary by content mix, but the tradeoffs are representative.

Putting It Into Practice: A Compressive Cache Layer That Doesn't Kill TTFB

Look, you've picked your algorithm — let's say Zstd level 3 after your benchmarks — now you've got to actually wire it into the edge without turning every cache hit into a CPU tax. We don't compress on-the-fly for every request. That's a rookie mistake that'll spike your p95 TTFB by 30-40ms. Instead, we compress once at ingest and store both the compressed blob and a tiny uncompressed header in RAM. On a hit, you're just decompressing.

Our standard approach is to keep a 16KB dictionary trained on your most common content-types (HTML, JSON API responses) and share it across POPs. With a shared dictionary, we see 3.1x compression on JSON API payloads versus 2.2x without it, and decompression stays under 0.5ms for objects under 128KB on a standard 2.2GHz edge core. The trick is to bypass compression entirely for objects under ~1KB — the metadata overhead isn't worth it — and for anything already compressed like JPEG, WebP, or WOFF2. We flag those with a X-Compress-Skip header at the origin.

Here's the skimmed-down version of what we run in OpenResty/Lua at the edge to handle this. It checks the content type, decides whether to compress, and handles the decompress-on-hit path with dictionary support. It’s not production-complete, but it’s the core loop that let us push 62% more objects into the same 64GB RAM per node:

-- compressive cache handler (OpenResty / lua-resty)
local zstd = require "resty.zstd"
local DICT = zstd.load_dict("/etc/edge/dict.json.zdict") -- 16KB shared dict

function cache_store(key, body, content_type)
  -- Don't waste cycles on tiny or pre-compressed objects
  if #body < 1024 or content_type:match("image/") or content_type:match("font/") then
    ngx.shared.edge_cache:set(key, body)
    return
  end

  -- Compress once at ingest with dictionary
  local compressed, err = zstd.compress(body, 3, DICT) -- level 3 is our sweet spot
  if not compressed then
    ngx.log(ngx.WARN, "zstd fail, storing raw: ", err)
    ngx.shared.edge_cache:set(key, body)
    return
  end
  
  -- Store with 1-byte flag + original size for fast alloc on hit
  ngx.shared.edge_cache:set(key, string.char(1) .. compressed)
end

function cache_fetch(key)
  local blob = ngx.shared.edge_cache:get(key)
  if not blob then return nil end
  if blob:byte(1) ~= 1 then return blob end -- was stored uncompressed
  
  -- sub-0.5ms decompress for typical <128KB objects
  return zstd.decompress(blob:sub(2), DICT)
end

Advanced Patterns and When Your Hit Ratio Still Drops

So you’ve shipped compression and your RAM utilization looks great — down from 88% to 51% — but your origin shield traffic hasn't actually dropped. We've seen this twice. The first time, we were compressing so aggressively (Zstd 9) that decompression CPU pushed our edge event loop to 85%+ utilization, causing slower eviction handling and premature LRU churn. Dropping back to level 3 cut CPU by 4x and actually *increased* effective hit ratio by 7.2% because the eviction logic could keep up. Don't chase the best compression ratio; chase the best hit ratio per watt.

The second gotcha is interaction with your distribution layer. If you're using compression to fit more objects per node, but your Consistent Hashing for CDN Cache Distribution: A Practical Implementation Guide is still mapping keys naively, you can end up with hot shards where compressed objects are tiny and cold shards where they're not — your memory savings won't be uniform. We now weight our consistent hash ring by actual compressed byte size, not object count. And when you start compressing tagged objects, you need to think about invalidation. A purge that has to decompress 50k objects just to read their headers will stall. That's why we keep Cache Tag Design Patterns: Organizing Content for Granular Invalidation and the surrogate key index uncompressed in a separate slab. It lets us execute Surrogate Keys for Targeted Cache Purging: Beyond URL-Based Invalidation in ~12ms even on a node holding 4.5M compressed objects.

If your decompression latency ever creeps above 2ms at p99, check two things immediately: 1) Are you accidentally double-compressing? Log the Content-Encoding on ingest. 2) Did your dictionary get out of sync across POPs after a training job? A mismatched dictionary means silent decompression failures and a fallback to origin — it'll look like a hit ratio cliff, not a compression error. We version our dictionaries (dict-v37) and ship them via the same config pipeline as VCL, with a 5-minute canary before global rollout.

Frequently Asked Questions