Image CDN Format Negotiation: Serving AVIF, WebP and JPEG XL by Browser

Serving the optimal image format for every browser shouldn't require maintaining three separate URLs or forcing your frontend to guess. Yet I still see teams doing exactly that — hardcoding .webp everywhere or running client-side JavaScript detection before an image even loads. After running edge image optimization for catalogs with hundreds of millions of requests per day, I can tell you that proper format negotiation at the CDN edge is cleaner, faster, and far more maintainable. The idea is simple: the browser tells you what it can decode via the Accept header, and the edge delivers the most efficient format you have available, all from a single canonical URL. The execution, however, involves cache key design, Vary semantics, fallback logic, and a clear understanding of where AVIF, WebP, and JPEG XL actually pay off.

How the Accept Header Actually Drives Format Selection at the Edge

In my experience, most confusion around format negotiation starts with misunderstanding what the browser actually sends. For navigation requests, browsers send a broad Accept: text/html,... header. For images loaded via <img>, background-image, or fetch, modern browsers send a distinct Accept header that advertises image capabilities. A current Chrome request for an image looks like Accept: image/avif,image/webp,image/apng,image/svg+xml,image/*,*/*;q=0.8. Safari as of 17+ sends Accept: image/avif,image/webp,*/* on supporting versions, while older Safari will send image/webp only if supported, otherwise image/* or */*. Firefox sends a similar ordered list with quality values.

The order and the q values matter. The header is not just a list; it is a weighted preference. image/avif;q=0.9,image/webp;q=0.8 means the client prefers AVIF but will accept WebP. In practice, most CDNs and edge image services do not implement full RFC 9110 quality-value parsing — they perform ordered substring matching. That is usually sufficient, but you should be aware of the gap if you have a custom implementation. The HTTP Working Group Specifications define this content negotiation mechanism, and it is worth reading the section on proactive negotiation to understand why server-driven selection based on Accept is the correct model for images, rather than reactive negotiation with a 300 Multiple Choices response.

Parsing Quality Values and Wildcards Without Over-Engineering

I've found that trying to perfectly respect every q=0.2 permutation creates brittle logic. A pragmatic edge rule is: if Accept contains image/avif, serve AVIF if you have it encoded and the source image benefits from it. Else if it contains image/webp, serve WebP. Else if it contains image/jxl (for the handful of browsers that support JPEG XL), serve JXL. Otherwise, serve the original JPEG or PNG. This covers 99.9% of traffic. Wildcards like image/* or */* should be treated as "no modern format supported" unless a more specific type is present — don't assume wildcard means WebP capable.

One nuance: do not trigger format negotiation based on User-Agent sniffing. I have migrated several properties off User-Agent logic because it requires constant maintenance, breaks with new browser releases, and fails with privacy-preserving User-Agent reduction. Accept is the explicit contract for this purpose.

Where Client Hints Fit In

Client Hints like Sec-CH-Prefers-Reduced-Data or the deprecated Accept-CH: DPR, Width, Viewport-Width are sometimes proposed for format selection. In practice, they are for resource hints and responsive sizing, not format capability. Keep format negotiation on Accept. If you are building a fully responsive image pipeline, pair your format negotiation with proper markup as described in Responsive Images: srcset, sizes and Art Direction for Every Screen — the edge chooses the best format, while srcset and sizes choose the best resolution. They solve different axes of the problem.

AVIF vs WebP vs JPEG XL: Where Each Format Wins on Real Traffic

From a pure compression standpoint, AVIF consistently delivers the smallest files for photographic content at equivalent perceptual quality. In production tests across fashion and marketplace catalogs, I typically see AVIF at 35-50% smaller than JPEG at SSIM 0.96+, and 20-30% smaller than WebP at the same quality target. WebP is more modest — roughly 25-35% smaller than JPEG — but encodes 8-10x faster and decodes with significantly lower CPU cost on mobile.

That trade-off is the core decision. AVIF uses AV1 intra-frame coding, which is computationally heavy. Encoding a 2048px hero image to AVIF can take 1-2 seconds on a single core at effort 6, versus 80-150ms for WebP. Decoding AVIF on a low-end Android device can add 60-120ms of main-thread time for large images. If you serve a product listing page with 40 images, forcing AVIF for every thumbnail can hurt Largest Contentful Paint and interaction more than the byte savings help.

JPEG XL is the wildcard. Technically it is excellent — lossless JPEG transcoding, progressive decoding, and 30-40% savings over JPEG with much faster encoding than AVIF. But browser support is the blocker. As of late 2024-2025, JPEG XL is supported in Safari 17+ but not in Chrome or Firefox stable without flags. I have evaluated it for origin-to-edge archival where you can transcode JPEGs losslessly to JXL for storage and then derive WebP/AVIF at the edge; that saves origin storage. For direct-to-browser delivery, JXL is not yet a primary negotiation target unless your audience is heavily Safari-weighted.

My current policy for most e-commerce and media sites is: generate WebP eagerly for the entire catalog (fast, safe default), generate AVIF asynchronously for images larger than 800px on the long side or for hero/detail images where bytes matter most, and hold JXL as an origin format. If you need guidance on tuning quality targets so AVIF at quality 50 looks like WebP at 75, see Image Compression Quality: SSIM, PSNR and Perceptual Metrics for CDN Optimization — visual metrics are essential to avoid blanket quality settings that either bloat files or introduce banding.

Content Negotiation Without Breaking CDN Cache Hit Ratios

The most common operational failure I see is a team enabling Accept-based negotiation and then watching their cache hit ratio drop from 96% to 62%. The reason is cache key explosion. If your CDN caches variants based on the full Accept header value, every slight difference in header order or q value creates a separate cache entry: image/avif,image/webp,*/* and image/webp,image/avif,*/* become two objects for the same image.

You need to normalize the variation into a small, discrete set of cache keys. In my configurations, I map the header to one of four buckets: avif, webp, jxl, or jpeg. The edge logic classifies the request into a bucket, and the cache key includes only the bucket, not the raw header string. Most commercial CDNs now support this as a built-in "image format negotiation" or "auto WebP/AVIF" feature that handles the normalization for you. If you are building it yourself on Varnish, Fastly, CloudFront Functions, or Cloudflare Workers, you must implement the bucketing explicitly.

Origin Story: To Store Variants or Transcode on Demand?

There are two architectures. Pre-generate and store: your image pipeline writes 12345.avif, 12345.webp, and 12345.jpg to origin storage (S3/GCS) and the edge simply selects the file. This gives predictable latency but multiplies storage and invalidation complexity. On-the-fly transcoding: you store only the master (ideally a high-quality JPEG or PNG) and the edge image optimizer transcodes at request time and caches the derivative. This is far more storage-efficient and I recommend it for catalogs over 1M images.

The hybrid I often deploy is origin holds the master plus a WebP derivative (since WebP is cheap to generate synchronously), while AVIF is generated lazily on first AVIF-capable request and then cached at the edge and in a persistent derivative store. That way you don't pay AVIF encode cost for images that are never requested by capable browsers, and you avoid an AVIF encode blocking the first request — you can serve WebP immediately and warm AVIF in the background if your platform supports async generation.

Vary Headers, Cache Keys, and Why Your CDN Serves the Wrong Format

Even with correct bucket classification, you must emit the right Vary response header. Vary: Accept tells shared caches and browsers that the response varies based on the Accept request header. Without it, an intermediate cache that doesn't understand your custom cache key may serve a cached AVIF to a browser that only understands JPEG, resulting in a broken image.

However, Vary: Accept alone is insufficient if you also vary on other dimensions. For example, if you do responsive resizing via query parameters like ?width=800, you don't need Vary for that — the URL already differs. But if you use Accept-CH: DPR or vary on Accept-Language for image overlays, you need Vary: Accept, Accept-Language. Be precise; don't emit Vary: * as it disables caching entirely in many implementations.

In my experience, another subtle bug is double-encoding content negotiation with Content-Type. When you serve an AVIF from a .jpg URL, you must change the Content-Type to image/avif and ensure Content-Length and ETag reflect the variant, not the source JPEG. Failing to update ETag causes 304 Not Modified responses to mismatch and browsers to hold stale variants. The correct pattern is to generate a variant-specific ETag, e.g., ETag: W/"abc123-avif", or to disable ETag and rely on Cache-Control and Last-Modified.

For debugging, I always add a response header like X-Image-Format: avif or X-Cache-Variant: webp so you can verify from curl what was negotiated without inspecting binary content. Remove it in production if you are strict about header bloat, but during rollout it saves hours.

Implementing Format Negotiation in VCL, Workers, and Nginx

The implementation depends on where you terminate. Here are three patterns I have used in production. All of them follow the same bucket model described earlier.

VCL (Fastly / Varnish) - Bucket Classification and Key Control

In VCL you can classify in vcl_recv and normalize before lookup, so the hash key stays clean. You also set a header to pass the variant to your image backend or to select the file extension.

sub vcl_recv {
    # Normalize Accept into discrete variant
    if (req.http.Accept ~ "image/avif") {
        set req.http.X-Img-Variant = "avif";
    } elsif (req.http.Accept ~ "image/webp") {
        set req.http.X-Img-Variant = "webp";
    } elsif (req.http.Accept ~ "image/jxl") {
        set req.http.X-Img-Variant = "jxl";
    } else {
        set req.http.X-Img-Variant = "jpeg";
    }
    # Use variant as part of cache key
    set req.http.Vary = "Accept";
}

sub vcl_hash {
    hash_data(req.url);
    hash_data(req.http.X-Img-Variant);
    return (lookup);
}

sub vcl_backend_response {
    # Ensure downstream caches respect variation
    if (!beresp.http.Vary) {
        set beresp.http.Vary = "Accept";
    } else if (beresp.http.Vary !~ "Accept") {
        set beresp.http.Vary = beresp.http.Vary + ", Accept";
    }
    set beresp.http.X-Img-Variant-Served = bereq.http.X-Img-Variant;
    set beresp.http.Content-Type = if(bereq.http.X-Img-Variant == "avif", "image/avif",
                                   if(bereq.http.X-Img-Variant == "webp", "image/webp",
                                   if(bereq.http.X-Img-Variant == "jxl", "image/jxl", "image/jpeg")));
}

Edge Worker (Cloudflare / Fastly Compute / Lambda@Edge)

Workers give you more flexibility to inspect headers and rewrite the origin request to a variant path or to an image optimization service. Keep the logic synchronous and fast — avoid fetching or heavy computation in the worker itself.

export default {
  async fetch(request, env, ctx) {
    const accept = request.headers.get("Accept") || "";
    let variant = "jpeg";
    if (accept.includes("image/avif")) variant = "avif";
    else if (accept.includes("image/webp")) variant = "webp";
    else if (accept.includes("image/jxl")) variant = "jxl";

    // Clone request and add variant header for origin/image service
    const url = new URL(request.url);
    // Example: pass to image resizing service via query or header
    url.searchParams.set("format", variant);

    const newReq = new Request(url, request);
    newReq.headers.set("X-Img-Variant", variant);
    newReq.headers.set("Vary", "Accept");

    let resp = await fetch(newReq, { cf: { cacheKey: url.toString() + "|" + variant } });
    resp = new Response(resp.body, resp);
    resp.headers.set("Vary", "Accept");
    resp.headers.set("Content-Type", variant === "avif" ? "image/avif" :
                                     variant === "webp" ? "image/webp" :
                                     variant === "jxl" ? "image/jxl" : "image/jpeg");
    resp.headers.set("X-Img-Variant-Served", variant);
    return resp;
  }
}

Nginx Origin with Conditional Serving

If negotiation happens at origin Nginx rather than the edge, use a map to avoid repetitive if blocks and to keep configuration testable.

map $http_accept $img_variant {
    default                 jpeg;
    "~*image/avif"          avif;
    "~*image/webp"          webp;
    "~*image/jxl"           jxl;
}

server {
    location ~* ^/images/(.*)\.(jpg|png)$ {
        add_header Vary Accept;
        add_header X-Img-Variant-Served $img_variant;

        # Try variant file first, fall back to original
        try_files /cache/$1.$img_variant /origin/$1.$2 =404;

        # Set correct Content-Type based on variant
        location ~* \.avif$ { default_type image/avif; }
        location ~* \.webp$ { default_type image/webp; }
        location ~* \.jxl$  { default_type image/jxl; }
    }
}

Note that the Nginx example serves pre-generated files from disk. For on-the-fly transcoding, replace try_files with a proxy pass to an image service like Thumbor, Imagor, or your own libvips pipeline, passing $img_variant as a query parameter.

Fallback Strategies and Origin Shielding for Mixed-Format Image Catalogs

A robust negotiation layer must assume failure. AVIF files can be corrupt due to a bad encode job, a variant may not yet exist in the derivative store, or a crawler may send a nonsensical Accept header. I always implement a fallback chain: attempt to serve the negotiated variant; if the file is missing or the transcode fails, fall back to WebP if available, then to the source JPEG/PNG. Never return a 404 for a missing variant when the source exists — the image URL is canonical and should resolve.

You should also handle range requests and conditional requests correctly. If a client sends If-None-Match for an AVIF variant, ensure you compare against the variant's ETag, not the JPEG's. I've seen image optimizers incorrectly reuse the origin JPEG's ETag for all variants, which causes browsers to incorrectly reuse a cached JPEG when they asked for AVIF.

Shielding, Stale-While-Revalidate, and Thundering Herds

When you deploy AVIF negotiation across a large catalog, the first wave of AVIF-capable traffic will cause a surge of transcodes — the thundering herd. Origin shielding and stale-while-revalidate help. Configure the edge to collapse concurrent requests for the same variant (request coalescing / shielding) so only one transcode hits origin. Then serve stale WebP while AVIF is being generated asynchronously if your platform supports it.

Set sensible Cache-Control values per variant. I use public, max-age=31536000, immutable for versioned image URLs (with fingerprint in path) and public, max-age=86400, stale-while-revalidate=604800 for unversioned URLs. The long immutable TTL for versioned assets means a variant, once cached at the edge, stays there without revalidation — which is why correct Vary and cache key separation from day one is so critical.

Finally, measure the impact correctly. Byte savings alone can be misleading if decode cost increases. Pair your CDN logs with Real User Monitoring. I track format distribution (percentage served as AVIF/WebP/JPEG), median bytes per image, and LCP regression by format. In one rollout, shifting product thumbnails to AVIF saved 28% bytes but increased LCP by 90ms on Moto G4 — we reverted thumbnails to WebP and kept AVIF only for PDP heroes, netting the best of both. This kind of trade-off analysis pairs well with Lazy Loading with Intersection Observer: Reducing CDN Bandwidth with Smart Loading because deferring offscreen images amplifies the value of format savings while reducing decode pressure.

References for implementation details include the negotiation semantics documented at the HTTP Working Group Specifications and practical guidance on next-gen formats at web.dev Articles, particularly their coverage of AVIF and WebP serving patterns.

Frequently Asked Questions