Face Detection and Smart Cropping at CDN Edge: AI-Powered Image Composition

Last spring a fashion retailer called me at 11pm because their new 1:1 category thumbnails were decapitating models. Center-cropping had worked fine for their old landscape hero images, but once the mobile team requested square and 4:5 variants for the feed, faces kept drifting out of frame. We'd seen it before — what looks like a simple resize job turns into a composition failure when you’ve got 4 million product images and no human in the loop. That’s exactly where face detection and smart cropping have to move from a nice-to-have into your CDN’s image pipeline.

The Center-Crop Trap: Why Naive Resizing Breaks Faces

I’ve audited a lot of image pipelines over the last decade, and the default is almost always the same: take the source image, calculate the target aspect ratio, and crop from the center. It’s fast, it’s deterministic, and it completely ignores what’s actually in the picture. When you’re serving a single desktop size it’s tolerable. But the moment you start generating responsive variants on the fly — say, a 16:9 desktop hero, a 3:2 tablet card, and a 1:1 avatar — you’re not just resizing, you’re reframing. A face that sat nicely at 0.5, 0.3 in the original can easily end up at the edge or outside the crop window entirely.

This gets painful fast at CDN scale. You can’t manually set focal points for every image, and you shouldn’t expect your CMS editors to. Content-aware cropping solves this by scoring every region of the image before we cut. Modern AI image processing pipelines don’t just look for skin tones; they combine face bounding boxes, saliency maps, and edge entropy to find where a human eye would actually look. If you’re designing your broader pipeline, it helps to think of smart crop as a routing decision rather than a filter — you’re choosing which pixels survive. I laid out that pattern in Image CDN Architecture Design: Building a Scalable Image Processing Pipeline, and composition is where that architecture pays off. Without it, you’re burning bandwidth on beautifully optimized images that show the wrong thing.

The simplest way to expose this at the edge is through URL directives that accept a gravity strategy. Here’s how we typically model it on our edge image service — note the difference between dumb and smart crop:

# Naive center crop - fast but oblivious
GET /cdn-cgi/image/width=400,height=400,fit=crop,gravity=center/images/model-1184.jpg

# Content-aware smart crop with face priority
GET /cdn-cgi/image/width=400,height=400,fit=cover,gravity=auto:face,focus=entropy
      /images/model-1184.jpg

# Verbose directive for debugging composition
# ?smartcrop=1&face_weight=0.85&saliency_weight=0.15&fallback=entropy
# X-Debug-Crop-Box: x=312,y=88,w=504,h=504 confidence=0.97

Running Face Detection at the Edge Without Blowing Your Latency Budget

Putting face detection on the origin is easy, but it means you’re pre-computing crops for every possible variant or doing an origin round-trip for every new size. Neither scales. We want that decision at the edge, as close to the cache as possible, ideally inside a Worker or Compute@Edge function that runs before you fetch from storage. The catch I learned the hard way: you don’t have a GPU and you’ve got Maybe 30ms before you’ve wrecked your TTFB.

We tried shipping a full RetinaFace model to Fastly Compute in 2021 and it was a disaster — 12MB of weights, 180ms cold start, and instant OOM kills. What actually works at the edge are tiny, quantized models. MediaPipe BlazeFace (short-range) is ~200KB quantized, and the newer face detectors distilled for ONNX Runtime Web are even smaller. You run them on a downscaled 128x128 copy, not the full 3000px source. You’re not trying to detect a 10px face in a crowd; you’re trying to protect the primary subject from a bad crop, and that trade-off saves you ~10x in inference time. As outlined in the HTTP Working Group Specifications, you can also cache the detection result with a Vary on the gravity param and serve it with stale-while-revalidate, so you only pay that cost once per source image.

Below is a stripped-down pattern we use now on Cloudflare Workers with WASM. It resizes first, runs detection, then calculates the smart crop box with face weighting. We don’t run AI on every request — we memoize the face boxes in KV or a cache API for 24 hours:

import { detectFaces } from './blazeface-wasm.js';

export default {
  async fetch(req, env) {
    const url = new URL(req.url);
    const gravity = url.searchParams.get('gravity') || 'auto:face';
    
    // 1. Fetch low-res proxy for detection (128px)
    const thumb = await fetchImageThumb(url.pathname, 128);
    const faces = gravity.includes('face') 
      ? await detectFaces(thumb, { threshold: 0.55 }) 
      : [];

    // 2. Score crop windows - faces get heavy weight
    const cropBox = getSmartCropBox({
      image: thumb,
      targetRatio: getTargetRatio(url),
      faces,                 // [{x,y,w,h, confidence}]
      weights: { face: 0.85, saliency: 0.15 },
      fallback: 'entropy'    // if no face, use entropy/saliency
    });

    // 3. Issue subrequest with calculated crop, cache face data
    const res = await fetchImageWithCrop(url.pathname, cropBox, url.searchParams);
    res.headers.set('X-Face-Count', String(faces.length));
    res.headers.set('X-Crop-Box', JSON.stringify(cropBox));
    env.FACE_CACHE.put(url.pathname, JSON.stringify(faces), { expirationTtl: 86400 });
    return res;
  }
}

How We Benchmark Smart Cropping: Accuracy, Speed, and Visual Quality

You can’t just trust your eyes on this — I’ve seen smart crops that looked fine in testing but failed on darker skin tones or on profile faces. We benchmark on three axes: localization accuracy, latency impact, and perceived quality. For accuracy we use Intersection over Union (IoU) against hand-labeled focal boxes, not just face boxes. A crop that includes the face but cuts off the chin or hair still fails if your brand guidelines require headroom. For perceived quality we borrow heavily from web.dev Articles on Cumulative Layout Shift and Largest Contentful Paint — a perfectly cropped image that arrives 120ms later can still hurt your Core Web Vitals more than a slightly off-center crop served from cache.

There’s also a systems trade-off people don’t talk about enough. A heavier model gives you better recall on small or occluded faces, but it forces you to choose between higher edge CPU time and more cache variants. If you’re already generating responsive variants automatically, as described in Responsive Image CDN Automation: Generating srcset Variants on the Fly, adding smart crop shouldn’t multiply your origin storage. We generate crops just-in-time and cache them with a surrogate key like `img:123:smartcrop`, which lets us purge the source and all its crops together. And because cropping affects what’s preloaded above the fold, you’ll want to coordinate this with your loading strategy — see Balancing Lazy Loading and Preloading with CDN: Optimal Image Loading Strategies — so the hero variant your smart crop worked hardest on is actually the one the browser fetches first.

From Theory to Edge: Shipping Smart Crop in a Worker

We’d burned two sprints trying to get this right without melting our edge latency budget. The trick isn’t just picking a good model — it’s knowing when *not* to run it. On Cloudflare Workers and Fastly Compute@Edge, you’ve got maybe 50ms of CPU before you start eating into TTFB. Our production setup runs a distilled BlazeFace variant (roughly 1.2MB WASM) that hits ~8ms p50 and 18ms p95 for a 640x640 input on an isolate. Anything larger and we downscale first; you don’t need 4K resolution to find a face box.

We also cache aggressively. The face coordinates get cached as a tiny JSON blob at the edge for 30 days with a stale-while-revalidate, keyed by the image’s content hash, not URL. That means the detector only runs once per unique image. On a typical media site doing 200M image requests/day, we saw detector invocations drop from 200M to about 1.8M — a 99.1% reduction. Cache hit ratio for the derived crops sits at 96.4%. If you’re already handling other image transforms, this slots right into that pipeline alongside things like CSS Sprite Generation and CDN Optimization: When Sprites Still Beat HTTP/2 — you don’t want to re-run compute you’ve already paid for.

Here’s the stripped-down pattern we run in production. It checks the cache first, falls back to detection, then computes the focal point for cropping. We don’t crop to the face itself — we crop to keep the face’s center of gravity in the safe zone:

// Edge Worker - Smart Crop with WASM Face Detection
import { detectFaces } from './blazeface-lite.wasm';

export default {
  async fetch(req, env, ctx) {
    const url = new URL(req.url);
    const targetW = parseInt(url.searchParams.get('w') || '600');
    const targetH = parseInt(url.searchParams.get('h') || '600');
    
    // 1. Check coordinate cache first (avoids WASM spin-up)
    const cacheKey = `face:${hash(url.pathname)}`;
    let faces = await env.FACE_CACHE.get(cacheKey, 'json');
    
    if (!faces) {
      const imgBuffer = await fetchOriginImage(url.pathname);
      const smallImg = await downscale(imgBuffer, 640);
      faces = await detectFaces(smallImg); // ~8-18ms
      ctx.waitUntil(env.FACE_CACHE.put(cacheKey, JSON.stringify(faces), {
        expirationTtl: 2592000
      }));
    }
    
    // 2. Calculate smart focal point (weighted face center + rule-of-thirds)
    const focalPoint = getFocalPoint(faces, targetW, targetH);
    return fetchAndCrop(url.pathname, focalPoint, targetW, targetH);
  }
}

Advanced Patterns and When the Crop Still Fails

Face detection will betray you on group shots. We learned this the hard way when a 12-person team photo got cropped to a tight shot of the person on the far left because their face scored 0.99 confidence and everyone else was slightly out of focus at 0.82. Now we don’t just take the highest-confidence face. We compute a weighted bounding box that encompasses all detected faces, then add 15% padding on each side. If the encompassing box would require more than a 40% zoom to fit the target aspect ratio, we fall back to entropy-based saliency instead of face priority. That fallback saved us on about 7% of requests.

You’ll also need a strategy for images without faces. Product shots, landscapes, illustrations — your detector will return nothing about 62% of the time on a general-purpose site. For those, we run a fast saliency pass (we use a 3KB gradient/entropy heuristic, not a second ML model) that takes under 2ms. And for vector assets, don’t overthink it: smart cropping is a raster problem. If you’re serving icons or logos, you’re better off with proper SVG Delivery Optimization Through CDN: Inline, External, and Sprite Approaches than trying to crop them. We bypass the entire detection pipeline if the content-type is image/svg+xml.

The nastiest bug we hit wasn’t visual — it was privacy. Our detector was preserving EXIF orientation and face coordinates in cache headers during debugging, and one of our log pipelines was shipping face bounding boxes to a third-party analytics vendor. Don’t do that. Strip EXIF at the edge before you do anything else, and treat face geometry as PII. We now run Image Metadata Stripping at CDN Edge: EXIF Removal for Privacy and Performance as a mandatory first step — it shaves 12-35KB per image anyway — and we never log coordinates outside our isolated image service. If you need auditability, hash the coordinates.

Frequently Asked Questions