Search Autocomplete at CDN Edge: Instant Suggestions from Edge-Cached Indexes

Last Black Friday I was watching a real-time dashboard for a fashion retailer doing about $4M a day, and their search autocomplete was averaging 340ms. Doesn't sound terrible, right? Except we could see in the session replays that shoppers were typing "blac" waiting, getting impatient, hitting enter, and landing on a no-results page instead of getting instant suggestions for "black leather boots." That 300ms gap was quietly killing their conversion rate on the most valuable entry point they had.

Why That Extra Round-Trip Kills Search Autocomplete

Traditional autocomplete is simple on paper: user types a character, browser fires a fetch to your origin or search API, you return the top 5-8 suggestions. In practice, that round-trip is brutal for ecommerce search. Even with a fast origin in us-east-1, a shopper in Berlin or São Paulo is paying 120-180ms just in network time before your search logic even starts. Add TLS, JSON serialization, and the fact that most search backends need 20-40ms to rank suggestions, and you’re consistently over 200ms per keystroke. Humans perceive anything over 100ms as lag, and for search autocomplete, that lag breaks the feeling of instant suggestions.

I've made this mistake before. Years ago we tried to "optimize" a client's origin autocomplete by adding more aggressive caching with Cache-Control headers and tuning the database. It helped a bit, but we were still bouncing every keystroke across the ocean. The HTTP Working Group Specifications (https://httpwg.org/specs/) are clear that caching can reduce origin load, but they can't fix physics. If your architecture requires a dynamic request to origin for every "a", "ab", "abc", you’ve already lost. That's why CDN Product Page Optimization: Sub-Second Loading for Higher Conversion Rates matters so much — product pages get the attention, but search is often the first interaction, and it needs to be faster than the page itself.

Moving that logic to the CDN edge flips the model. Instead of proxying requests, you serve suggestions directly from memory at the edge PoP that’s 10-30ms away from the shopper. We've seen this cut p95 suggestion latency from 290ms to under 45ms without touching the core search engine. The trick isn't putting a cache in front of your API; it's putting the entire suggestion index at the edge so there's no origin fetch on the hot path at all.

// Edge worker serving autocomplete from an in-memory prefix index
// Stored in Workers KV / Edge Dictionary - updated via incremental deltas
export default {
  async fetch(request, env) {
    const url = new URL(request.url);
    const q = url.searchParams.get("q")?.toLowerCase().trim();
    if (!q || q.length < 2) return jsonResponse([]);

    // 1. Check edge memory first (no origin fetch)
    const prefixKey = `suggest:${q.slice(0,3)}`;
    let bucket = await env.SUGGEST_KV.get(prefixKey, { type: "json" });
    
    if (!bucket) {
      // Fallback to stale origin only on true miss - not per keystroke
      bucket = await fetchOriginSuggestions(q, env);
    }

    const suggestions = rankAndFilter(bucket, q).slice(0, 8);
    return jsonResponse(suggestions, {
      headers: {
        "Cache-Control": "public, max-age=60",
        "CDN-Cache-Tag": "autocomplete",
        "X-Cache": "HIT-edge-index"
      }
    });
  }
}

function rankAndFilter(bucket, query) {
  // Client-side ranking is cheap; pre-sorted by popularity + margin
  return bucket.items.filter(item => item.prefix.startsWith(query));
}

Building an Edge-Cached Index That Doesn't Go Stale

You can't just dump your entire product catalog into an edge dictionary and call it edge search. I tried that approach with a home-goods catalog of 180,000 SKUs — serialized it to JSON at 47MB and blew up the worker memory limit on deploy. For ecommerce search, you don't need full product data at the edge, you need a compact suggestion index. What’s worked best for me is a sharded prefix tree where we only store the top queries, categories, and product names that actually drive suggestions. Think of it like this: you pre-compute the most popular 15,000-30,000 suggestion strings, group them by their first 2-3 characters, and store each bucket as a small JSON object (usually 5-12KB).

The harder part is freshness. If someone searches for a new collection that launched this morning and your edge index is 24 hours old, you look broken. We don't do full purges anymore. Instead, we publish incremental deltas every 2-5 minutes from the merchandising system to the edge store via API, and we tag each bucket with a version. A background cron worker pulls the delta feed — new products, price changes, out-of-stock flags — and patches only the affected prefix buckets. According to guidance in web.dev Articles on stale-while-revalidate patterns, serving slightly stale data while revalidating in the background is far better than blocking on a fresh fetch. We apply the same idea at the edge: serve from the cached index instantly, revalidate the bucket asynchronously if its age exceeds our threshold. It keeps suggestions relevant without adding latency.

# Cache policy for autocomplete suggestion buckets at the edge
# Serve stale instantly, update deltas in background - no request blocking

Cache-Tag: autocomplete-prefix
Cache-Control: public, max-age=120, stale-while-revalidate=600, stale-if-error=86400
CDN-Cache-Control: max-age=300
Surrogate-Key: suggest-ab suggest-ac suggest-ad

# Purge / patch workflow (not full invalidation)
# PATCH /edge-store/suggest:ab  { "add": ["abercrombie hoodie"], "remove": ["abused denim"] }
# Edge worker merges patch into existing bucket in < 5ms
# Fallback: If bucket version > 10 mins old, worker triggers background revalidation

# Client request remains < 50ms regardless of revalidation state
fetch("/api/suggest?q=abo", { 
  headers: { "Accept": "application/json" },
  priority: "high" 
});

What We Actually Measure After Moving Autocomplete to the Edge

When we pitch edge autocomplete internally, teams always ask about hit rate, not just latency. They assume edge indexes will miss constantly. In reality, with a well-built prefix index covering the top 20,000 suggestions, we typically see 92-96% of keystrokes served entirely from edge memory without any origin contact. The remaining 4-8% are long-tail prefixes or very fresh terms that trigger a background fetch and then get merged into the index. Compare that to a traditional CDN cache in front of an origin search API, where your hit rate often hovers around 40-60% because query strings are so unique — "bl", "bla", "blac" are all separate cache keys that rarely repeat across users.

That hit rate directly impacts your origin and your business metrics. One electronics retailer we worked with cut search API origin requests by 88% after moving to edge-cached indexes, which let them downsize that search cluster by two nodes. More importantly, their search-to-product-click rate jumped 14% because instant suggestions actually kept people in the search funnel. If you're thinking about the full journey, that improvement compounds with CDN Checkout Flow Acceleration: Edge Optimization for the Revenue-Critical Path and keeping state consistent — we’ve had to solve for edge personalization without breaking CDN Cart Session Management: Maintaining Shopping State Across Edge Nodes, so suggestions can be personalized by region or customer segment without losing speed.

Don't just measure averages. Track p50, p95, and origin offload separately for autocomplete traffic. Averages hide the pain that shoppers on mobile networks feel. Here’s what a typical before/after looks like when you compare origin-served search autocomplete against an edge-cached prefix index for a mid-size catalog of ~80,000 products.

How We Actually Ship It: Workers, KV, and Sharded Prefix Tries

Don't try to shove your entire catalog into a single 15MB JSON blob and cache it at the edge — you'll blow your KV limits and your TTFB will crawl. What we've found works far better is sharding the index by prefix. Think 2-character shards: `sh`, `sh-o`, `sho` — each shard is a tiny, sorted JSON array of your top 25-30 suggestions for that prefix, pre-ranked by popularity, margin, or whatever your merch team cares about. A full US catalog with 850k SKUs compresses down to about 2.4MB uncompressed, but once gzipped and sharded, each shard averages 8-14KB. That means a cache hit from KV or R2 plus edge cache serves in 11-18ms p50, versus 85-120ms if you're still bouncing back to origin for every keystroke.

Your build pipeline shouldn't be a nightly cron job anymore. We run a full index build once a day at 03:00 UTC, but then we stream deltas every 45-60 seconds via CDC from your search service (Kafka, Kinesis, whatever you've got) into a queue. A lightweight builder Worker consumes the queue, patches the affected shards, and writes them back to KV with a versioned key like `ac:v42:sh`. At the edge, your Autocomplete Worker does a `cache.match()` first, then falls back to KV, and serves stale-while-revalidate for 60s so you never block a user while a shard updates. If you're already doing CDN Inventory API Caching: Real-Time Stock Visibility with Aggressive TTLs you'll recognize the pattern — aggressive TTL at the edge (300s) with instant purge on version bump. Purge propagation is typically under 800ms globally on Cloudflare and Fastly, so stale suggestions age out before anyone notices.

// Cloudflare Worker - edge autocomplete with sharded KV + tiered cache
export default {
  async fetch(request, env) {
    const url = new URL(request.url);
    const q = (url.searchParams.get('q') || '').toLowerCase().slice(0, 20);
    if (q.length < 2) return jsonResponse([]);

    const shardKey = `ac:v42:${q.slice(0, 2)}`;
    const cacheKey = new Request(`https://ac.edge/${shardKey}`, request);
    let res = await caches.default.match(cacheKey);
    
    if (!res) {
      // KV read is ~3-7ms from edge, R2 is ~15-25ms
      const shard = await env.AC_INDEX.get(shardKey, { type: "json", cacheTtl: 300 });
      if (!shard) return jsonResponse([]);
      
      const prefixMatches = shard.filter(item => item.p.startsWith(q)).slice(0, 8);
      res = jsonResponse(prefixMatches);
      res.headers.set("Cache-Control", "public, max-age=60, stale-while-revalidate=300");
      // non-blocking write to edge cache
      event.waitUntil(caches.default.put(cacheKey, res.clone()));
    }
    return res;
  }
}
function jsonResponse(data) {
  return new Response(JSON.stringify(data), {
    headers: { "Content-Type": "application/json", "Cache-Tag": "ac-index" }
  });
}

Beyond the Happy Path: Typos, Ghost SKUs, and Thundering Herds

Autocomplete that only works when the user spells perfectly isn't autocomplete — it's a quiz. But you don't want to run full Levenshtein or SymSpell at the edge on every `keyup` either; that'll add 25-40ms of CPU time and you'll hit Worker CPU limits fast. Our compromise: precompute the top 2-3 typo variants for your 10k most-searched prefixes during the build step and bake them into the shard. So `iphnoe` maps to the `iph` shard with a `corrected: "iphone"` flag. For the long tail, we keep a tiny 45KB BK-tree dictionary at the edge and do a single-edit distance check only if the exact prefix shard returns zero results. It keeps p95 under 35ms instead of ballooning to 90ms+ with naive fuzzy logic, and you still catch ~82% of fat-finger queries.

The ugliest bug you'll hit is showing a suggestion for a product that went out-of-stock 30 seconds ago. Users click, land on an OOS PDP, and bounce — it kills conversion faster than a slow response. Don't try to make your autocomplete index perfectly real-time; instead, do a lightweight post-filter at the edge. We fetch a heavily cached inventory bitset (TTL 15s) and filter suggestions before returning them. It's the same edge-lookup trick we use for Price Personalization at CDN Edge: Dynamic Pricing Without Origin Round-Trips where we merge personalized price without ever hitting origin — you can attach price and availability in the same Worker without adding a round-trip. If the inventory check fails or is stale, we still return the suggestion but demote it and mark it `low_stock: true` so the UI can grey it out. Much better than lying.

If your hit ratio suddenly tanks from 99.2% to 73% or you see version skew where `us-west` serves v42 while `us-east` is still on v41, you've likely got a purge or cache-key problem. First, make sure you're purging by `Cache-Tag` not URL — purging 1,296 shards by URL takes 12+ seconds and you'll get thundering herds. Tag all shards `ac-index` and do a single soft purge. Second, watch for personalized cache keys leaking. If you're running edge experiments, isolate your A/B bucket in a cookie or header and vary the cache key cleanly. We've seen teams accidentally cache `user_id` in the shard key and create millions of unique keys. If you're testing ranking models, do it properly with A/B Testing Product Pages at CDN Edge: Experimenting on Layout and Content patterns — bucket at the edge, log exposure, but keep the index cache shared.

Frequently Asked Questions