JSON API Aggregation at CDN Edge: Combining Backend Calls into Single Responses

Last Black Friday, I watched our product detail page fire off seven separate API calls from the browser - prices, inventory, reviews, recommendations, shipping estimates, personalization, and promos. On a fast office connection it looked fine. On a throttled 4G trace, the waterfall stretched to 1.9 seconds before the first meaningful paint. We didn't need faster backends; we needed fewer round-trips. That's when we moved aggregation to the CDN edge and let a single edge request do the fan-out for us.

Why Client-Side Waterfalls Don't Scale on Real Networks

Every additional client-to-origin round-trip pays a tax you can't optimize away with backend tuning. Even with HTTP/2 or HTTP/3 multiplexing, you're still bound by RF propagation, TCP handshakes, and TLS negotiation on mobile networks. I've seen teams spend months shaving 20ms off a database query while their app still makes five sequential XHRs that each cost 80-120ms in network overhead alone. The math never works in your favor, especially outside major metros where our edge PoPs are often 10-30ms away but the origin is 100ms+ away.

Moving that composition to the edge flips the topology. The client makes one request to the nearest PoP, and the edge worker fans out to your microservices over the CDN's backbone - often over warm, keep-alive connections inside the same region or even the same data center federation. You collapse five client round-trips into one, and you run the backend calls in parallel where latency is cheapest. It's not a new idea; it's the Backend-for-Frontend pattern, but executed in V8 isolates instead of a centralized BFF cluster that becomes another hop.

There are trade-offs, and I've learned to respect them. Edge isolates have tight CPU and memory ceilings, subrequest limits can bite you at fan-out scale, and cold starts still matter for that first aggregated response. If you're planning a wide fan-out (say, 6+ backends), you'll want to read up on Edge Function Cold Start Optimization: Reducing Initialization Latency to Milliseconds and Edge Function Execution Limits: CPU Time, Subrequests, and Creative Workarounds before you ship. Done right, though, the latency win is immediate and measurable.

// Cloudflare Workers / Fastly Compute@Edge style aggregation
// Single client request -> parallel backend fan-out over backbone

export default {
  async fetch(request, env) {
    const url = new URL(request.url);
    const productId = url.searchParams.get('id');
    if (!productId) return new Response('Missing id', { status: 400 });

    const headers = { 'accept': 'application/json' };

    // Fan-out in parallel - don't await sequentially
    const [price, inventory, reviews] = await Promise.all([
      fetch(`https://api.internal/prices/${productId}`, { headers, cf: { cacheTtl: 5 } }),
      fetch(`https://api.internal/inventory/${productId}`, { headers, cf: { cacheTtl: 2 } }),
      fetch(`https://api.internal/reviews/${productId}?limit=3`, { headers, cf: { cacheTtl: 60 } })
    ]);

    // Fail fast if critical data is missing, degrade gracefully otherwise
    if (!price.ok) return new Response('Price unavailable', { status: 502 });

    const body = JSON.stringify({
      id: productId,
      price: await price.json(),
      inventory: inventory.ok ? await inventory.json() : { status: 'unknown' },
      reviews: reviews.ok ? await reviews.json() : []
    });

    return new Response(body, {
      headers: {
        'content-type': 'application/json',
        'cache-control': 'public, max-age=5, stale-while-revalidate=30',
        'cdn-cache-control': 'max-age=60' // per HTTP Working Group Specifications (https://httpwg.org/specs/)
      }
    });
  }
}

Graceful Degradation: When One Backend Fails, Don't Fail Them All

The first version we shipped treated aggregation as all-or-nothing. If recommendations timed out, the whole product response returned 500. That's worse than the client-side waterfall - at least then you'd get partial content. I spent a full weekend back in 2021 untangling that mistake after a flaky personalization service took down the entire PDP during a sale. Now I treat every aggregated endpoint as a contract with tiers: critical vs. degradable data.

Critical data (price, inventory availability) should fail the response if it can't be fetched. Everything else should be optional and have a sensible fallback - empty array, cached stale value, or null with a flag the UI understands. The key is Promise.allSettled() instead of Promise.all(), plus explicit timeouts per subrequest. You can't rely on the edge runtime's default fetch timeout; set your own AbortController per call so a slow review service doesn't burn your 50ms CPU wall. For patterns on staying inside those walls, I've found Edge Function Memory Management: Working Within Tight Limits at CDN Scale essential when you're buffering multiple JSON bodies at once.

You also need to think about headers and caching semantics. Don't blindly merge Cache-Control from backends. The aggregated response needs its own cache policy that reflects the lowest TTL among critical sources, with stale-while-revalidate handling the degradable ones. We also tag the aggregated JSON with a versioned surrogate key so we can purge by product ID without purging the whole edge cache. The guidance in web.dev Articles (https://web.dev/articles/) on fetch priority and caching was surprisingly useful when we tuned this for Chrome's back-forward cache behavior.

// Partial failure with per-fetch timeouts and allSettled
async function fetchWithTimeout(url, ms = 800) {
  const controller = new AbortController();
  const t = setTimeout(() => controller.abort(), ms);
  try {
    const res = await fetch(url, { signal: controller.signal });
    return res.ok ? await res.json() : null;
  } catch {
    return null; // treat timeout / abort as degradable failure
  } finally {
    clearTimeout(t);
  }
}

export async function aggregateProduct(productId) {
  const results = await Promise.allSettled([
    fetchWithTimeout(`https://api.internal/prices/${productId}`, 600),      // critical
    fetchWithTimeout(`https://api.internal/inventory/${productId}`, 600),   // critical
    fetchWithTimeout(`https://api.internal/reviews/${productId}`, 1000),    // degradable
    fetchWithTimeout(`https://api.internal/reco/${productId}`, 900)         // degradable
  ]);

  const [price, inventory, reviews, reco] = results.map(r => r.value ?? null);

  if (!price || !inventory) {
    throw new Error('critical_data_missing');
  }

  return {
    id: productId,
    price,
    inventory,
    reviews: reviews ?? [],
    reco: reco ?? [],
    _meta: {
      degraded: !reviews || !reco,
      servedAt: Date.now()
    }
  };
}

What We Actually Measured After Moving Aggregation to the Edge

We A/B tested client-side fan-out versus edge aggregation on 12% of traffic for two weeks - same backends, same data, just different orchestration point. The edge path ran on Cloudflare Workers with the PoP hitting our origin over Argo. No other changes. The numbers below are p50/p95 from our RUM dataset (approx. 4.2M page views) for the PDP on mobile, not synthetic lab tests.

The biggest surprise wasn't the latency drop, it was the origin load reduction. Browsers were retrying failed individual calls and often requesting the same data twice due to component-level fetching. The edge coalesces that into one deduplicated set of subrequests and then caches the combined JSON for a few seconds. That short TTL - we use 5 seconds with 30 seconds stale-while-revalidate - absorbs thundering herds during flash sales without serving stale prices long enough to matter. You can tune those values per RFC 9111 semantics as described by the HTTP Working Group Specifications (https://httpwg.org/specs/).

Building the Aggregator: Parallel Fetch with Hard Timeouts

Look, the naive implementation is just a Promise.all() that fans out to three or four origins and waits. That works in a demo, but it’ll kill you in production. If one backend hangs for 4 seconds, your entire edge function hangs for 4 seconds — and you’re paying for that CPU time on every single request. We learned early to never use Promise.all() for aggregation. Use Promise.allSettled() with per-fetch timeouts and a global budget. Our rule of thumb: 800ms per-backend timeout, 1,200ms total wall-clock budget for the aggregator. If a backend can’t answer in that window, it doesn’t deserve to block the page.

You’ll also want to be explicit about cacheability, because an aggregated response is only as cacheable as its least-cacheable part. Don’t cache the final JSON for 5 minutes if the pricing service says max-age=10. We tag each sub-response with its own cache policy and compute the final TTL as the minimum of the set. On Cloudflare and Fastly, we also set a Surrogate-Key or Cache-Tag per backing service so we can purge just the product fragment without nuking the whole aggregated response. Get this wrong and you’ll either serve stale prices for minutes or get a 15% hit ratio when you should be hitting 85-92%.

// Cloudflare Workers - edge aggregator with budgets and graceful degradation
export default {
  async fetch(req, env) {
    const url = new URL(req.url);
    const controller = new AbortController();
    const globalTimeout = setTimeout(() => controller.abort(), 1200);

    const withTimeout = (promise, ms = 800) => 
      Promise.race([
        promise,
        new Promise((_, reject) => setTimeout(() => reject(new Error('timeout')), ms))
      ]);

    const endpoints = {
      product: `https://api.origin.com/products/${url.searchParams.get('id')}`,
      inventory: `https://api.origin.com/inventory/${url.searchParams.get('id')}`,
      reviews: `https://api.origin.com/reviews/${url.searchParams.get('id')}?limit=5`
    };

    const results = await Promise.allSettled(
      Object.entries(endpoints).map(async ([key, endpoint]) => {
        const res = await withTimeout(
          fetch(endpoint, { signal: controller.signal, cf: { cacheTtl: 10 } })
        );
        if (!res.ok) throw new Error(`${key} failed: ${res.status}`);
        return [key, await res.json()];
      })
    );

    clearTimeout(globalTimeout);

    const data = {};
    const errors = {};
    let minTtl = 60;

    for (const r of results) {
      if (r.status === 'fulfilled') {
        const [key, value] = r.value;
        data[key] = value;
        // In real code, parse Cache-Control from sub-response to compute minTtl
      } else {
        errors[r.reason.message.split(' ')[0]] = 'degraded';
      }
    }

    // Never fail the whole response if reviews/inventory are down
    if (!data.product) {
      return new Response(JSON.stringify({ error: 'product unavailable' }), { status: 503 });
    }

    return new Response(JSON.stringify({ data, _degraded: errors }), {
      headers: {
        'Content-Type': 'application/json',
        'Cache-Control': `public, max-age=${minTtl}, stale-while-revalidate=60`,
        'Cache-Tag': 'agg-product,agg-inventory,agg-reviews'
      }
    });
  }
}

Advanced Patterns and Production Gotchas: Collapsing, Circuits, and Cold Starts

The first pattern everyone needs but nobody ships on day one is request collapsing (also called coalescing). Without it, when your aggregated response expires, you’ll get 400 concurrent edge requests all fanning out to the same three origins at once — a classic thundering herd that will instantly spike your origin p95 from 60ms to 2.5s. Enable collapsed forwarding at the edge if your platform supports it, or implement a simple in-memory promise cache inside the isolate. Combined with stale-while-revalidate, we kept origin QPS flat during a purge event that would’ve otherwise tripled load. If you’re seeing collapsed fetch errors in your logs and can’t tell which sub-fetch is lagging, you’ll want better observability — see Edge Function Debugging Techniques: Console, Tail Workers, and Remote Inspection for how we trace individual fan-out spans without flooding logs.

The second gotcha is failing open. We put a lightweight circuit breaker around each backend, not just a timeout. If the recommendations API has thrown 50% errors over the last 30 seconds, we stop calling it for the next 10 seconds and instantly return a degraded stub. That saved us during a regional outage last quarter — the aggregator still served 99.4% of requests in ~210ms with a missing recommendations block, instead of failing 100% of aggregated requests. You’ve got to test this behavior explicitly, not just hope it works. We run contract tests against mocked 500s, timeouts, and malformed JSON in CI before any deploy, exactly as outlined in Edge Function Testing Strategies: Unit, Integration, and End-to-End at the Edge.

Finally, treat your aggregator like a real service with versioning. Don’t just wrangler publish on top of production. We ship aggregator changes behind a canary — 5% of traffic on the new code for 10 minutes, comparing error rates and median fan-out latency before promoting. If the new build introduces a JSON parsing regression, you don’t want 100% of your product pages returning broken payloads. A gradual rollout with instant rollback is non-negotiable, and the patterns in Edge Function Versioning and Deployment Patterns: Blue-Green, Canary, and Gradual map perfectly to aggregated endpoints.

Frequently Asked Questions