CDN Catalog Syndication: Delivering Product Data Across Multiple Sales Channels

Back in 2019 I watched a mid-size retailer with about 80,000 SKUs try to syndicate their catalog directly from their Magento origin to nine different sales channels. Every time they ran a price update, their origin would get hammered by simultaneous feed pulls from Amazon, eBay, Google, plus their own mobile apps. The site went down for 40 minutes during a Black Friday price sync. That's when we moved their entire product data distribution to the edge, and I've done it the same way ever since.

Why Your Origin Shouldn't Be Your Syndication Hub

When you're selling on multiple channels, your product data isn't just sitting on your website anymore. That same SKU, with its images, pricing, inventory, and variants, needs to live on Amazon Marketplace, Instagram Shopping, Google Merchant Center, and maybe a dozen retail partners. If each of those channels is polling your origin database or your CMS API every 15 minutes, you're not doing catalog syndication — you're running a self-inflicted DDoS.

I've seen teams try to solve this by throwing more origin capacity at it or by setting up a separate feed server. It doesn't scale. What does scale is treating your product catalog as a distributable asset and pushing it to your CDN. Your CDN isn't just for images and product pages anymore. Modern edge networks are perfectly suited for distributing structured product data with the same cache hierarchy, invalidation, and transformation logic you'd use for any other content. You're essentially creating a single source of truth at the edge that all channels consume, instead of letting them all fight over your origin.

This approach also solves the consistency problem. Without a centralized distribution layer, Channel A might pull a feed at 10:02 with old pricing while Channel B pulls at 10:05 with new pricing. Customers see different prices on different platforms, and your support team gets the fallout. By syndicating through the CDN, you control the version and the timing. For high-traffic storefronts, this edges into the same performance territory as CDN Product Page Optimization: Sub-Second Loading for Higher Conversion Rates, where every millisecond of data freshness impacts conversion and trust.

// Edge worker routing channel-specific catalog feeds
// Deployed to all edge PoPs - serves transformed feeds from cache

export default {
  async fetch(request, env) {
    const url = new URL(request.url);
    const channel = url.searchParams.get('channel') || 'default';
    
    // Canonical product data cached at the edge - 1 source of truth
    const canonicalKey = 'https://api.origin.example.com/v2/catalog/canonical.json';
    
    let response = await caches.default.match(canonicalKey);
    if (!response) {
      response = await fetch(canonicalKey, {
        cf: { cacheTtl: 300, cacheEverything: true }
      });
      // Stale-while-revalidate per HTTP spec - serve stale while updating
      response = new Response(response.body, response);
      response.headers.set('Cache-Control', 'public, max-age=300, stale-while-revalidate=60');
      response.headers.set('Surrogate-Key', 'catalog-canonical');
      await caches.default.put(canonicalKey, response.clone());
    }
    
    // Transform at edge for channel-specific requirements
    return transformForChannel(response, channel);
  }
}

Structuring Product Data for Multi-Channel CDN Distribution

Here's where most teams get tripped up: they try to create a separate feed file for every channel and push them all to the CDN as static files. That works for three channels, but it becomes a nightmare when you're managing 15 marketplaces that each want slightly different attribute names, image sizes, and taxonomies. I don't do it that way anymore. Instead, we maintain one canonical JSON feed and do the transformation at the edge.

Your canonical feed should be complete, clean, and channel-agnostic. Think of it as your master record — every field your product could need, normalized to your internal schema. We're talking GTINs, MPNs, variant matrices, inventory by warehouse, rich media URLs, and localized descriptions. Store this as a single JSON object or a paginated JSON feed on origin, then cache it aggressively at the edge with surrogate keys. The channel-specific logic happens in an edge worker or through edge compute that maps your canonical fields to whatever Amazon or Google expects. For example, Google wants `google_product_category` while Amazon wants `product_type` — that's a simple mapping function, not a reason to maintain two separate files.

The performance win here is huge. Instead of invalidating and regenerating 15 different feeds when a price changes, you purge one surrogate key and let the edge rebuild the variants on the next request. This pattern aligns closely with how we handle CDN Cart Session Management: Maintaining Shopping State Across Edge Nodes — keep state and logic at the edge, keep your origin simple. And don't forget to set proper HTTP cache headers as defined in the HTTP Working Group Specifications and follow caching best practices from web.dev Articles; your downstream channels will respect them if you send them.

{
  "canonical_product": {
    "id": "SKU-88421",
    "surrogate_key": "product-88421",
    "title": "Merino Wool Crewneck - Charcoal",
    "brand": "North & Ridge",
    "gtin": "00884391234567",
    "price": { "amount": 128.00, "currency": "USD" },
    "inventory": { "total": 42, "warehouses": {"us-east": 30, "eu-west": 12} },
    "media": [
      {"url": "https://cdn.example.com/p/88421/main.jpg", "role": "primary", "width": 2000},
      {"url": "https://cdn.example.com/p/88421/alt1.jpg", "role": "alternate"}
    ],
    "variants": [
      {"id": "SKU-88421-S", "size": "S", "inventory": 10},
      {"id": "SKU-88421-M", "size": "M", "inventory": 32}
    ]
  },
  "_edge_transform_example": "map canonical.price.amount -> google: price, amazon: standard_price"
}

Cache Invalidation and Freshness Without the Chaos

Caching product data is easy. Invalidating it correctly across multiple sales channels without breaking things is where experience matters. You can't just set a 24-hour TTL and hope prices don't change. You also can't set a 60-second TTL and expect your origin to survive. The sweet spot I've found after years of tuning this is a layered strategy: short TTLs (5 minutes) with stale-while-revalidate, plus precise surrogate key purging for critical updates.

Here's how it works in practice. We tag every product and every feed with Surrogate-Key headers — for example, `product-88421`, `category-apparel`, `brand-north-ridge`. When inventory drops to zero for a single SKU, we issue a purge request for just `product-88421`. The CDN purges that object from all PoPs in under 150ms (on Fastly or Cloudflare Enterprise), and the next request for any channel's feed that includes that SKU will trigger a revalidation to origin. For bulk price updates, we purge `catalog-canonical` once. The edge workers then regenerate all channel feeds automatically. This is far more reliable than time-based expiry, especially when your checkout flow depends on accurate inventory — a problem we also address in CDN Checkout Flow Acceleration: Edge Optimization for the Revenue-Critical Path.

Don't rely on manual purges. Automate them from your PIM or ERP webhook. When a product manager hits publish in your Akeneo or Salsify, that webhook should call the CDN purge API instantly. We also add a `stale-while-revalidate=60` directive so that if a channel requests a feed during the 200ms it takes to refetch from origin, they get the slightly stale version instantly instead of waiting. Users never see a delay, channels never get a 500.

Putting It Into Practice: Channel-Aware Transforms at the Edge

You don't want your origin building 17 different feed formats every 15 minutes. The pattern that actually scales is what we call canonical-plus-transform: your PIM pushes one canonical JSON catalog to object storage (R2, S3, GCS), the CDN pulls it once, and edge workers transform it per-channel on demand. You'll cut origin egress by 90%+ and you finally stop waking up because Wayfair's validator choked on a missing attribute.

In our last rollout for a 2.4M SKU retailer, we stored the canonical catalog as gzipped NDJSON sharded by category (about 180MB compressed total), fronted by the CDN with a 300s TTL and stale-while-revalidate=86400. Each channel feed — Google Shopping, Amazon Marketplace, Instagram Commerce — is just a Worker route like /feeds/google.xml that fetches the canonical shard, maps fields, and caches the result separately. Cache hit ratio went from 62% (when the origin was rendering feeds) to 98.7%, and p95 time-to-first-byte for a full 50k-item feed dropped from 1.8s to 47ms. Origin traffic fell from ~4,200 RPS during peak syndication windows to under 180 RPS.

The trick is surrogate keys and layered caching. Tag every cached feed response with Surrogate-Key: catalog cat:electronics channel:google. When a price or stock update comes in, you don't purge everything — you purge just cat:electronics and it invalidates 3 channel variants at once in under 150ms. For stock specifically, don't bundle it into the catalog TTL. Keep inventory as a separate edge lookup with a 10-20s TTL as we cover in CDN Inventory API Caching: Real-Time Stock Visibility with Aggressive TTLs — that way your catalog can stay warm for 5 minutes while availability stays near-real-time.

// Cloudflare Worker: canonical -> channel transform with surrogate keys
export default {
  async fetch(req, env) {
    const url = new URL(req.url);
    // e.g., /feeds/google.xml?category=electronics
    const category = url.searchParams.get('category') || 'all';
    const channel = url.pathname.split('/')[2]; // google, amazon, meta

    const canonicalKey = `https://catalog-origin.example.com/shards/${category}.json.gz`;
    
    // Tiered cache: check edge first, 5 min TTL + 24h SWR
    let res = await fetch(canonicalKey, { cf: { cacheTtl: 300, cacheEverything: true } });
    if (!res.ok) return new Response('Shard miss', { status: 502 });

    const products = await res.json(); // decompressed automatically at edge
    const transformed = transformForChannel(products, channel);

    return new Response(transformed.body, {
      headers: {
        'Content-Type': transformed.contentType,
        'Cache-Control': 'public, max-age=300, stale-while-revalidate=86400',
        'Surrogate-Key': `catalog cat:${category} channel:${channel}`,
        'CDN-Cache-Tag': `catalog cat:${category} channel:${channel}`
      }
    });
  }
}

function transformForChannel(products, channel) {
  if (channel === 'google') {
    // Map to Google Shopping RSS 2.0 + g: namespace
    return { contentType: 'application/xml', body: toGoogleFeed(products) };
  }
  // amazon, meta, etc. - strict schema per channel
  return { contentType: 'application/json', body: JSON.stringify(products) };
}

Advanced Patterns: When Feeds Break, Drift, and Poison Your Cache

If you've syndicated long enough, you know the failure mode isn't your CDN — it's schema drift. Google adds a required g:availability_date for backorders, Amazon starts rejecting listings without material_composition, and suddenly your perfectly cached feed is serving 100% invalid XML for 5 minutes. Don't let a poisoned cache amplify the problem. We always put a lightweight validator Worker in front of the cache write: if the transformed output fails XSD/JSON Schema validation (we run a compiled AJV schema at the edge in <12ms), we serve stale and alert instead of overwriting the good cache. It's saved us at least three full delistings this year alone.

You'll also hit partial freshness issues. Price and catalog metadata shouldn't share the same cache lifecycle. I see teams invalidate their entire 500k-item feed because one SKU's price changed 2% — that's wasteful. Instead, compose responses at the edge: cache the mostly-static catalog (title, images, attributes) for 10-15 minutes, then overlay price/stock via an edge dictionary or Workers KV lookup with a 5-15s TTL. This is the same edge-composition pattern we use for Price Personalization at CDN Edge: Dynamic Pricing Without Origin Round-Trips. You get personalized or rapidly changing pricing without blowing your 95%+ hit ratio, and a purge for a price change touches a 2KB KV key, not a 40MB feed.

Finally, watch your purge limits. Most CDNs rate-limit you to ~2,000-5,000 purge tags per second, and Fastly/Cloudflare will queue surges. If you're bulk-updating 80k SKUs after an ERP sync, don't fire 80k single-key purges. Batch by surrogate key prefix (cat:*) or use soft purges with stale-while-revalidate so the next request revalidates in background. And if you're experimenting with channel-specific titles or images, isolate that logic with edge flags instead of forking feeds — see A/B Testing Product Pages at CDN Edge: Experimenting on Layout and Content for a clean way to test without duplicating your syndication pipeline.

Frequently Asked Questions