Event-Driven Patterns in Edge Functions: Webhooks, Queues, and Pub/Sub at CDN

Last Black Friday, a client's payment webhook provider retried 40,000 failed delivery attempts in under three minutes. Their origin was already saturated, so every retry just made the outage worse. We'd moved their webhook ingestion to the edge two weeks earlier, and that was the only reason their order queue survived — the CDN PoPs absorbed the burst, de-duplicated the events, and drained them slowly to the origin once it recovered. If you're still pointing webhooks directly at a single endpoint, you're one traffic spike away from the same meltdown.

When Webhooks Hit the Edge: Reliable Ingestion Without an Origin

Most teams treat webhooks like a simple HTTP POST handler. You validate a signature, parse JSON, write to a database, and return 200. That works fine in a single region until you’ve got a provider like Stripe or Shopify firing thousands of events per second from globally distributed sources. The round-trip to your origin adds 80-150ms of latency, and any 500 or timeout triggers exponential retries that amplify failures. Moving ingestion to an Edge Function Cold Start Optimization: Reducing Initialization Latency to Milliseconds pattern changes the math completely — you’re acknowledging events within 10-20ms at the PoP closest to the sender, not in us-east-1.

I've learned the hard way that idempotency isn't optional at the edge. We had a customer processing inventory updates via webhooks where a carrier's system sent the same "shipped" event 14 times over 8 minutes because our edge function was returning 200 before actually persisting. The function looked successful, but we'd dropped the write due to a downstream timeout. Now I always design edge webhook handlers around three guarantees: verify before you acknowledge, persist before you respond, and never trust a single delivery. HTTP semantics matter here too — the HTTP Working Group Specifications are pretty clear that a 2xx should mean "I own this now," not "I saw it." If you can’t guarantee ownership in under 50ms, return 202 and queue it.

That queue step is where event-driven edge patterns start to pay off. Instead of doing heavy lifting inside the webhook handler, you push a validated, normalized event into an edge queue and let a separate consumer handle enrichment or fan-out. You stay well under the tight CPU limits that way, and you decouple ingestion from processing so a slow database doesn’t become a bottleneck for acknowledgments. We typically keep the webhook handler under 15ms of CPU and 3ms of wall time for validation.

// Edge webhook ingestion with HMAC verification and queue forwarding
export default {
  async fetch(request, env, ctx) {
    if (request.method !== 'POST') return new Response('Method Not Allowed', { status: 405 });

    const signature = request.headers.get('X-Webhook-Signature');
    const rawBody = await request.text();
    
    // Constant-time signature check - don't skip this at the edge
    const valid = await verifyHMAC(rawBody, signature, env.WEBHOOK_SECRET);
    if (!valid) return new Response('Invalid signature', { status: 401 });

    const event = JSON.parse(rawBody);
    const dedupeKey = `wh:${event.id}:${event.type}`;

    // Idempotent write with 24h TTL - critical for retry storms
    const isNew = await env.IDEMPOTENCY_KV.put(dedupeKey, '1', { 
      expirationTtl: 86400, 
      nx: true 
    });
    if (isNew === null) return new Response('Already processed', { status: 200 });

    // Enqueue for async processing, don't block on origin
    ctx.waitUntil(env.EDGE_QUEUE.send({
      type: event.type,
      payload: event.data,
      receivedAt: Date.now(),
      pop: request.cf.colo
    }));

    return new Response(JSON.stringify({ received: true }), {
      status: 202,
      headers: { 'Content-Type': 'application/json' }
    });
  }
}

Building Edge Queues That Don't Lose Messages at 2 AM

An edge queue isn't just a Cloudflare Queue or an Upstash Redis list with a different name. The constraints are fundamentally different from a central SQS queue. Your producer and consumer are running in 300+ PoPs with no shared memory, clocks that aren't perfectly synced, and Edge Function Memory Management: Working Within Tight Limits at CDN Scale that force you to process in tiny batches. I once tried to batch 1,000 messages per invocation to improve throughput — the function OOM'd at 128MB and we lost the entire batch because we'd already acknowledged it to the queue. Lesson learned: small, fast, and durable beats big and clever every time.

The pattern that’s worked best for us is a two-tier queue: a durable edge-native queue for ingestion and a central ordered queue for exactly-once processing. The edge tier accepts writes locally with eventual replication, which gives you sub-10ms enqueue latency and survives regional partitions. The central tier then pulls from all PoPs, reorders by timestamp, and handles deduplication. You’ll need to design for at-least-once delivery — retries will happen, and your consumer must be idempotent. I've found that adding a dead-letter queue with exponential backoff (1s, 5s, 25s, 125s) catches 99% of transient failures without hammering a struggling downstream. The web.dev Articles on reliable background processing map surprisingly well to edge queue consumers, even though they were written for service workers.

Don't underestimate Edge Function Execution Limits: CPU Time, Subrequests, and Creative Workarounds here. Most edge runtimes cap you at 50ms CPU time and 6-20 subrequests per invocation. That means your queue consumer can’t make 15 API calls to enrich an event. We batch aggressively — 10-25 messages per invocation, one bulk write to storage, one fetch to the origin if needed — and we use streaming to avoid buffering everything in memory. If you need more compute, chain consumers: Consumer A does validation and enrichment, pushes to Queue B, and Consumer B does the heavy API work.

// Batched edge queue consumer with idempotent processing and DLQ
export async function queue(batch, env) {
  const messages = batch.messages;
  const toRetry = [];
  const toDLQ = [];

  // Group by type to allow bulk operations
  const byType = Map.groupBy(messages, m => m.body.type);
  
  for (const [type, msgs] of byType) {
    try {
      // Single bulk idempotency check per batch
      const keys = msgs.map(m => `evt:${m.body.payload.id}`);
      const existing = await env.STATE_KV.get(keys);

      const newMsgs = msgs.filter((_, i) => !existing[i]);
      if (newMsgs.length === 0) {
        batch.ackAll();
        continue;
      }

      await processBatch(type, newMsgs, env);

      // Mark processed atomically
      await env.STATE_KV.mput(
        newMsgs.map(m => [`evt:${m.body.payload.id}`, '1', { expirationTtl: 259200 }])
      );
    } catch (err) {
      for (const msg of msgs) {
        if (msg.attempts < 4) {
          toRetry.push(msg);
        } else {
          toDLQ.push({ ...msg.body, error: err.message, failedAt: Date.now() });
        }
      }
    }
  }

  if (toDLQ.length) await env.DEAD_LETTER_QUEUE.sendBatch(toDLQ);
  if (toRetry.length) batch.retryAll({ delaySeconds: Math.pow(5, batch.messages[0].attempts) });
}

async function processBatch(type, msgs, env) {
  // One subrequest per batch, not per message
  if (type === 'order.created') {
    await env.ORIGIN_API.bulkInsert('orders', msgs.map(m => m.body.payload));
  }
}

Pub/Sub at the CDN: Fanning Out to 300 PoPs in Under 50ms

Pub/sub is where edge eventing gets really interesting — and where most teams get burned by consistency assumptions. At the edge, you're not publishing to a single broker; you're publishing to a distributed mesh that has to converge. We use pub/sub for things like cache invalidation, feature flag propagation, and real-time config updates where waiting 500ms for origin polling isn't acceptable. The publisher fires once at any PoP, and subscribers in every other PoP react within tens of milliseconds. I remember pushing a price change for a flash sale and watching it propagate to 280 PoPs in 38ms — that would have taken 2-3 seconds with origin polling.

The trade-off is that you can't have both low latency and strong ordering across all PoPs without paying a huge coordination cost. For cache purges, last-write-wins is fine — you just need the latest version to win eventually. For inventory decrements, you need stronger guarantees and you shouldn't use edge pub/sub alone. We classify topics into two buckets: ephemeral state (purges, flags, presence) where we accept eventual consistency and at-most-once delivery, and durable events (orders, payments) where we need at-least-once and replay. The durable topics go through the queue system described earlier; the ephemeral ones use a lightweight pub/sub layer with no persistence. Keeping that separation prevents you from trying to build a database on top of a broadcast channel.

Choosing the right primitive depends entirely on your delivery and latency requirements. I've seen teams try to use webhooks for fan-out to thousands of edge clients — it falls apart after about 50 subscribers due to retry amplification. Conversely, using pub/sub for critical payment events without persistence is asking for silent data loss. Here's how we decide which pattern to reach for in production, based on 18 months of running all three at scale across a high-traffic retail CDN:

Putting It All Together: A Resilient Edge Event Pipeline in Code

We've talked theory — ingestion that's idempotent, queues that don't drop messages when a PoP goes dark, and pub/sub that actually hits 300+ PoPs in under 50ms. Now let's wire it up the way we do in production. The trick isn't any single pattern, it's making 'em play nicely when you're handling 45k webhook events per second and you can't afford to lose one because a container recycled at 2:07 AM.

In a real pipeline, your edge function does three things in under 15ms: validates, deduplicates, and enqueues. Don't do heavy processing inline. We learned this the hard way — a 250ms image transform inside the webhook handler took our p95 from 18ms to 340ms and triggered upstream retries that doubled our load. Instead, verify the HMAC signature, check an idempotency key against a globally replicated KV with a 24-hour TTL, and push to a durable queue. That's it. Return 200. Everything else happens asynchronously.

The code below is a stripped-down version of what we run on Cloudflare Workers with Queues and Durable Objects, but the pattern ports directly to Fastly Compute + Fanout or AWS Lambda@Edge + SQS. Notice we're setting a 5-second visibility timeout and explicitly handling the dedupe check before we ever touch the queue. You'll thank yourself later when Stripe resends that same `invoice.paid` event four times in 30 seconds.

export default {
  async fetch(request, env, ctx) {
    if (request.method !== 'POST') return new Response('Method Not Allowed', { status: 405 });

    // 1. Verify webhook signature (constant-time compare)
    const signature = request.headers.get('x-webhook-signature');
    const rawBody = await request.text();
    if (!(await verifyHMAC(rawBody, signature, env.WEBHOOK_SECRET))) {
      return new Response('Invalid signature', { status: 401 });
    }

    const event = JSON.parse(rawBody);
    const idempotencyKey = `evt:${event.id}`; // Stripe, GitHub, etc. provide this

    // 2. Deduplicate at the edge - 24h window
    const seen = await env.IDEMPOTENCY_KV.get(idempotencyKey);
    if (seen) return new Response('Already processed', { status: 200 });

    // 3. Enqueue for async processing - don't await, use waitUntil
    ctx.waitUntil((async () => {
      await env.EVENT_QUEUE.send({
        body: event,
        contentType: 'json',
        delaySeconds: 0
      });
      await env.IDEMPOTENCY_KV.put(idempotencyKey, '1', { expirationTtl: 86400 });
    })());

    return new Response(JSON.stringify({ received: true }), {
      status: 200,
      headers: { 'content-type': 'application/json' }
    });
  },

  // Queue consumer with batching + DLQ handling
  async queue(batch, env) {
    for (const msg of batch.messages) {
      try {
        await fanOutToSubscribers(msg.body, env); // pub/sub to 300 PoPs
        msg.ack();
      } catch (err) {
        if (msg.attempts > 3) msg.retry({ delaySeconds: 300 }); // backoff
        else msg.sendToDLQ();
      }
    }
  }
};

async function verifyHMAC(payload, sig, secret) {
  const key = await crypto.subtle.importKey('raw', new TextEncoder().encode(secret), { name: 'HMAC', hash: 'SHA-256' }, false, ['sign']);
  const digest = await crypto.subtle.sign('HMAC', key, new TextEncoder().encode(payload));
  const expected = Array.from(new Uint8Array(digest)).map(b => b.toString(16).padStart(2,'0')).join('');
  return expected === sig;
}

Advanced Patterns and Troubleshooting When You're Fanning Out Globally

Once you're past the happy path, you'll hit the weird stuff. Duplicate delivery is the big one. Even with exactly-once queues (spoiler: they don't exist), you'll see 0.3-0.8% duplicates under load due to ack timeouts and PoP handoffs. Your consumers *must* be idempotent. We tag every fan-out message with a `deliveryId` and a 10-minute dedupe window in a Durable Object. Without it, we once double-credited a customer's usage metrics 12k times during a retry storm. And if you're seeing poison messages clogging your queue — that one malformed JSON payload that fails parsing every time — don't let it block the whole batch. Isolate it to a dead-letter queue after 3 attempts and alert, don't retry forever. That's how you end up with 2 AM pages.

The other killer is thundering-herd fan-out. Publishing one event to 15,000 subscribers from the edge sounds great until you try to open 15k WebSocket connections from a single isolate with a 128MB limit. We use coalescing and filtered topics now. Instead of fanning out raw, we push to regional aggregators — 8-12 hubs globally — and let them handle last-mile delivery. We also push filtering to the edge: if a subscriber only wants `order.created` for `region=eu`, don't send 'em `order.updated` for `region=us`. That cut our egress by 62% and dropped p99 fan-out latency from 87ms to 41ms. For stateful coordination like ordered delivery per user ID, we pin to a Durable Object or equivalent — otherwise you'll chase out-of-order events forever.

When things *do* break — and they will — you need visibility beyond `console.log`. Tail workers and live log streaming are non-negotiable for reproducing that intermittent 500 that only happens in SIN and GRU. Check out our guide on Edge Function Debugging Techniques: Console, Tail Workers, and Remote Inspection for setting up sampled tracing without blowing your log budget. And please, don't test this stuff directly in prod. We've got a full rundown of Edge Function Testing Strategies: Unit, Integration, and End-to-End at the Edge that covers local queue emulation and contract testing for webhooks. When you finally ship the fix, roll it with Edge Function Versioning and Deployment Patterns: Blue-Green, Canary, and Gradual — we canary to 5% of PoPs for 20 minutes before going global, which has saved us from at least three bad deploys this quarter alone.

Frequently Asked Questions