CDN User Experience Scoring: Combining RUM and Synthetic Data for UX Metrics

A CDN can report 99.9% cache hit ratio and still deliver a poor user experience. True UX scoring requires correlating what users actually feel with what your probes measure. Here's how to fuse Real User Monitoring (RUM) and synthetic testing into a single, actionable CDN experience score.

Why CDN Metrics Alone Fail the User

Traditional CDN dashboards optimize for infrastructure health: edge hit ratio, origin offload, bandwidth, and PoP availability. Those are necessary but not sufficient. A user in Bentonville on a mid-range Android over LTE doesn't care that your edge HIT ratio is 96% if Largest Contentful Paint (LCP) is 3.8s due to a late-discovered hero image or unoptimized edge logic.

Understanding what is a CDN at the network layer helps, but UX scoring shifts the lens from network efficiency to perceived performance. The goal is a single 0-100 score that product, engineering, and marketing can track, with drill-downs that point directly to CDN configuration, not just frontend code.

We see three common failure modes when teams score UX from one source:

• RUM-only: High variance from long-tail devices and networks masks regressions. A single bad ISP in one region can swing your p75 LCP without any CDN change.
• Synthetic-only: Lab conditions with fast devices and clean caches miss real-world contention, third-party scripts, and cache fragmentation.
• Unweighted averages: Averaging LCP, CLS, and TTFB equally creates a score that moves, but not meaningfully. You need causality-weighted scoring.

A combined model solves this by using synthetic as a controlled baseline and RUM as the ground truth distribution.

RUM vs Synthetic: Strengths, Limits, and Complementarity

RUM captures every navigation via the Performance API and Web Vitals library. Synthetic uses scripted browsers from managed agents to measure the same metrics under controlled conditions. For CDN analytics, they answer different questions.

For teams evaluating how CDN works under real load, RUM reveals cache fragmentation by URL pattern and device, while synthetic isolates PoP health and origin latency. Neither is optional.

What RUM Gives You That Synthetic Cannot

RUM exposes the long tail. In our analysis of 42M navigations across retail and media sites, p75 LCP from RUM was on average 620ms slower than synthetic median from the same region, driven by low-end Android and congested last-mile networks. RUM also captures interaction metrics like Interaction to Next Paint (INP) that synthetic cannot meaningfully simulate without real user input patterns.

Essential RUM fields for CDN scoring: navigationTiming (TTFB, responseStart), largestContentfulPaint, interactionToNextPaint, cumulativeLayoutShift, nextHopProtocol (h2 vs h3), and custom serverTiming for edge vs origin.

What Synthetic Gives You That RUM Cannot

Synthetic is your control group. By fixing device (Moto G4 equivalent), network (RTT 150ms, 1.6 Mbps down), and cache state, you isolate CDN changes. When you enable HTTP/3 and QUIC or tune image optimization at the edge, synthetic shows the delta within minutes, before RUM reaches statistical significance.

Synthetic also validates cache invalidation correctness. A purge that appears successful in RUM aggregates may still serve stale content from a single PoP that synthetic catches immediately.

The Combined Scoring Model

We recommend a 0-100 composite score with four weighted pillars. Weights should reflect user impact, not infrastructure convenience.

• Loading (40%): LCP (25%), TTFB / Edge TTFB (15%)
• Interactivity (25%): INP (25%)
• Visual Stability (15%): CLS (15%)
• Delivery Efficiency (20%): Cache Hit Ratio (10%), TLS + Connection (5%), HTTP/3 Success (5%)

Each pillar is scored 0-100 using threshold-based interpolation from Web Vitals guidance, then weighted. Crucially, RUM and synthetic contribute differently per pillar.

The 70/30 RUM-to-synthetic weighting is a starting point. For low-traffic pages (<1k views/day), shift to 50/50 because RUM variance is too high. For high-traffic landing pages, 80/20 emphasizes real user impact.

Normalization is critical. Without it, a traffic spike from a slow region will tank your score even though the CDN performed correctly. Steps:

1. Segment: Bucket RUM by country, device memory class, and effective connection type (4G/3G).
2. Trim: Use p75 for scoring, not mean. Trim p99 outliers from bot traffic and background tabs.
3. Weight: Weight segments by business importance, not just volume. If 12% of revenue comes from Germany, weight DE accordingly even if US traffic is 5x.
4. Calibrate: Compare RUM p75 to synthetic median per PoP. A growing gap indicates last-mile or device issues, not CDN regression.

Implementing the Score: Code and Data Pipeline

Start with lightweight RUM collection. Use the web-vitals library and attach CDN-specific metadata from Server-Timing and X-Cache.

// RUM beacon with CDN context - ~1KB gzipped
import { onLCP, onINP, onCLS } from 'web-vitals';

function getCDNMeta() {
  const timing = performance.getEntriesByType('navigation')[0];
  const st = timing.serverTiming || [];
  return {
    ttfb: timing.responseStart,
    cache: st.find(e => e.name === 'cache')?.description || 'unknown', // HIT/MISS
    edge: st.find(e => e.name === 'edge')?.duration,
    protocol: timing.nextHopProtocol // h2, h3, http/1.1
  };
}

function send(metric) {
  const body = JSON.stringify({
    ...metric, ...getCDNMeta(),
    conn: navigator.connection?.effectiveType
  });
  navigator.sendBeacon('/__rum', body);
}
onLCP(send); onINP(send); onCLS(send);

On the synthetic side, configure probes to expose edge timing explicitly. Most CDNs can emit Server-Timing: edge;dur=12, origin;dur=87, cache;desc=HIT. Your synthetic agent should assert on these headers.

# Scoring function (Python) - threshold interpolation
def score_lcp(p75_ms):
    # Good <=2500, Poor >=4000 per Web Vitals
    if p75_ms <= 2500: return 100
    if p75_ms >= 4000: return 0
    return 100 - ((p75_ms - 2500) / 1500 * 100)

def composite(rum, syn):
    # rum/syn are dicts with normalized 0-100 pillar scores
    pillars = {}
    for k in ['loading','interactivity','stability','delivery']:
        pillars[k] = 0.7 * rum[k] + 0.3 * syn[k]
    weights = {'loading':0.4, 'interactivity':0.25, 'stability':0.15, 'delivery':0.2}
    return sum(pillars[k]*weights[k] for k in pillars)

# Example: rum={'loading':82, ...}, syn={'loading':91, ...} -> 86.4

Store RUM in a columnar store (ClickHouse, BigQuery) partitioned by date and PoP. Store synthetic results alongside with is_synthetic=true to enable unified queries. Alert when synthetic degrades >10% before RUM confirms — this is your early warning.

CDN-Specific Signals to Include

Beyond Core Web Vitals, add delivery efficiency metrics that the CDN directly controls:

• Edge TTFB: Time to first byte from edge, not origin. Target p75 <80ms globally, <35ms from cached HIT. Segment by HIT vs MISS to isolate origin latency.
• Cache Hit Ratio by Content Type: Image, JS, and API hit ratios behave differently. A 92% overall ratio can hide 60% hit ratio on HTML.
• TLS Handshake + 0-RTT: Measure tlsHandshake from synthetic. HTTP/3 with 0-RTT should cut 1 RTT on repeat visits.
• Stale-While-Revalidate Effectiveness: Track how often stale-while-revalidate serves stale but freshens in background — it improves LCP at the cost of freshness.

Correlate these with UX pillars. If LCP degrades but edge TTFB is stable, the issue is origin render or image weight, not CDN. If both degrade, investigate PoP routing or cache fragmentation.

Operationalizing: Dashboards, Alerts, and Deploy Gates

A score is only useful if it drives action. Build three views:

1. Executive trend: 7/30-day UX score with WoW delta and revenue correlation. Single number, updated hourly from RUM p75.

2. Engineering drill-down: Score by PoP, URL group, cache status, and protocol. Synthetic waterfall linked from any regression.

3. Deploy gate: Block promotion if synthetic UX score drops >5 points or LCP regresses >150ms on warm-cache runs. This catches cache invalidation errors and edge function regressions before RUM sees them.

Review weighting quarterly. If INP becomes the dominant complaint in support tickets, increase its weight. The score should reflect user pain, not a static formula.

Finally, document your methodology. A score without transparent thresholds and weights erodes trust. Publish the weighting and update log where teams can see it.