Video Quality Metrics for CDN: VMAF, SSIM, and PSNR in Practice

Last winter we pushed a new encoding ladder for a sports customer without checking VMAF deltas properly, and the support tickets flooded in within hours — not because the bitrate was too low, but because fast motion on the ice was turning into a blocky mess that PSNR said was "fine." If you've spent any time tuning encoding quality for CDN delivery, you've hit this exact disconnect between what your video quality metrics report and what your viewers actually complain about. I've learned the hard way that you can't pick a single number to judge quality, and you certainly can't trust one metric across every content type.

Why PSNR Still Deserves a Spot in Your Pipeline

I've heard plenty of engineers dismiss PSNR as outdated, and I get why. It doesn't match human perception very well — it'll give you nearly identical scores for two encodes where one looks noticeably worse. PSNR is just a math equation comparing the maximum possible signal power to the noise introduced by compression. It calculates mean squared error pixel-by-pixel and converts that to a decibel scale. It has no idea that your eye cares more about faces than background walls, or that a slight blur in a flat sky bothers you less than blocking on a player's jersey.

So why do I still run it on every encode? Because it's brutally honest, fast, and fantastic for catching regressions. When you're iterating on encoder settings — say testing different x264 or x265 presets before you pack for CMAF Packaging for CDN Delivery: Unified Streaming Format for HLS and DASH — PSNR will tell you immediately if you broke something fundamental. It's also deterministic and cheap to compute, which makes it perfect for automated QA in your origin workflow. I don't use it to decide if a rendition is "good enough" for viewers, but I do use it to flag when an encode is suddenly 3dB lower than last week's build. That signal has saved us more than once.

Think of PSNR as your smoke detector, not your interior designer. It won't tell you if the room looks beautiful, but it'll scream if there's a fire. For that reason, we log it for every rendition alongside more perceptual metrics. When you need to compare two encodes with identical settings and just want to know which GOP structure or deblocking filter gave you less mathematical error, PSNR is exactly the right tool. The HTTP Working Group Specifications (https://httpwg.org/specs/) won't help you tune this, but your ffmpeg logs will.

# Calculate PSNR and SSIM in one pass with ffmpeg
# Reference must be raw YUV/Y4M for accurate results - don't compare against another encode
ffmpeg -i reference.y4m -i encoded_1080p.mp4 \
  -lavfi "ssim;psnr" \
  -f null -

# Typical output you'll parse in your pipeline:
# [Parsed_psnr_1 @ 0x...] PSNR y:45.12 u:48.33 v:48.01 average:45.89 min:42.10 max:51.23
# [Parsed_ssim_0 @ 0x...] SSIM Y:0.981 U:0.985 V:0.984 All:0.982 (17.45)

SSIM and VMAF: Getting Closer to Human Eyes

SSIM was the first metric that made me feel like we weren't lying to ourselves. Instead of just measuring pixel differences, it looks at structure — luminance, contrast, and structural information across local windows. In practice, that means it penalizes artifacts that mess with edges and textures more than random noise you'd never notice. An SSIM of 0.97 and above generally looks transparent to most viewers at 1080p, while anything below 0.95 usually means you'll start getting complaints, especially on animation or text-heavy content where structure matters.

But even SSIM has blind spots. It doesn't handle temporal quality well — flickering, motion judder, the kind of stuff you see in 60fps sports — and it doesn't account for viewing conditions. That's where VMAF changed our workflow completely. Developed by Netflix and now maintained as an open model, VMAF fuses several elementary metrics with machine learning trained on human opinion scores. It gives you a 0-100 score where 93+ is generally considered excellent and viewers won't easily spot impairments on a typical TV at normal viewing distance. When we rebuilt our per-title encoding logic, switching our ladder decisions from bitrate-first to VMAF-first cut our storage and CDN egress by about 22% without any drop in viewer QoE. We weren't guessing anymore which bitrates we could drop.

Getting VMAF right takes a bit more setup than PSNR. You need the right model for your resolution and viewing model — the default 1080p model isn't ideal for 4K or mobile. You'll also want to run it at the right sampling rate; we've found that running VMAF on every frame is overkill for VOD, every 2nd or 3rd frame is often enough for a stable score. For live workflows where you're tuning for Low-Latency HLS Configuration for CDN: Sub-3-Second Live Streaming, you’ll care more about speed, so consider the faster VMAF models or even SSIM as a proxy during the live session and run full VMAF in your post-event QA. Guidance from web.dev Articles (https://web.dev/articles/) on perceptual performance helped us frame how to weigh these temporal factors against pure image fidelity.

# Full VMAF calculation with libvmaf (VMAF v0.6.1 model)
# Use model appropriate for your resolution - 4K model for UHD content
ffmpeg -i reference.y4m -i encoded_1080p.mp4 \
  -lavfi "libvmaf=model_path=/usr/local/share/model/vmaf_v0.6.1.json:log_path=vmaf.json:log_fmt=json:feature=name=psnr|name=float_ssim" \
  -f null -

# For a faster, per-frame CSV for ladder analysis:
ffmpeg -i reference.y4m -i encoded_720p.mp4 \
  -lavfi "libvmaf=model_path=/usr/local/share/model/vmaf_v0.6.1.json:log_path=vmaf_720p.csv:log_fmt=csv" \
  -f null -
# Parse vmaf.json -> look for "pooled_metrics": {"vmaf": {"mean": 94.21}}

Building an Encoding Ladder That Balances Bits and Perception

Here's where theory hits your CDN bill. The goal isn't to max out VMAF or PSNR — it's to hit the quality target your viewers expect with the fewest bits possible. We've settled on a simple rule: pick a VMAF target per resolution and then find the lowest bitrate that gets you there for that title, rather than using fixed bitrate ladders for everything. A talking-head webinar might hit VMAF 93 at 2.5 Mbps for 1080p, while a high-motion soccer match needs 5.5 Mbps for the same score. If you use a single ladder for both, you're either wasting bits or shipping bad quality.

We run a three-stage check before any ladder goes live. First, we encode a dozen representative clips from the customer's library at various bitrates and plot the rate-distortion curves for all three metrics. Second, we set our rung thresholds — say VMAF 93 for top rungs, SSIM 0.97 as a secondary gate — and let the per-title optimizer pick bitrates. Third, we validate that choice across different devices, because the gap between DASH vs HLS for CDN Streaming: Protocol Comparison and Selection Guide packaging can influence how players switch rungs during congestion. We've seen ladders that looked perfect in the lab cause more rebuffering because the bitrate steps were too aggressive for adaptive switching.

Putting VMAF Into Practice: Wiring Metrics Into Your Transcoding Pipeline

It's one thing to talk about VMAF scores in a slide deck, it's another to run them at scale without grinding your transcode farm to a halt. If you're processing thousands of assets a day, you can't afford to run full-reference VMAF on every rendition at full resolution. We've learned to be surgical about it. Don't run VMAF on the mezzanine-to-source check — that's wasted cycles. Run it where it matters: after your per-title ladder generation, comparing each ladder rung against the de-scaled source.

Our current setup samples 3-5 representative chunks per asset — typically a high-motion, a low-motion, and a grainy/detailed scene — instead of scanning the entire 90-minute file. That cuts compute time by ~85% and still catches 95% of the bad rungs. We target a VMAF delta of no more than 4-6 points between adjacent rungs; if the 1080p at 4.5 Mbps scores 93.2 and the 720p at 2.8 Mbps scores 92.8, you're burning bits for nothing. Drop that top rung. For live, you'll want to bake this into your origin packager. Check out our breakdown of CDN Live Streaming Architecture: Ingest, Transcode, Package, and Deliver at Scale to see where that quality-check hook fits between transcode and package.

Here's the workhorse command we use in our QA containers. It calculates VMAF, PSNR, and SSIM in a single pass and dumps it to JSON for our ladder optimizer to parse:

#!/bin/bash
# Compare transcoded rendition against scaled reference
# Uses 8 threads, logs PSNR/SSIM alongside VMAF for quick triage

ffmpeg -r 24 -i source_1080p.y4m -r 24 -i rendition_720p_2800k.mp4 \
  -filter_complex "[0:v]scale=1280:720:flags=bicubic:reference[ref]; \
                   [ref][1:v]libvmaf=n_threads=8:model=path=/models/vmaf_v0.6.1.json:log_path=/tmp/vmaf_rendition.json:psnr=1:ssim=1:log_fmt=json" \
  -f null -

# Quick check: fail the rung if VMAF < 88 or PSNR < 37.5 dB
VMAF_SCORE=$(jq '.pooled_metrics.vmaf.mean' /tmp/vmaf_rendition.json)
if (( $(echo "$VMAF_SCORE < 88" | bc -l) )); then
  echo "REJECT: VMAF $VMAF_SCORE below threshold for 720p"
  exit 1
fi

When Your Metrics Lie: Troubleshooting and Per-Title Tuning

You'll quickly find that blind trust in a single number will burn you. VMAF is phenomenal, but it's trained on human opinions of mostly clean, progressive content. Throw it film grain, artificial noise, or hard-chroma animation, and it'll often punish you unfairly. We had a classic case with a noir film where grain preservation dropped VMAF from 94 to 81, even though PSNR stayed stable at 41.2 dB and our golden-eyes QA team preferred the grainy encode. If you’re seeing VMAF plummet but SSIM stays above 0.97, you’re likely looking at texture, not actual blockiness. Don't just crank the bitrate to chase the score — you'll bloat your CDN cache for no perceptual gain.

Another gotcha is resolution switching. We've seen ladders where the 1080p VMAF looks great at 93+, but the 540p rung scores an 89 and still looks terrible on a 65-inch TV because the upscaler is doing heavy lifting. Always calculate VMAF at the display resolution — scale your distorted video *up* to the reference, not the other way around. And for live workflows, you can't run a 5-minute VMAF analysis. We use a lightweight SSIM check on every 2-second CMAF chunk for real-time alerting (SSIM < 0.93 triggers a re-encode flag), then run full VMAF asynchronously on DVR segments at the edge. That edge-time-shifted approach we detail in CDN DVR Functionality at the Edge: Time-Shifted Viewing for Live Streams is perfect for this — you've got a few seconds of buffer to validate quality before the viewer rewinds. For sub-3-second latency setups, see Low-Latency DASH with CMAF Chunks: Achieving Near-Real-Time CDN Delivery — we've tuned our metric sampling there to add less than 120ms overhead per chunk.

Finally, don't set a single global VMAF target. A cartoon can look transparent at VMAF 91 and 1.8 Mbps, while a UEFA Champions League night game needs VMAF 94 at 6 Mbps to avoid grass smearing. We run per-title convex hulls and let the optimizer pick the rung where the VMAF-per-bitrate curve flattens — typically where you gain less than 0.5 VMAF points for a 15% bitrate bump. That's your sweet spot.

Frequently Asked Questions