§ 5.3  Module 5 — The Codec Wars and the Science of Compression

Seeing in Numbers: Color Spaces and Chroma Subsampling

5.3.1Why video is not RGB

A camera sensor and a canvas both hand you red, green, and blue. Call ctx.getImageData() on a frame drawn from a getUserMedia track and you receive an RGBA byte array: four bytes per pixel, each channel at full resolution, each treated as equally important. No video codec accepts that format, and the reason is that RGB spends bits with perfect egalitarianism on channels the eye does not weight equally. Every one of the three channels carries a full-resolution copy of the scene's brightness structure — the three are highly correlated — so compressing them independently wastes work, and compressing them equally wastes bits on color detail no viewer can see.

The fix is Valensi's: rotate the coordinate system. Separate what the eye is sharp at — brightness — from what it is blurry at — color — and give each the resolution it deserves. The rotated space is Y′CbCr: a luma component Y′ that is a weighted sum of the gamma-corrected R′G′B′ channels, and two chroma difference components, Cb (how blue the pixel is relative to its luma) and Cr (how red). The BT.601 full-range transform, the one JPEG's JFIF file format (Eric Hamilton, 1992) standardized and the one you will implement below, is:

Y′ =  0.299·R + 0.587·G + 0.114·B
Cb = -0.168736·R - 0.331264·G + 0.5·B      + 128
Cr =  0.5·R      - 0.418688·G - 0.081312·B + 128

R  = Y′ + 1.402·(Cr − 128)
G  = Y′ − 0.344136·(Cb − 128) − 0.714136·(Cr − 128)
B  = Y′ + 1.772·(Cb − 128)

Listing 5.3 — The BT.601 full-range transform and its inverse. All values in 0–255.

Read the luma row and you can see 1953 in it: green contributes 58.7 percent, red 29.9, blue 11.4 — weights derived from the luminous efficiency of the NTSC phosphor primaries. A gray pixel (R = G = B) produces Cb = Cr = 128 exactly: the chroma planes of a grayscale image are flat, which is precisely the property that let color TV coexist with monochrome sets, and which today lets an encoder spend almost nothing on them.

5.3.2Two matrices and two ranges

There is not one Y′CbCr; there are several, and confusing them produces bugs that are maddening precisely because the picture still looks almost right. Two axes of variation matter in practice.

BT.601 versus BT.709. When high-definition television was standardized in ITU-R Recommendation BT.709 (1990), the reference primaries and white point changed, and with them the luma weights: K_r moved from 0.299 to 0.2126 and K_b from 0.114 to 0.0722 (leaving K_g = 0.7152). The convention that grew around it — SD content uses 601, HD content uses 709 — is exactly the kind of implicit contract that breaks silently. Decode 709-encoded chroma with the 601 inverse matrix and nothing crashes: you get a picture with subtly wrong hues, most visibly in saturated reds and greens and in flesh tones. Modern bitstreams carry explicit signaling (the matrix_coefficients field in H.264/HEVC VUI, colorspace in VP9/AV1 headers), but a hand-rolled canvas pipeline like the one you are about to write has no such metadata — you must simply know which matrix you applied.

Full range versus studio range. The equations in Listing 5.3 are full range: Y′ spans 0–255. Broadcast-derived video instead uses studio range (also "limited" or "video" range): Y′ occupies 16–235 and chroma 16–240. The reservation is another fossil with a concrete origin — in serial digital interfaces per BT.656, the byte values 0 and 255 are reserved for the SAV/EAV timing reference codes that mark line boundaries, and the remaining headroom and footroom absorb overshoot from analog filter ringing. The studio-range luma equation is Y′ = 16 + 219·(0.299R + 0.587G + 0.114B)/255, and the two ranges are the classic source of washed-out or crushed video: interpret a studio-range frame as full range and black lifts to dark gray while white dims; apply the expansion twice and blacks crush and highlights clip. WebRTC's internal pipelines (libyuv's I420 buffers, as used by libwebrtc since Google open-sourced the stack in 2011) conventionally carry limited-range BT.601-coded frames from capture, which is why range confusion appears so often in custom rendering code downstream of a decoder.

5.3.3The subsampling ladder and the 1.5-byte pixel

Rotating into Y′CbCr saves nothing by itself — three planes of w×h bytes is the same 3 bytes per pixel RGB cost. The saving comes from the second step, chroma subsampling: storing the chroma planes at lower spatial resolution than luma, on the NTSC theory that the eye will not notice.

The J:a:b notation describes sampling over a reference block four pixels wide and two rows tall: J is the width (always 4), a is the number of chroma samples in the first row, b the number of additional chroma samples in the second row. So 4:4:4 is no subsampling; 4:2:2 halves chroma horizontally (Rec. 601's studio format); 4:2:0 halves it in both dimensions, one chroma pair per 2×2 block of luma pixels. The "4" itself is a fossil: Rec. 601 sampled luma at 13.5 MHz, four times a 3.375 MHz base rate chosen as a common multiple compatible with both NTSC and PAL line frequencies, and the digits originally counted multiples of that base clock.

The arithmetic is worth doing exactly, because it is the arithmetic your buffers will obey. A 4:2:0 frame of width w and height h stores a full-resolution luma plane of w·h bytes, plus a Cb plane and a Cr plane of (w/2)·(h/2) bytes each: w·h + 2·(w·h/4) = w·h·1.5 bytes. For 1280×720 that is 921,600 + 230,400 + 230,400 = 1,382,400 bytes per frame — exactly half of the 2,764,800 bytes of 24-bit RGB or 4:4:4. At 30 frames per second, 4:2:0 alone drops raw 720p from about 664 Mbps to about 332 Mbps: a free 2:1 before the encoder proper has done any work. The remaining factor of a few hundred is the business of prediction and transform coding — the subject of video compression proper, and of the next lesson.

The canonical memory layout, and the one WebRTC's native pipeline uses everywhere, is planar I420: all Y′ bytes first in raster order, then the whole Cb plane, then the whole Cr plane. Its sibling NV12 (common in hardware encoders) keeps the Y plane but interleaves Cb and Cr in a single half-resolution plane.

5.3.4Why the eye forgives: luma acuity versus chroma acuity

Subsampling only works because it targets a genuine asymmetry in the visual system, and the asymmetry has been measured. The retina contains roughly 120 million rods and only about 6 million cones, and spatial detail in daylight is carried by the summed response of the long- and medium-wavelength cones — a luminance signal — routed through the high-resolution parvocellular pathway. Color, by contrast, is computed as opponent differences between cone classes (red–green and blue–yellow channels, a structure Cb and Cr loosely echo), and those difference channels are spatially coarse. Kathy Mullen's 1985 measurements in the Journal of Physiology put numbers on it: luminance gratings remain visible out to roughly 50–60 cycles per degree, while red–green chromatic gratings cut off near 11–12 cycles per degree and blue–yellow lower still. The blue–yellow channel is further starved by anatomy — S-cones are about 5–10 percent of all cones and are absent from the very center of the fovea.

Halving chroma resolution in each dimension therefore discards information most viewers cannot resolve at normal viewing distances — the purest example in this module of the encoder's real job: discarding what the brain would have discarded anyway. It is psychovisual exploitation, and it is honest work. But "most viewers, most content" is not "always," and the failure cases are worth knowing by sight.

5.3.5Where the compromise bites: red text and screen content

Chroma subsampling was tuned for camera imagery — soft-edged, noisy, natural. It fails on content whose sharp edges live in the chroma planes. The classic exhibit is saturated red text on a dark background. Pure red (255, 0, 0) has luma of only about 76, so against black the luma plane sees modest contrast, while the chroma planes see an enormous step (Cr swings from 128 to its ceiling). After 4:2:0, that chroma step is stored at half resolution and interpolated back on display: the edge smears across a 2-pixel halo, and thin red strokes read as fuzzy, dim, or fringed. Blue text suffers the same way; green and white text, whose edges are luma-dominated, survive far better.

Screen sharing concentrates the pathology: one-pixel window borders, saturated syntax-highlighted code, and subpixel-antialiased text (ClearType deliberately injects chroma detail at the pixel scale) are everything subsampling assumes away. This is why the codec world grew screen-content tooling — H.264's High 4:4:4 Predictive profile, VP9 profiles 1 and 3, AV1's 4:4:4 support, and HEVC's Screen Content Coding extensions (2016) all exist to keep chroma at full resolution when the content demands it — and why WebRTC exposes track.contentHint = 'text' and the 'detail' hint, which bias the encoder toward spatial fidelity over frame rate. In practice, though, the browser codec pipeline for calls remains overwhelmingly 4:2:0 I420 end to end: when your screenshared terminal looks worse than your face, you are looking at 1953's bandwidth budget, still being paid.

Problems for § 5.3Check your understanding

One anchor exercise: build the color pipeline yourself, byte by byte, and prove the 1.5-byte pixel on real frame data.

¶Field quirks

What the specifications say, and what the browsers actually do, are two different documents. These are the divergences that cost production teams their weekends.