§ 4.2  Module 4 — The Browser Gets a Dial Tone

Capturing the World: getUserMedia and Tracks

4.2.1A permission model forged by failure

The call that starts every capture pipeline is one line:

const stream = await navigator.mediaDevices.getUserMedia({ audio: true, video: true });

Listing 4.2.1 — the request. Everything interesting is in what happens before the promise settles.

Three rules govern that line, and each traces to a specific failure of the plugin era. First, the prompt belongs to the browser, not the page. The permission UI is drawn in browser chrome — outside the page's coordinate space — precisely because the 2008 clickjacking attack proved that anything a page can position, a page can disguise. A page cannot style, move, or pre-click the prompt; it can only await the outcome, and a denial arrives as a rejection with NotAllowedError.

Second, secure contexts only. Since Chrome 47, navigator.mediaDevices is simply absent on plain HTTP (with an exception for localhost and 127.0.0.1, which the Secure Contexts specification deems potentially trustworthy — this is why the course's exercises run from local files and local servers without ceremony). The reason is subtler than "encryption is good": permission grants persist per origin. If http://example.com could hold a standing camera grant, then any attacker on the network path — a coffee-shop router, a compromised ISP cache — could inject JavaScript into that origin and inherit the grant. HTTPS is not protecting the media; it is protecting the integrity of the decision. The media itself will be encrypted end-to-end by DTLS-SRTP regardless (Module 5).

Third, use must be visible. While any track from a device is live, the browser paints an indicator — the red dot in the tab strip, the camera glyph in the address bar, the OS-level orange dot on macOS — that the page cannot suppress. Flash's indicator lived inside the plugin's rectangle; the web's lives in chrome the page cannot reach. The pattern generalizes: every capability the page wants (prompting, indicators, device labels) is mediated by UI the page cannot draw.

4.2.2The track is the atom

What getUserMedia resolves with is a MediaStream, but the stream is nearly a fiction — a labeled bag whose only real job is grouping. The unit of capture, control, and transmission is the MediaStreamTrack: one track per camera, one per microphone, each carrying its own settings, its own lifecycle, and — once we reach RTCPeerConnection in § 4.3 — its own sender. When you mute a call, you operate on a track. When the encoder adapts, it adapts a track. Learn the track's states now and the rest of the module reads easily.

A track has exactly two readyState values: live and ended, and the transition is one-way. Calling track.stop() ends that track immediately and irrevocably — there is no restart; you call getUserMedia again if you want the device back. Notably, stop() does not fire the track's ended event: the event exists to tell you about ends you did not cause — the USB camera unplugged, the user revoking permission from the address bar, the OS reclaiming the device. Code that waits for ended after calling stop() waits forever.

Orthogonal to the lifecycle are two flags that beginners persistently conflate. track.enabled is application-controlled: set it to false and the track stays live but produces black frames or silence — the far end keeps receiving media, just contentless media. This is what a mute button should do, because re-enabling is instant. track.muted, by contrast, is user-agent-controlled and read-only: the browser sets it when the source cannot provide data (a backgrounded tab throttling the camera, an OS-level mic mute, a hardware privacy shutter). Your page cannot set muted; it can only listen for mute/unmute events and update its UI honestly.

Finally, track.clone() forks the track: the clone shares the underlying source but has independent enabled, independent constraints, and an independent lifecycle — stopping the clone leaves the original live. Cloning is how one camera feeds both a full-resolution local preview and a heavily constrained outgoing track.

4.2.3Constraints and the fitness distance

You rarely want "a camera"; you want 720p at 30 frames per second, echo-cancelled audio. The constraints dictionary is how you say so:

const stream = await navigator.mediaDevices.getUserMedia({
  audio: { echoCancellation: { exact: true } },
  video: { width: { ideal: 1920 }, frameRate: { ideal: 30 } },
});

Listing 4.2.2 — one mandatory constraint (exact) and two hints (ideal).

Every constrainable property — width, height, frameRate, facingMode, deviceId, echoCancellation (the switch for the acoustic echo canceller you will meet again in the voice pipeline), and a dozen more — accepts either a bare value or an object with exact, ideal, min, and max members. The semantics split cleanly in two. exact (and min/max) are mandatory: a device that cannot satisfy them is disqualified, and if every device is disqualified the promise rejects with OverconstrainedError, whose constraint attribute names the offending property. ideal (and bare values, which are sugar for ideal) are hints: they can never cause failure; they only rank the survivors.

The ranking is not folklore — the specification pins it down as the SelectSettings algorithm, built on a number called the fitness distance. For each candidate configuration a device could produce, and each constraint you supplied, the distance is 0 if the candidate matches the ideal exactly; for numeric properties it is

|actual − ideal| / max(|actual|, |ideal|)

— a normalized error between 0 and 1 — and for strings and booleans it is simply 0 or 1. Unmet exact means distance +∞: disqualification. The browser sums the distances across all constraints and picks the candidate with the smallest total. Ask for width: { ideal: 1920 } from a 1280-wide laptop camera and a 3840-wide USB camera, and the laptop wins: 640/1920 ≈ 0.333 beats 1920/3840 = 0.5. No error, no warning — a quiet, deterministic compromise. You will implement this exact arithmetic in Problem 1.

One more clause of the algorithm is a genuine trap: unknown constraint names are ignored. The spec requires the browser to discard any dictionary member it does not recognize, so { echoCancelation: { exact: true } } — one l missing — is not an error. The call succeeds, the misspelled requirement evaporates, and you discover it three weeks later in an echo complaint. Real browsers must tolerate this for forward compatibility (a new constraint added to the spec must not break old browsers); your code must therefore never rely on the platform to catch typos. The resolver you write in Problem 1 will be stricter than the spec on purpose, so the failure is loud once in your life.

Predict before you peek — the three questions below are answerable entirely from § 4.2.2 and § 4.2.3.

4.2.4What a page can learn without asking

navigator.mediaDevices.enumerateDevices() lists cameras, microphones, and audio outputs — and it works before any permission grant, because a call UI needs to know whether a microphone exists at all before it offers to place a call. That tension — utility versus fingerprinting — produced one of the API's more careful compromises: before a grant, the page may learn device kinds and rough counts, but the label fields come back as empty strings, and deviceId values are unstable or blank. Only after the user grants capture (or the origin holds a persistent grant) do labels like "Logitech BRIO" appear. Device IDs, when present, are hashed per origin and reset when site data is cleared, so they cannot serve as a cross-site supercookie. A page that shows a device picker with real names before ever calling getUserMedia is a page you should distrust on sight — it cannot be using the standard API honestly.

Fingerprinting through capture APIs is not hypothetical. In January 2015, Daniel Roesler published a one-page demo showing that an RTCPeerConnection — no permission prompt involved — would happily enumerate your machine's private IP addresses in its ICE candidates, handing trackers a durable identifier and a map of your home network. The eventual fix, mDNS candidate obfuscation (Chrome 75, 2019), replaces host IPs in candidates with ephemeral .local names that only peers on the same network can resolve — you will see those candidates yourself in Module 3's tooling and again in § 4.4. The episode set the modern rule of thumb: capability without consent must not carry identity.

Screen capture plays by deliberately harsher rules. getDisplayMedia() (the W3C Screen Capture specification; Chrome 72, January 2019) always prompts, never persists a grant, and — critically — the page cannot choose the surface. It may express preferences, but the picker listing your screens, windows, and tabs is browser chrome, and the choice is the user's alone. The hazards justify the harshness: sharing "entire screen" broadcasts every notification toast, every password manager autofill, every other tab; a shared browser tab can leak session-bound URLs to everyone on the call. Production conferencing UIs steer users toward sharing a single window or tab for exactly this reason, and the API's design makes it impossible for a malicious page to quietly self-select your banking window.

4.2.5Deterministic media: the course's stunt doubles

This course grades hundreds of assertions against live media pipelines, and a graded test that pops a camera permission prompt is a broken test: permission may be denied, the machine may have no camera, and no two webcams produce the same pixels. So from here to the final exam, graded work uses two permission-free sources that are better than real for verification, because they are deterministic.

For video: draw on a canvas and call canvas.captureStream(fps) (specified in Media Capture from DOM Elements), which returns a MediaStream whose video track is a first-class MediaStreamTrack — cloneable, stoppable, sendable through a peer connection. Embed a frame counter in the pixels and the receiving end can prove not merely that video arrived but which frame arrived, turning "does it look right?" into an assertion. For audio: the Web Audio API's OscillatorNode feeding a MediaStreamAudioDestinationNode yields an audio track carrying a pure tone of known frequency — an analyzer on the far side can confirm the 440 Hz actually crossed the wire.

function mockVideoTrack(width = 640, height = 360, fps = 15) {
  const canvas = document.createElement('canvas');
  canvas.width = width; canvas.height = height;
  const ctx = canvas.getContext('2d');
  let frame = 0;
  setInterval(() => {
    ctx.fillStyle = '#123'; ctx.fillRect(0, 0, width, height);
    ctx.fillStyle = '#fff'; ctx.font = '48px monospace';
    ctx.fillText('frame ' + frame++, 20, height / 2);   // counter baked into pixels
  }, 1000 / fps);
  return canvas.captureStream(fps).getVideoTracks()[0];
}

function mockAudioTrack(audioCtx) {          // audioCtx created inside a click handler
  const osc = audioCtx.createOscillator();   // 440 Hz by default
  const dest = audioCtx.createMediaStreamDestination();
  osc.connect(dest);
  osc.start();
  return dest.stream.getAudioTracks()[0];
}

Listing 4.2.3 — the course's mock media sources: a canvas pattern with an embedded frame counter, and an oscillator tone. No prompts, no hardware, no flakiness.

One rule of engagement carries over from the harness itself: browsers suspend an AudioContext created outside a user gesture, so mock audio contexts are constructed — or resume()d — synchronously inside the Run button's click handler, and the harness verifies state === 'running' before grading any audio-flow assertion. When you later see an exercise fail with a setup error rather than a red test, this is what tripped.

Problems for § 4.2Check your understanding

One implementation of the spec's selection algorithm over a canned device list, then a short quiz on what constraints and permissions actually promise.

¶Field quirks

What the specifications say, and what the browsers actually do, are two different documents. These are the divergences that cost production teams their weekends.