§ 3.5  Module 3 — The Broken Internet: NAT and the Art of Hole Punching

Trickle, Restarts, and Real-World Traversal

3.5.1Shipping candidates as they appear

Vanilla ICE is a strict pipeline: gather every candidate, write them all into the SDP, send the offer, wait while the remote side gathers everything in turn, receive the answer, and only then begin connectivity checks. Each stage waits for the slowest member of the one before it — and the slowest member of gathering is almost always a timeout: an unanswering STUN server, a TURN allocation over a congested path, a VPN adapter that swallows packets. The user hears the result as dead air after “calling…”.

Trickle ICE dissolves the pipeline into parallel streams. Calling setLocalDescription starts gathering; the offer may leave immediately, containing whatever candidates happen to exist at that instant — often none. Each candidate that materializes afterward is surfaced through onicecandidate, shipped over your signaling channel, and delivered to the far side with addIceCandidate. Pairing and connectivity checks begin as soon as the first candidates meet, so gathering, signaling, and checking all overlap. In the common case the time to a working pair collapses to roughly one signaling round trip plus one check round trip; the STUN and TURN stragglers arrive later and join a contest already under way. Support is advertised with a=ice-options:trickle, and an agent facing a vanilla peer can run half trickle — gather fully before offering while still accepting incremental candidates from the other direction.

Incremental delivery creates a question vanilla ICE never had to ask: how does the receiving agent know the stream of candidates has ended? Until it knows, it cannot declare the pair set exhausted, so failure could only be diagnosed by timeout. The answer is an explicit end-of-candidates signal, and it wears three costumes. On the wire it is the SDP attribute a=end-of-candidates. In the browser API, the per-transport marker is an RTCIceCandidate whose candidate string is empty, and the end of gathering for the whole connection is announced by onicecandidate firing with event.candidate === null — at which moment iceGatheringState becomes 'complete'. The null is a period at the end of a sentence, not an error message; forwarding the indication to the peer (calling addIceCandidate() with no argument signals end-of-candidates remotely) lets its ICE agent fail fast instead of waiting out a timer.

pc.onicecandidate = ({ candidate }) => {
  if (candidate) {
    signaling.send({ candidate: candidate.toJSON() }); // ship it the moment it exists
  } else {
    signaling.send({ endOfCandidates: true });         // gathering finished — not an error
  }
};

// receiving side
signaling.onmessage = async ({ data }) => {
  const msg = JSON.parse(data);
  if (msg.candidate)            await pc.addIceCandidate(msg.candidate);
  else if (msg.endOfCandidates) await pc.addIceCandidate(); // end-of-candidates indication
};

Listing 3.5 — Trickle wiring in full. Candidates travel as individual signaling messages, and the null event is translated into an explicit end-of-candidates indication rather than being dropped on the floor.

3.5.2The address that told too much

Host candidates are honest to a fault: they carry the machine's actual interface addresses — the RFC 1918 private addresses that NAT normally hides from the wider internet. In early 2015 Daniel Roesler published a GitHub demonstration of what that honesty was worth: a page could construct an RTCPeerConnection, add a data channel, call createOffer, and read the user's local IP addresses out of the candidate events — no permission prompt, nothing for the user to decline. Server-reflexive gathering could meanwhile disclose the public address even where a proxy or VPN was supposed to conceal it, and the technique promptly joined the ad-tech fingerprinting toolkit: a local IP is a fine cross-site identifier.

Browser vendors sat on an uncomfortable trade for four years. Removing host candidates would have broken the best case of the whole architecture — two machines on the same LAN connecting directly instead of hairpinning through a distant relay. The resolution, deployed in 2019, was to keep the candidates and hide the addresses: the browser replaces each private IP with a freshly minted UUID hostname ending in .local and registers that name via multicast DNS (RFC 6762, February 2013, Stuart Cheshire and Marc Krochmal: the protocol behind Apple's Bonjour). A peer on the same link resolves the name by asking the local multicast group and connects exactly as before; the page sees only a random identifier that changes between sessions. Related-address fields on reflexive candidates are zeroed out for the same reason. Chrome enabled mDNS candidates by default in mid-2019 (M75); the draft came jointly from Apple and Google — Youenn Fablet with Justin Uberti and Qingsi Wang — and Safari shipped the same scheme. The cost is modest: name resolution adds a beat to checks, and networks that filter multicast leave .local candidates unresolvable, pushing those LANs onto reflexive or relayed pairs.

3.5.3TCP candidates, ICE-lite, and the policy knob

Some networks simply refuse UDP — corporate firewalls, hotel middleboxes, badly configured guest Wi-Fi. RFC 6544 (March 2012; Rosenberg, Keränen, Lowekamp, Roach) extends ICE with TCP candidates for exactly this terrain, and gives each a tcptype: active (this agent will open the connection outward), passive (this agent listens and accepts), and so (both sides attempt a TCP simultaneous-open toward each other). A browser typically gathers active host TCP candidates, and their SDP betrays a small idiom worth recognizing: the port reads 9 — the discard service — because an active candidate never listens; the field is a placeholder, not an address anyone will dial. Media over a TCP pair is framed with a 16-bit length prefix and inherits TCP's pathologies — head-of-line blocking, retransmission delay — so the candidate-type preferences keep TCP at the bottom of the pile: a TCP pair wins only when every UDP pair has failed. In production the pattern that actually rescues UDP-hostile networks is usually TURN reached over TCP or TLS on port 443, which wears the costume of ordinary HTTPS traffic; direct TCP candidates matter mostly when one side is a media server that can hold a real listening socket.

Servers get a second dispensation. A host with a public, unfiltered address does not need reflexive discovery, relays, or even the ability to originate checks — so RFC 8445 defines ICE-lite: a minimal implementation that offers only host candidates, never initiates connectivity checks, only answers them, and always takes the controlled role. It announces itself with a single SDP line, a=ice-lite. Media servers and SIP gateways use it to shed most of the state machine; browsers are always full ICE. Finally, the application holds one blunt policy instrument: iceTransportPolicy in the RTCPeerConnection configuration. Its default, 'all', gathers everything; 'relay' suppresses host and server-reflexive candidates entirely, forcing every pair through TURN — the tool of enterprises pinning media to auditable relays, of strict privacy modes, and of engineers proving their TURN deployment before the day it is needed.

3.5.4Restart: changing horses under a live call

A nominated candidate pair is a promise about two concrete addresses, and networks break such promises casually: a laptop roams from Wi-Fi to a phone hotspot, a VPN connects or drops, a train changes cells. The addresses your call was built on stop existing while the session — tracks, codecs, the DTLS association — has no reason to die. ICE restart is the surgical option: rebuild the transport underneath a session without touching the session. The signal, defined by RFC 8445 and carried through JSEP, is an offer in which a media section's a=ice-ufrag and a=ice-pwd change. New credentials mean new generation: both agents discard their candidate lists, re-gather, and re-run checks, while everything above the transport — the m-line structure, negotiated codecs, DTLS certificates — carries over from the offer/answer state already established. Crucially, restart is make-before-break: the old pair keeps carrying media, if it still can, until a new pair is nominated.

The modern API is pc.restartIce(), added to WebRTC 1.0 in 2019. It sends nothing itself: it flags the next createOffer to mint fresh credentials and fires negotiationneeded so your ordinary signaling path runs. (The older spelling, createOffer({ iceRestart: true }), comes from JSEP — RFC 8829, January 2021, Uberti, Jennings, Rescorla — and does the same thing explicitly.) iceConnectionState of 'failed' is the canonical trigger — checks have been exhausted and no pair works — as is an operating-system signal that the network changed. A transient 'disconnected' usually deserves patience: checks are still running, Wi-Fi blips heal in seconds, and restart-on-every-wobble produces flapping renegotiations. And restart is not a general-purpose fix-it: it cannot repair a broken signaling channel (the restart offer could not be delivered), it does nothing for codec or track changes (ordinary renegotiation and replaceTrack cover those), and because it rides a normal offer, an ill-timed restart can collide head-on with the peer's own offer — glare, whose systematic cure is the perfect negotiation pattern of Module 4.

pc.oniceconnectionstatechange = () => {
  if (pc.iceConnectionState === 'failed') {
    pc.restartIce();          // flags the next createOffer; fires negotiationneeded
  }
};
// Your existing negotiationneeded handler then produces and signals
// an offer whose a=ice-ufrag / a=ice-pwd are brand new.

Listing 3.6 — The minimal survival reflex. Restart on 'failed'; give 'disconnected' a grace period before escalating.

3.5.5Consent, and reading the wire's pulse

Reaching 'connected' is not a perpetual license to transmit. From the network's point of view a browser is a packet cannon that strangers' JavaScript can aim, so WebRTC demands ongoing proof that the far side still wants your traffic. RFC 7675 (October 2015) supplies it: consent freshness. On the pair in use, the agent sends an authenticated STUN Binding Request roughly every five seconds — the interval is randomized so flows do not synchronize — and each response renews consent. Thirty seconds without a response and consent expires: the agent must stop sending on that pair. This is distinct from the keepalives that maintain NAT bindings (STUN Binding Indications, fire-and-forget, no reply expected): an indication can hold a mapping open, but only a request–response round trip can prove a live, willing peer. Consent expiry is why yanking a cable silences outbound media within half a minute, even before any state-change event fires.

All of this machinery is observable through getStats. For now you need only a survival recipe; the full statistics graph gets its own treatment in §6.4. The shape: pc.getStats() resolves to an RTCStatsReport, a map-like object keyed by opaque id, whose values are flat dictionaries each carrying id, timestamp, and — the field you navigate by — type. The types that matter this week are 'transport', 'candidate-pair', 'local-candidate', and 'remote-candidate'. To find the pair carrying your media: take the 'transport' entry, follow its selectedCandidatePairId into the report, then follow the pair's localCandidateId and remoteCandidateId to the two candidates and read their candidateType. One warning: every counter is cumulative since the connection began — requestsSent, responsesReceived, bytesSent become rates only by differencing two snapshots, the discipline deferred to §6.4.

const stats = await pc.getStats();

let pair = null;
stats.forEach((s) => {
  if (s.type === 'transport' && s.selectedCandidatePairId) {
    pair = stats.get(s.selectedCandidatePairId);
  }
});

const local  = stats.get(pair.localCandidateId);
const remote = stats.get(pair.remoteCandidateId);

console.log(local.candidateType, '→', remote.candidateType);   // e.g. "host → host"
console.log('requests out:', pair.requestsSent,
            'responses back:', pair.responsesReceived);

Listing 3.7 — The selected-pair recipe: transport → selectedCandidatePairId → pair → its two candidates. Chrome renders the same objects live at chrome://webrtc-internals.

Learn one stats signature by heart, because it diagnoses the perennial “connects on my Wi-Fi, not from the office” report: a candidate pair whose requestsSent climbs while responsesReceived stays pinned at zero. Your checks are leaving; nothing is coming back. On a server-reflexive pair this is the fingerprint of an address- and port-dependent mapping — the symmetric NAT of Lesson 3.1 — on the far side: the reflexive address the peer advertised was minted by its NAT for the STUN server's benefit, and packets from anyone else arrive at a mapping that never existed for them. The pair will sit at 'in-progress' until ICE gives up on it, and the connection, if it survives, survives on a relay. When only one side is symmetric the story usually ends better: its outgoing checks arrive at the other's candidate from an unexpected port, the receiver mints a peer-reflexive candidate on the spot, and the call proceeds — “symmetric NAT kills P2P” is folklore unless both ends offend at once. A healthy selected pair, by contrast, shows both counters climbing forever: that steady responsesReceived increment is consent freshness itself, visible in the ledger.

Problems for § 3.5Check your understanding

One live lab and one bench check: watch a real ICE session gather, connect, and restart on a loopback pair, then read canned evidence the way an on-call engineer would.

¶Field quirks

What the specifications say, and what the browsers actually do, are two different documents. These are the divergences that cost production teams their weekends.