§ 3.4  Module 3 — The Broken Internet: NAT and the Art of Hole Punching

ICE: The State Machine That Saved P2P

This document describes a protocol for Network Address Translator (NAT) traversal for UDP-based communication.— RFC 8445, abstract, July 2018

By the end of this lesson

  • Parse a=candidate lines field-exactly, including mDNS and tcptype candidates
  • Implement candidate priority and RFC 8445 pair priority (2^32*MIN + 2*MAX + (G>D?1:0)) and construct a sorted checklist
  • Walk the pair state machine: frozen/waiting/in-progress/succeeded/failed
  • Explain controlling/controlled roles, tie-breakers, and regular vs aggressive nomination

Historical note

When Niklas Zennström and Janus Friis shipped Skype in August 2003, its calls connected from behind hotel routers and office firewalls — proof, at consumer scale, that the hole-punching tricks its engineers carried over from the KaZaA file-sharing network could defeat NAT. But Skype was a closed island: its traversal logic was proprietary folklore, unavailable to anyone building on open protocols. That same year Jonathan Rosenberg, then at dynamicsoft and already a co-author of SIP and STUN (RFC 3489, March 2003), circulated the first drafts of Interactive Connectivity Establishment at the IETF's MMUSIC working group. The idea was to replace cleverness with exhaustiveness: gather every plausible address a peer might be reachable at, form every plausible pair, probe them all with STUN checks in a strict priority order, and nominate a winner. Standardizing that took seven years — ICE became RFC 5245 in April 2010, alongside TURN (RFC 5766) — and eight more to repair: RFC 8445 (Keränen, Holmberg, and Rosenberg, July 2018) rewrote the algorithm, deleted aggressive nomination, and split the SDP encoding out into RFC 8839. ICE is the most intricate state machine in WebRTC, and its output is almost comically small: one working 5-tuple. It is why your calls take two seconds to connect instead of failing entirely.

3.4.1The cast of candidates

ICE begins from a confession: an endpoint does not know its own address. Behind a NAT, the address configured on the interface is not the address the world sees; behind two NATs, even the operator of the network may not know. So an ICE agent gathers candidates — transport addresses at which it might, plausibly, be reachable — and lets measurement decide. RFC 8445 defines four types. A host candidate is an address read directly off a local interface. A server-reflexive candidate (srflx) is the public mapping a STUN server observed on the agent's outbound packet — the NAT's outside face, as discovered in §3.2. A relayed candidate (relay) is an address allocated on a TURN server, traffic to which the server forwards — the last resort of §3.3. And a peer-reflexive candidate (prflx) is never gathered at all: it is discovered mid-conversation, when a connectivity check arrives from an address nobody advertised. We will meet it in §3.4.3.

Two more attributes organize the pile. Every candidate carries a component ID: component 1 carries RTP, component 2 carries RTCP, a separation inherited from the days before RTCP multiplexing (RFC 5761) folded both onto one port — modern browsers bundle everything onto component 1. And every candidate carries a foundation, an opaque token that is equal for two candidates when they are the same type, from the same base address, over the same protocol, obtained from the same STUN or TURN server. Foundations exist so ICE can generalize: if a check on one path through a given NAT succeeded, sibling paths through the same NAT are suddenly promising, and ICE uses the foundation to unfreeze them together (§3.4.5).

On the wire — that is, in the SDP your application ferries — each candidate is one attribute line. The first six fields after the colon are positional: foundation, component ID, transport, priority, connection address, and port. Then comes the literal token typ and the type; then, for reflexive and relayed candidates, raddr/rport giving the related address — the base the candidate was derived from, included purely for diagnostics. Everything after that is an open-ended series of extension attribute pairs (generation 0, ufrag 8hhY, network-cost 999…), which a robust parser must skip without complaint.

candidate: 842163049 1 udp 1686052607 203.0.113.5 61041 typ srflx foundation transport connection address candidate type component priority port raddr 192.168.1.7 rport 54297 ← related address: the base it was derived from (diagnostic only)
Figure 3.4a — Anatomy of a server-reflexive a=candidate line (RFC 8839 §5.1). Fields after typ srflx arrive as open-ended attribute pairs; parsers must tolerate ones they do not recognize.

Two modern wrinkles complicate the address field itself. Since 2019, browsers stopped writing host candidates' private IPs into SDP at all: Chrome (and then Safari) began publishing an ephemeral multicast-DNS name instead — 9b36eaac-….local — resolvable only on the local network, so that a web page cannot harvest your LAN topology. Your parser will therefore meet connection addresses that are not IP addresses, and must not reject them. And RFC 6544 adds TCP candidates, whose lines carry a tcptype extension: active (will open the connection), passive (will accept one), or so (simultaneous-open). TCP candidates advertise port 9 (the "discard" port) when active, since the source port of an outbound TCP connection is unknowable in advance.

3.4.2Priority arithmetic and the checklist

With perhaps five candidates on each side, there may be twenty-five conceivable pairings, and probing them in a stupid order costs seconds. ICE's answer is two formulas, evaluated identically by both peers, so that both arrive at the same probing order without negotiating it.

The first assigns each single candidate a 32-bit priority (RFC 8445 §5.1.2.1):

priority = (2^24)·(type preference)
         + (2^8)·(local preference)
         + (256 − component ID)

Listing 3.4a — The candidate priority formula. Type preference dominates; components break the final tie.

The recommended type preferences (§5.1.2.2) are 126 for host, 110 for peer-reflexive, 100 for server-reflexive, and 0 for relayed — direct paths first, relays absolutely last. Peer-reflexive outranks server-reflexive deliberately: a prflx address was learned from a check that actually arrived, which is better evidence than a STUN server's third-party observation. The local preference (0–65535) orders an agent's own interfaces — Wi-Fi versus cellular versus VPN; a single-homed host uses 65535. Run the formula on a single-homed host candidate, component 1, and you get 126·2²⁴ + 65535·2⁸ + 255 = 2,130,706,431 — a number you have already seen in every chrome://webrtc-internals dump of your life.

The second formula scores a pair. Let G be the candidate priority computed by the controlling agent and D the one computed by the controlled agent (roles are defined in §3.4.4). Then (RFC 8445 §6.1.2.3):

pair priority = 2^32 · MIN(G, D)  +  2 · MAX(G, D)  +  (G > D ? 1 : 0)

Listing 3.4b — The pair priority formula. The result exceeds 2⁵³, so JavaScript implementations need BigInt.

The weighting encodes a judgment: multiplying the minimum by 2³² means a pair is only as good as its worse half — one relayed side drags the whole pair to the bottom regardless of how splendid the other side is. The 2·MAX term ranks pairs with equal minima, and the final bit merely breaks the symmetry between (G, D) and (D, G) so that both agents produce a total order with no ties. Because the formula uses MIN and MAX rather than "mine and yours," both peers compute identical values and sort their checklists identically — coordination through arithmetic, with no messages exchanged.

The checklist is then built mechanically (§6.1.2): pair every local candidate with every remote candidate of the same component and IP address family; replace any local server-reflexive candidate by its base, because checks are always sent from the base anyway (the NAT applies the mapping; you cannot bind a socket to an address you do not own); prune the duplicates this creates, keeping the higher-priority survivor; sort descending by pair priority; and cap the list — RFC 8445 recommends a default limit of 100 pairs.

Check your understanding

Auto-graded check

2 auto-graded questions with an explanation for every wrong answer. Requires JavaScript. (m3-l4-check1)

Interactive minigame

Punch the hole — two NATs, one stranger, and the packet that must go first

A 3D instrument you drive yourself, one variable at a time. It needs JavaScript and WebGL, so it is not shown in this static copy of the page.

Peer-reflexive outranks server-reflexive in the priority formulaspec

RFC 8445 §5.1.2.1 gives priority as (2^24)*type-pref + (2^8)*local-pref + (256 - component-id), with recommended type preferences of host 126, peer-reflexive 110, server-reflexive 100, and relay 0. Everyone remembers host > srflx > relay and forgets that a peer-reflexive candidate — one discovered from an inbound check arriving from a mapping you never gathered — beats your own srflx. This is why the selected pair sometimes uses an address that appears nowhere in either side's signalled candidate list, which looks like a bug in your logging.

What to do

Expect prflx candidates in getStats() and do not treat "address not in our candidate list" as corruption. When reconciling logs, remember that prflx candidates are never signalled by definition.

3.4.3Connectivity checks: STUN on the media path

Everything so far is bookkeeping. The measurement is the connectivity check: a STUN Binding request sent not to a STUN server but from the local candidate's base directly to the remote candidate — on the very ports the media will use. This dual use is the design coup. Because the check traverses the same 5-tuple as the future RTP, its success is not a prediction that media will flow; it is a demonstration. And because the outbound check punches an outbound mapping in the sender's own NAT, each side's probing opens the door for the other side's — Skype's hole-punch, formalized.

Checks are authenticated. Each agent advertises an ice-ufrag and an ice-pwd in its SDP; a Binding request carries a USERNAME of the form remote-ufrag:local-ufrag and a MESSAGE-INTEGRITY HMAC keyed with the receiver's password, so a check proves the sender actually saw the peer's session description. Each request also carries a PRIORITY attribute — the priority the sending candidate would have if it were peer-reflexive — and this is where prflx candidates are born: when a Binding response's XOR-MAPPED-ADDRESS reveals a mapped address the agent never gathered (say, an unexpected NAT between the peers), or when a request arrives from an unknown source address, the agent mints a new peer-reflexive candidate on the spot, priced by that attribute, and keeps going. ICE treats surprise addresses as inventory, not errors.

One rule accelerates convergence dramatically: the triggered check. When a check arrives on some pair, the receiving agent does not merely answer it — it enqueues a check of its own in the reverse direction on that same pair, jumping the normal priority queue. The intuition is symmetry: an inbound packet that survived both NATs is the strongest possible hint that an outbound packet on the same pair will too. In practice, most working pairs are validated by a request–triggered-request exchange within one round trip, which is why a call across two home routers connects in well under a second once checks begin.

A successful check requires more than any response: the Binding response must arrive from the same address the request was sent to, and the transaction must authenticate. The validated pair — possibly rewritten with a freshly discovered prflx candidate — is added to the valid list, the set of pairs proven to carry traffic in both directions.

3.4.4Who is in charge: roles, tie-breakers, conflict

Somebody must decide when to stop probing and which pair wins; a symmetric protocol would deadlock or diverge. So ICE assigns asymmetric roles. When both agents are full implementations, the initiating agent — in offer/answer usage, the one that sent the offer — takes the controlling role; the other is controlled. If one side is ICE-lite — a minimal implementation permitted to servers with public addresses, which answers checks but never sends its own — the full agent is always controlling, regardless of who offered.

Roles can collide. After an ICE restart, or under crossed signaling, both agents may believe they are controlling. ICE resolves this without a referee: at startup each agent draws a random 64-bit tie-breaker, and every Binding request carries the sender's role and number in an ICE-CONTROLLING or ICE-CONTROLLED attribute. A controlling agent that receives a request claiming ICE-CONTROLLING compares numbers (RFC 8445 §7.3.1.1): if its own tie-breaker is greater than or equal to the one in the request, it stands its ground and rejects the check with error 487 (Role Conflict), whereupon the sender switches to controlled and retries; if its own number is smaller, it quietly switches to controlled itself. One comparison, at most one extra round trip, and exactly one agent is left in charge — a distributed election decided by two random numbers.

Check your understanding

Auto-graded check

2 auto-graded questions with an explanation for every wrong answer. Requires JavaScript. (m3-l4-check2)

Interactive minigame

The candidate pair tournament — priority, checks, nomination

A 3D instrument you drive yourself, one variable at a time. It needs JavaScript and WebGL, so it is not shown in this static copy of the page.

The selected pair changes mid-call and resets per-pair countersapi

ICE reaches connected on the first successful pair but keeps checking, and it will switch to a better pair later — a relay-to-direct upgrade, or a new interface appearing. selectedCandidatePairId then points at a different candidate-pair object with its own counters starting from zero and its own currentRoundTripTime. Your RTT graph shows a discontinuity and your bytes-per-second calculation shows a negative delta at exactly the moment the connection got better.

What to do

Track selectedCandidatePairId as part of your state and annotate the timeline when it changes, rather than treating pair stats as a continuous series. For aggregate throughput, use the transport object, which survives pair switches.

3.4.5Pair states, nomination, and the finish line

Each pair in the checklist walks a five-state machine. Every pair begins Frozen. Then, per foundation, the highest-priority pair of the lowest component is promoted to Waiting — one scout per NAT-path family. Every Ta milliseconds the agent takes the highest-priority Waiting pair, sends its check, and marks it In-Progress. A valid response moves it to Succeeded — and, crucially, unfreezes the Frozen pairs that share its foundation, because the scout has proven the route. A timeout or an unrecoverable error moves it to Failed. This is the freezing algorithm's economy: ICE explores one representative of each plausible path first and widens the search only where reality has cooperated.

Frozen Waiting In-Progress Succeeded Failed foundation unfrozen check sent (one per Ta) valid response timeout / error triggered check: inbound Binding request re-queues the pair all pairs start here
Figure 3.4b — The candidate pair state machine of RFC 8445 §6.1.2.6. One pair per foundation starts Waiting; a success unfreezes its foundation-siblings; an inbound check can pull a pair into the queue out of turn.

Succeeded pairs populate the valid list, but a valid pair is not yet the answer — ICE keeps checking, because a better pair may yet succeed. Ending the search is the controlling agent's privilege, and it is called nomination. Under regular nomination — the only kind RFC 8445 retains — the controlling agent watches the valid list, decides at leisure (highest priority, or lowest measured RTT, or vendor policy), and then repeats a check on the chosen pair with the USE-CANDIDATE attribute set. When that flagged check succeeds, both agents mark the pair nominated, it becomes the selected pair for its component, remaining checks are abandoned, and media flows. RFC 5245 also allowed aggressive nomination — setting USE-CANDIDATE on every check, so the first pair to validate self-nominates — which shaved a round trip at the cost of the selected pair flapping mid-setup as later, better checks landed; libwebrtc used it for years. RFC 8445 deleted it, trading those milliseconds back for determinism.

In the browser, this drama is legible from the outside. RTCPeerConnection summarizes the checklist in iceConnectionState: checking while pairs are In-Progress, connected once a usable pair exists, completed once nomination has concluded everywhere. And getStats() exposes each pair as a candidate-pair stats entry whose state field is exactly the RFC's — frozen, waiting, in-progress, succeeded, failed — plus a nominated boolean. The abstraction leaks on purpose; when a call fails to set up, this is where you look. How candidates flow incrementally while all this runs — trickle, onicecandidate, and restarts — is the business of §3.5.

A common error

"The more ICE candidates I see, the better my connectivity will be." Candidate count measures nothing but the number of interfaces, STUN servers, and transports you asked for. Exactly one pair per component is ultimately selected; every other candidate's work product is discarded the moment nomination concludes. A host with two candidates that include a reachable relay will connect where a host with fourteen candidates and no relay fails. When debugging, do not count candidates — read the candidate-pair states and find out why every pair that mattered went to failed.

Problems for § 3.4Deep lab — budget up to 45 minutes

Build the two halves of an ICE implementation you can build without a network — the candidate parser and the priority engine — then prove you can predict who is in charge.

Lab 1

The candidate parser

Graded in the browser against 5 assertions; the editor and harness require JavaScript.

Lab 2

Priority math and the checklist

Graded in the browser against 4 assertions; the editor and harness require JavaScript.

Check your understanding

Auto-graded check

5 auto-graded questions with an explanation for every wrong answer. Requires JavaScript. (m3-l4-ex3)

Field quirks

What the specifications say, and what the browsers actually do, are two different documents. These are the divergences that cost production teams their weekends.

Browsers only gather active TCP candidates, so direct TCP P2P is impossiblebrowser

ICE-TCP candidates carry tcptype active, passive, or so (simultaneous open). Browsers gather active candidates only, and an active candidate can connect only to a passive one. Since neither browser offers a passive candidate, two browsers can never establish a direct TCP connection to each other; the TCP candidates exist solely to reach a TURN server's passive listener. Seeing tcptype active in your SDP and concluding "we have a TCP fallback path" is wrong unless a TURN server is in the configuration.

What to do

Do not count TCP candidates as a P2P fallback. If UDP is blocked, TURN/TCP is your only option, which is another reason the relay configuration is not optional.

Nomination style differs between implementations and flips your selected pairspec

The controlling agent — normally the offerer — nominates. RFC 8445 specifies regular nomination; libwebrtc historically used aggressive nomination, which sets the USE-CANDIDATE flag on ordinary checks and effectively nominates the first thing that works. An SFU that implements only regular nomination against an aggressively nominating browser will see the selected pair chosen earlier and differently than it expected, sometimes landing on a relay pair that beat the direct pair by milliseconds. Role conflicts on ICE restart are resolved by the tiebreaker in ICE-CONTROLLING/ICE-CONTROLLED, and a buggy implementation can end up with both sides controlled and nothing nominated.

What to do

If you write server-side ICE, implement both nomination styles and the role-conflict tiebreaker. When a pair selection looks premature, log all pairs' state and nominated flags — the good pair is usually there in succeeded state, just not chosen.

The Impossible Call · Module 3, Lesson 4