§ 3.3 Module 3 — The Broken Internet: NAT and the Art of Hole Punching
In these situations, it is necessary for the host to use the services of an intermediate node that acts as a communication relay.— RFC 5766, April 2010
By the end of this lesson
Historical note
TURN is the standards body admitting defeat. STUN (RFC 3489, March 2003 — Rosenberg, Weinberger, Huitema, Mahy) let a host behind a NAT discover its public mapping, and for “cone” NATs that was enough to punch a hole. But when the NAT allocates a fresh mapping per destination — address-and-port-dependent mapping, in the later vocabulary of RFC 4787 (Audet & Jennings, January 2007) — the address STUN reports is useless: it is the mapping toward the STUN server, not toward the peer. Skype, launched in August 2003 by Niklas Zennström and Janus Friis with the Kazaa engineers Ahti Heinla, Priit Kasesalu, and Jaan Tallinn, solved this commercially by conscripting users’ own machines with public addresses as “supernodes” that silently relayed other people’s stubborn calls — brilliant, proprietary, and paid for with other people’s bandwidth. The open equivalent took far longer: Jonathan Rosenberg’s relay drafts began in 2003 (draft-rosenberg-midcom-turn) and emerged seven years later as TURN, RFC 5766 (April 2010 — Mahy, Matthews, Rosenberg), revised as RFC 8656 (February 2020 — Reddy, Johnston, Matthews, Rosenberg). Its premise is blunt: when hole punching fails, rent a public middleman and pay for every byte. One war story sets this lesson’s agenda: an enterprise rollout where the customer’s network blocked UDP outright and blocked TCP to port 3478, so only turns: over TLS on port 443 — traffic indistinguishable from HTTPS — passed the corporate proxy. The retro found that 100% of the failed calls had gathered no 443/TLS relay candidate. The fix was one line of configuration.
Lesson 3.2 ended with a taxonomy: RFC 4787 classifies a NAT by its mapping behavior (does the same internal socket get the same public mapping for every destination?) and its filtering behavior (which remote sources may send inbound through that mapping?). Hole punching, which ICE will systematize in the next lesson, relies on each side learning a public address for the other that will still be valid when packets actually fly.
Run the grid and one cell fails absolutely. If peer A’s NAT is endpoint-independent in its mapping, the server-reflexive address A learned from STUN is exactly the address A will present to B, and B’s first packets toward it open B’s own filter on the way out. This holds even if B’s NAT is “symmetric” — address-and-port-dependent mapping: B’s outbound checks toward A’s stable address create a fresh mapping, A sees the new source port, and ICE simply promotes it as a peer-reflexive candidate. One-sided symmetric NATs usually connect. But when both sides have address-and-port-dependent mapping, neither side can tell the other an address that will be valid for the real conversation: every prediction is stale on arrival. No cleverness fixes this — the information needed does not exist outside the two NATs. The only escape is a third party with a genuinely public address that both sides can reach outbound: a relay. That is the computation behind the phrase relay of last resort, and it is why any production deployment that omits TURN is quietly dropping the double-symmetric cell of the grid, plus every network that blocks UDP altogether.
TURN is an extension of STUN: same 20-byte header, same attribute encoding, same transaction model, new methods. A client begins by sending an Allocate request to the server (default port 3478, like STUN). The first attempt fails by design: the server answers 401 with a REALM and a NONCE, the client computes a message-integrity key from its credentials, and retries. On success the server reserves a relayed transport address — a public UDP socket on the server itself, say 203.0.113.15:49152 — that will forward traffic on the client’s behalf. That reservation is the allocation. It is soft state: the response carries a LIFETIME (600 seconds by default), and the client must send Refresh requests before expiry or the server reclaims the port. A Refresh with lifetime 0 deletes it politely.
An allocation alone relays nothing, because an open public socket that forwarded packets from anyone would be a reflection-attack machine. The client must install a permission for each peer it expects: a CreatePermission request naming the peer’s IP address. Permissions live 300 seconds and must likewise be refreshed; once a permission expires, the server silently discards that peer’s packets. With a permission in place there are two ways to move data. The simple way is a pair of STUN messages: the client wraps outgoing payloads in Send indications naming the peer (XOR-PEER-ADDRESS) and carrying the payload (DATA); the server unwraps them onto the wire and wraps inbound packets in Data indications going back. Indications need no acknowledgment, but each one costs roughly 36 bytes of framing — a 20-byte STUN header plus the two attributes — on every single packet, forever.
The efficient way is a channel. A ChannelBind request associates a 16-bit channel number (0x4000–0x4FFF, RFC 8656 §12) with one peer address; the binding lasts 600 seconds and installs or refreshes the matching permission as a side effect. Thereafter both directions use ChannelData messages: 2 bytes of channel number, 2 bytes of length, then the raw payload — 4 bytes of overhead instead of 36. For media flowing 50 packets a second in each direction, that difference is real money, which is a theme we will return to. Note one asymmetry that surprises people: whatever transport the client uses to reach the TURN server, the relayed leg from server to peer is always plain UDP under RFC 8656 (TCP allocations exist separately, in RFC 6062, and are rare in WebRTC practice).
A quick mid-lesson check before we climb the transport ladder.
Check your understanding
Auto-graded check
3 auto-graded questions with an explanation for every wrong answer. Requires JavaScript. (m3-l3-check1)
Anything in RTCIceServer.credential is readable by the user, by any script on the page, and by anyone who opens devtools; it is also recoverable from pc.getConfiguration(). A long-lived shared secret is scraped within weeks of shipping and your relay becomes free bandwidth for someone else's file transfers or, worse, a laundering hop. The bill arrives before the alert does, because you are almost certainly not monitoring allocations by credential.
Use time-limited credentials: the coturn REST scheme where username is <unix-expiry>:<userid> and credential is the base64 HMAC-SHA1 of that username under a server-side static-auth-secret. Issue them from your authenticated signaling endpoint with a lifetime of minutes, and alert on relayed bytes per credential.
Everything above assumed the client can reach the TURN server at all. Hostile networks defeat this in layers, and the configuration answer is a ladder of URIs, standardized in RFC 7065 (November 2013 — Petit-Huguenin, Nandakumar, Salgueiro, Jones). turn:host:3478?transport=udp is the happy path. Networks that drop all outbound UDP — a common corporate posture — fall through to turn:host:3478?transport=tcp. Networks fronted by a proxy that permits outbound TCP only to port 443, and often only when it looks like TLS, leave exactly one rung: turns:host:443?transport=tcp, TURN inside a TLS session on the HTTPS port. (Absent an explicit port, turn: defaults to 3478 and turns: to 5349.) That last rung is what saved the enterprise deployment in the opening note: from the proxy’s point of view the call became one more long-lived HTTPS connection. The rungs are ordered by quality as well as reachability — a TCP or TLS leg to the relay reintroduces head-of-line blocking for the media riding it, so you take the highest rung the network allows.
In the browser this ladder is just the iceServers list handed to RTCPeerConnection. Listing every rung is standard practice; ICE gathers relay candidates from each and its priority rules keep them at the bottom of the pile unless nothing else survives.
const pc = new RTCPeerConnection({
iceServers: [
{ urls: 'stun:stun.example.com:3478' },
{
urls: [
'turn:relay.example.com:3478?transport=udp',
'turn:relay.example.com:3478?transport=tcp',
'turns:relay.example.com:443?transport=tcp',
],
username: '1760812345:alice', // ephemeral, see § 3.3.4
credential: 'nGh2k…base64…',
},
],
// Drill mode: force relay to prove TURN works BEFORE a customer needs it.
// iceTransportPolicy: 'relay',
});
Listing 3.3.1 — The canonical hostile-network configuration: every rung of the ladder, plus the relay-only test switch.
iceTransportPolicy: 'relay' deserves respect as a testing tool. With it set, ICE discards host and server-reflexive candidates and will use relayed candidates only; with no working TURN server configured, gathering completes with zero candidates and the connection can never form. Running your integration tests with relay-only policy is the cheapest way to discover a broken TURN fleet before the 10–20% of users who genuinely need it do. It is also a deliberate product feature: “hide my IP address from the far end” in privacy-sensitive applications is implemented as exactly this policy.
Interactive 3D instrument
The relay of last resort — allocation, permission, framing, transport
A 3D instrument you drive yourself, one variable at a time. It needs JavaScript and WebGL, so it is not shown in this static copy of the page.
Because relayed bytes cost the operator money, every allocation is authenticated. The base mechanism is STUN’s long-term credential scheme (RFC 8489 §9.2): username, realm, and password combine into a key — classically MD5 of username:realm:password — that HMACs each request’s MESSAGE-INTEGRITY attribute, with the server’s NONCE preventing replay. The name “long-term” is the problem. A WebRTC application must place the username and credential into JavaScript running in the user’s browser, where “view source” defeats it; a static TURN password shipped to every client is best understood as public, and strangers will happily relay their traffic on your bill.
Production systems therefore mint ephemeral credentials, following a convention published by Justin Uberti in 2013 (draft-uberti-behave-turn-rest, “A REST API for Access to TURN Services”) and implemented by coturn — Oleg Moskalenko’s open-source server, itself descended from the rfc5766-turn-server project — as use-auth-secret. The trick requires no protocol change: your application server and the TURN server share one secret; the username is an expiry timestamp (optionally timestamp:userid), and the password is base64(HMAC-SHA1(secret, username)). The TURN server can verify any such pair statelessly, and a leaked credential dies at its timestamp. Hygiene follows directly: issue credentials from an authenticated endpoint at call-join time, scope lifetimes to hours not months, never commit the shared secret to client code or version control, and rotate it like any other key. When a browser shows you a prompt-free way to test a relay — the well-known Trickle ICE sample page — remember that anything you paste there stays as public as you made it.
Plain TCP on port 443 is blocked by TLS-inspecting proxies that expect a ClientHello and drop anything else, so turn:host:443?transport=tcp fails exactly where you most need it. Real TLS via turns: works. Two further constraints: browsers route TURN/TCP through a configured HTTP proxy but cannot proxy TURN/UDP at all, and turns: with an IP literal fails certificate validation and produces no relay candidates with no error surfaced to JavaScript.
Configure all four in order of preference: turn:host:3478?transport=udp, turn:host:3478?transport=tcp, turns:host:443?transport=tcp. Use a hostname with a publicly trusted certificate. Then verify the TLS path actually works by testing with UDP blocked at your firewall — iceTransportPolicy: 'relay' alone does not test it.
How often does the last resort actually fire? Consistently, across published operator measurements, on the order of 10–20% of production sessions: the double-symmetric grid cell, plus UDP-hostile corporate and hotel networks, plus carrier-grade NAT on mobile. That number turns TURN from a protocol detail into a line item. A relayed call routes its media — both directions — through your server: at a typical 1 Mbps-per-direction video call, every 100 concurrent relayed calls demand roughly 200 Mbps of sustained relay throughput, and cloud egress at that rate is often the single largest infrastructure cost of a calling product. This is why mature teams put TURN percentage on a dashboard next to revenue: a sudden rise means a network path regressed (or a big customer deployed a stricter proxy); a sudden fall to zero usually means your relay fleet is down and the 10–20% are failing silently. The measurement comes from getStats: find the succeeded, nominated candidate-pair, look up its localCandidateId, and check whether that candidate’s candidateType is 'relay' (the relayProtocol field even tells you which rung of the ladder — udp, tcp, or tls — carried it).
async function callUsedRelay(pc) {
const stats = await pc.getStats();
for (const s of stats.values()) {
if (s.type === 'candidate-pair' && s.state === 'succeeded' && s.nominated) {
const local = stats.get(s.localCandidateId);
return !!local && local.candidateType === 'relay';
}
}
return false;
}
Listing 3.3.2 — The one-liner behind every TURN-percentage dashboard.
The economics make people nervous about a different question: if all my media flows through a rented middlebox, can its operator watch my calls? No — and it is worth being precise about why. The keys that protect WebRTC media are negotiated end-to-end by DTLS-SRTP: the DTLS handshake runs between the two peers, through the relay, and TURN sits entirely below it in the stack. What the relay forwards is SRTP ciphertext (and DTLS records) it has no keys for. A compromised TURN server is not harmless — it sees metadata: both parties’ transport addresses, session timing, and traffic volume — but the media itself is opaque. This is also why lesson 7 will need a different tool (SFrame and insertable streams) for the SFU case, where the server, unlike TURN, deliberately terminates the encryption.
A common error
“Routing calls through a TURN server means the provider can record them.” It cannot. TURN relays the encrypted transport — SRTP ciphertext and DTLS records — and never participates in the DTLS handshake that derives the media keys; those exist only at the two endpoints. Confusing TURN (a dumb, blind pipe) with an SFU (which terminates DTLS-SRTP per hop and can read encoded frames) is the root of most WebRTC privacy misjudgments, in both directions: people distrust relays that cannot see anything and trust media servers that see everything.
One coding drill against the corporate-firewall war story, and one scenario matrix on when the last resort actually fires.
Lab 1
Survive the Corporate Firewall
Graded in the browser against 4 assertions; the editor and harness require JavaScript.
Check your understanding
Auto-graded check
5 auto-graded questions with an explanation for every wrong answer. Requires JavaScript. (m3-l3-ex2)
What the specifications say, and what the browsers actually do, are two different documents. These are the divergences that cost production teams their weekends.
RFC 8656 gives allocations a 600 s default lifetime, permissions 300 s, and channel bindings 600 s, each refreshed independently. Browsers handle all of this. Native clients and SFUs frequently refresh the allocation and forget the permission, which produces the worst failure mode in the catalogue: at the five-minute mark media stops in one direction while ICE still reports connected, consent checks still pass on the other direction, and nothing logs an error.
If you own a TURN client, refresh permissions at 240 s regardless of traffic. If you are debugging, the tell is one-way media beginning at almost exactly 5:00 into a relayed call — check inbound-rtp.packetsReceived flat while transport.bytesSent keeps rising.
Every relayed byte is received and re-sent, so a relayed 1 Mbit/s bidirectional call costs 4 Mbit/s of server traffic. ChannelData framing adds 4 bytes per packet once the channel is bound (36 bytes before that, using Send/Data indications), and over TURN/TLS you additionally pay TCP and TLS record framing. At 1000 concurrent relayed video calls you are looking at multiple gigabits of egress, which is usually the dominant line item in a WebRTC infrastructure bill.
Measure your relay rate as a first-class SLI, and place TURN servers close to users so the doubled traffic is at least short-haul. Note that iceTransportPolicy: 'relay' is also the only reliable way to test TURN — without it, host and srflx candidates succeed in your office and you never discover that your relay has been misconfigured for a month.
The Impossible Call · Module 3, Lesson 3