§ 4.1  Module 4 — The Browser Gets a Dial Tone

The Plugin Decade and the $68M Bet

Google paid $68 million so the web could say hello.— Epigraph to Module 4

4.1.1The web that couldn't talk: Flash, Java, and Skype

Browsers could capture a camera long before they could hold a conversation. Macromedia's Flash Player 6 (2002) exposed microphone and webcam capture to any web page, and the companion Flash Communication Server MX (2002) carried the media over RTMP — the Real-Time Messaging Protocol — on TCP port 1935. The architecture had two structural flaws. Every stream transited Adobe-licensed servers, so cost scaled with minutes; and RTMP rode TCP, so a single lost segment stalled every byte behind it while the kernel retransmitted. That head-of-line blocking is precisely why Module 2 built real-time media on UDP: a late packet is worth less than a lost one. Adobe — which had acquired Macromedia in December 2005 — knew it, and shipped RTMFP in Flash Player 10 (2008): a UDP-based, peer-to-peer secure media protocol built from technology acquired with the startup amicima, later documented by its designer Michael Thornburgh as RFC 7016 (November 2013). RTMFP-era Flash was genuinely capable — Chatroulette, launched in November 2009 by seventeen-year-old Andrey Ternovskiy, ran pairwise video chat for millions on it — but it remained a proprietary black box on a security-patch treadmill, controlled by one vendor and installed as a plugin the browser merely hosted.

Java applets, the other candidate runtime, had shipped in Netscape Navigator 2.0 in 1996 with the same promise. For conversational media they never got traction — JVM startup measured in tens of seconds, audio APIs varied by vendor, every session began with a security dialog — and the companies actually selling web conferencing routed around the browser entirely: WebEx (founded 1996 by Subrah Iyar and Min Zhu, bought by Cisco for $3.2 billion in 2007) simply asked users to install a native client.

Skype was the decade's triumph and its anti-pattern in one binary. Launched on August 29, 2003 by Niklas Zennström and Janus Friis with the Estonian engineers of Kazaa — Ahti Heinla, Priit Kasesalu, Jaan Tallinn — it demonstrated at consumer scale everything this course has treated as hard: an overlay of user-run supernodes for rendezvous, aggressive NAT traversal, adaptive wideband audio. By TeleGeography's 2010 estimate Skype carried roughly an eighth of the world's international calling minutes. And none of it composed with anything else: the protocol was deliberately obfuscated, the client closed, the network an island. You could not embed a Skype call in a page, bridge it to a standard endpoint, or audit what it sent. When Microsoft bought Skype for $8.5 billion on May 10, 2011 — three weeks before Google open-sourced WebRTC — the contrast in strategies could not have been sharper: one company bought the walled garden; the other was about to raze the wall.

4.1.2Buying the crown jewels: GIPS and On2

Global IP Solutions — founded in Stockholm in 1999 as Global IP Sound — was the arms dealer of the VoIP boom. Its licensed components ran inside early Skype, Yahoo! Messenger, AOL, WebEx, IBM Lotus Sametime, and Tencent's QQ. Four assets mattered. First, an acoustic echo canceller: the signal-processing stage that subtracts the far end's loudspeaker audio from your microphone so a call does not feed back — the heart of what Module 6 studies as the voice pipeline. Second, NetEQ, an adaptive jitter buffer that fused packet reordering, time-stretching, and loss concealment into one component and could shrink buffering delay below what fixed designs paid. Third and fourth, two codecs: iSAC, an adaptive wideband speech codec, and iLBC, a narrowband codec engineered so each 20–30 ms frame decoded independently, published through the IETF as RFC 3951 and its RTP payload format RFC 3952 (December 2004). Skype's earliest releases licensed GIPS audio before building its own SILK codec (shipped 2009) — SILK that would later fuse with Xiph.Org's CELT to become Opus (RFC 6716, September 2012), the codec that ultimately retired both iSAC and iLBC.

On2 Technologies supplied the other half. Founded in 1992 as the Duck Corporation, On2 sold the TrueMotion line of video codecs; its VP3 had been open-sourced in 2001 and became Theora, and its VP6 was, in a neat irony, the codec inside most Flash video. Google announced the acquisition on August 5, 2009, closed it on February 19, 2010 for roughly $124.6 million, and on May 19, 2010 released VP8 royalty-free as the video half of WebM (VP8 video plus Vorbis audio in a Matroska-derived container; the bitstream was later documented as RFC 6386, November 2011).

Seen together, the two purchases — driven inside Google by product manager Serge Lachapelle's Stockholm team — were a single strategic act: acquire the components whose per-unit license fees had made real-time communication a moat, then release them to make it a commodity. Free, they made the browser itself a viable endpoint — provided browsers could agree on how to expose them.

4.1.3June 2011: one codebase, two standards bodies

On June 1, 2011, Google engineer Harald Alvestrand announced the open-sourcing of WebRTC: the GIPS audio engine, NetEQ, echo cancellation, VP8 integration, and NAT traversal code, under a BSD license at webrtc.org. A code drop is not a standard, though, and the standardization split along a line the internet had drawn long before. The W3C — the body that standardizes what JavaScript can call — chartered its WebRTC Working Group in 2011, chaired by Alvestrand and Ericsson's Stefan Håkansson, to define the API surface: getUserMedia for capture and RTCPeerConnection for transport, culminating in the WebRTC 1.0 Recommendation of January 26, 2021. The IETF — the body that standardizes what crosses the network — chartered the rtcweb working group the same year to define the wire: ICE (RFC 8445) with STUN and TURN beneath it, mandatory DTLS-SRTP keying (RFC 5763/5764), the RTP usage profile (RFC 8834), the transport stack (RFC 8835), and the overview and security architecture (RFC 8825–8827) — the cluster published together in January 2021.

The split was not bureaucratic tidiness. An API binds only browsers and the JavaScript above them; the wire must interoperate between different browsers and with the existing telephony world — SIP trunks, gateways, conference bridges. Proof that the arrangement worked came on February 4, 2013, when Google and Mozilla jointly demonstrated the first interoperable call between Chrome 25 (beta) and Firefox Nightly: two independent implementations converging on the same RFCs. What the split could not prevent was seven years of warfare over the seam between the halves — whether to reuse SDP at all (the ORTC rebellion, a 2013 W3C community group led by Hookflash's Erik Lagerway and Robin Raymond and chaired by Microsoft's Bernard Aboba, whose object-oriented alternative shipped in Edge in 2015), and how many m= lines a multi-track call has (Plan B versus Unified Plan, settled for Unified Plan as Chrome's default only in M72, January 2019). Later lessons in this module walk that battlefield; here we need the treaty that made an API possible at all.

4.1.4JSEP: the browser does media, you do signaling

The first open question after the code drop was architectural: where does call-setup state live? The telephony tradition answered "in the protocol stack" — a SIP phone's state machine is baked into the device. WebRTC's earliest API followed suit: the 2011 PeerConnection prototype took a callback in its constructor and emitted opaque signaling strings the application was expected to ferry blindly, per the ROAP proposal (draft-jennings-rtcweb-signaling, 2011). The objection, pressed by Justin Uberti and formalized with Cullen Jennings in draft-uberti-rtcweb-jsep (January 2012, with Mozilla's Eric Rescorla later joining as editor), was that baking a signaling protocol into the browser would freeze it forever. Web services already spoke SIP, XMPP Jingle, and countless proprietary schemes; a browser opinionated about signaling would interoperate with none of them cleanly and could never be upgraded without a browser release.

JSEP — RFC 8829, January 2021 — inverts the design. The browser is a media engine with no signaling protocol at all. It will manufacture a description of what it proposes to send and receive (createOffer), manufacture a compatible reply (createAnswer), and apply descriptions to its own machinery (setLocalDescription, setRemoteDescription). The descriptions are SDP blobs following the offer/answer model of RFC 3264 — the pragmatic, endlessly contested choice, made largely because the SIP world already spoke it and gateways could translate. How an offer reaches the other peer is entirely, deliberately unspecified: WebSocket, HTTP poll, a data channel from a previous call, a QR code held up to a camera. The spec does not care, and that indifference is the feature. Your signaling layer can be upgraded, sharded, and secured on your schedule, not the browser's.

4.1.5The signalingState machine

JSEP's bargain has a price: if the application owns delivery, the browser must police order. It does so with an explicit state machine, exposed as RTCPeerConnection.signalingState and announced through signalingstatechange events. Six states exist: stable, have-local-offer, have-remote-offer, have-local-pranswer, have-remote-pranswer, and closed. A connection begins in stable — no negotiation in flight — and returns to stable every time an offer/answer exchange completes. Note what does not change state: createOffer and createAnswer are pure factories. Only the two set*Description calls move the machine.

The caller's happy path reads directly off the figure: setLocalDescription(offer) moves stable → have-local-offer; the peer's answer arrives over your signaling channel, and setRemoteDescription(answer) completes the round trip back to stable. The callee mirrors it on the right side. The pranswer states are telephony's fingerprint on the API: a provisional answer, descended from SIP's provisional responses (RFC 3262) and early media — think ringback audio flowing before anyone accepts the call. A pranswer parks the machine in have-*-pranswer until a final answer lands; browsers rarely see one in web-only applications, but gateways to SIP produce them routinely, and the validator you build below must handle them.

const pc = new RTCPeerConnection();
pc.onsignalingstatechange = () => console.log('→', pc.signalingState);

const offer = await pc.createOffer();   // pure factory: still 'stable'
await pc.setLocalDescription(offer);    // → 'have-local-offer'
signaling.send({ sdp: pc.localDescription });  // YOUR channel, not the browser's

const { sdp: answer } = await signaling.receive();
await pc.setRemoteDescription(answer);  // → 'stable'; negotiation complete

Listing 4.1 — One legal traversal of the machine, instrumented. Only the set*Description calls change signalingState.

Every other edge is an error. Apply an answer while stable, or call setLocalDescription(answer) when no remote offer is pending, and the promise rejects with InvalidStateError — the browser refusing to let your signaling bugs corrupt its media engine. Predict the machine before you automate it:

4.1.6Rollback, re-offers, and two conveniences

Because both peers may offer simultaneously — the glare condition that RFC 3264 inherited from a century of two people picking up the phone at once — JSEP provides an escape hatch. Applying a description of type rollback abandons the pending offer and returns the machine to stable, restoring the pre-offer state. Rollback is legal only while an offer is pending: in stable or either pranswer state, the API rejects it with InvalidStateError. The canonical use is the polite peer of perfect negotiation (the subject of a later lesson): holding a local offer when the remote's offer arrives, it rolls back its own, applies the remote offer from freshly stable ground, and answers. Note also the legal self-loops: while in have-local-offer you may setLocalDescription a revised offer, and a remote peer may likewise replace its pending offer — the machine stays put.

Two refinements complete the modern API. First, createOffer accepts options: { iceRestart: true } forces fresh ICE credentials (new ice-ufrag/ice-pwd) to re-gather a failing transport, and the legacy offerToReceiveAudio/offerToReceiveVideo flags survive from the era before addTransceiver gave precise, per-media-section control — prefer transceivers in new code. Second, setLocalDescription() may be called with no argument: the browser creates the appropriate description itself — an answer if the state is have-remote-offer, an offer otherwise — and applies it atomically. The implicit form is not mere sugar. In the explicit two-step, the SDP you apply was minted when createOffer ran and can be stale by the time setLocalDescription executes; the implicit form closes that race, which is why the perfect-negotiation pattern is built on it.

Everything in the rest of this module runs on this machine. Build its validator once, correctly, and every InvalidStateError you ever meet becomes a legible transition on Figure 4.1 rather than a mystery.

Problems for § 4.1Check your understanding

One machine to build and one history to keep straight: implement the JSEP signaling-state validator, then place the acquisitions, bodies, and design decisions where they belong.

¶Field quirks

What the specifications say, and what the browsers actually do, are two different documents. These are the divergences that cost production teams their weekends.