§ 1.3  Module 1 — When Voice Became Data

Best-Effort and Its Discontents: Why TCP Can't Carry a Call

The circuit promised everything and scaled to nothing. The datagram promises nothing — and never breaks its word.

1.3.1Four ways a datagram can disappoint

The circuit-switched telephone network gave a call one dedicated time slot, 8,000 times a second, end to end. Delay was fixed by geography, ordering was physical — a wire cannot reorder — and loss meant the call dropped entirely, a rare and unambiguous event. Packet switching replaces these certainties with four random variables, and everything in this course is ultimately a scheme for managing them.

Latency is the one-way transit time of a packet, and for voice the number that matters is mouth-to-ear: from pressure wave at the speaker's lips to pressure wave at the listener's eardrum. Network latency decomposes into propagation (bounded by the speed of light in fiber, roughly 200 km per millisecond), serialization (negligible at modern line rates), processing at each router, and — the volatile term — queueing, the time spent waiting behind other packets in router buffers. Propagation is constant for a given path; queueing is where the trouble lives.

Jitter is the variation in that delay. Packets carrying a steady 20 ms cadence of PCM-derived audio frames leave the sender like a metronome and arrive like rain: 40 ms, 41 ms, 58 ms, 40 ms. A loudspeaker, however, consumes samples at exactly 48,000 per second and will not wait. The receiver must therefore hold arriving packets in a jitter buffer deep enough to smooth the variation — buying regularity with delay, milliseconds it must repay from the latency budget of the next section.

Reordering happens when a later packet overtakes an earlier one — load-balanced parallel paths and mid-flight route changes are the usual culprits. With packets spaced 20 ms apart, packet n+1 overtakes packet n only when their delays differ by more than 20 ms, so reordering is really jitter past a threshold. It is rarer than the other pathologies but ruinous if the receiver assumes it cannot happen.

Loss is the network's pressure valve. Routers under congestion do not slow packets down politely; they drop them. Wi-Fi and cellular links add their own losses. For 20 ms audio frames, one isolated loss is 20 ms of missing sound — about half a phoneme — which, we will see, a good receiver can conceal almost inaudibly. What no receiver can conceal is the response to loss that TCP imposes.

Surviving hardwarephotographed 2007

1.3.2NVP: proof of concept, proof of a gap

The first people to face these four pathologies with live speech did so astonishingly early. In 1973, ARPA's Network Secure Communications program set out to send encrypted voice over the ARPANET, and Danny Cohen at USC's Information Sciences Institute designed the Network Voice Protocol (NVP) to carry it. By 1974, Cohen's group and James Forgie's team at MIT Lincoln Laboratory were holding real-time conversations across the country, with linear predictive coding squeezing speech to a few kilobits per second so it could fit through the ARPANET's 50 kbit/s trunks. Cohen wrote the protocol up as RFC 741 in 1977.

NVP's deepest lesson was architectural. The ARPANET's IMP subnet offered hosts a reliable, ordered delivery service — and Cohen had to ask for the raw, "uncontrolled" mode instead, because waiting for the subnet's retransmissions froze the audio. Timeliness beat reliability; a maxim usually paraphrased from Cohen's work puts it plainly: for voice, a packet that arrives late is no better than a packet that never arrives. That experience became one of the arguments, pressed by Cohen among others, for the 1978 decision to split the monolithic transmission protocol into IP below and TCP above — which is precisely what made room for UDP two years later. Real-time media did not merely benefit from the TCP/IP split; it helped cause it.

What NVP conspicuously lacked was a standard framing layer. Each implementation improvised its own way to mark which vocoder produced a packet, where it belonged in time, and what had been lost. Every packet-audio system for the next two decades — including the Mbone tools of the early 1990s — reinvented that framing, incompatibly. The gap was not closed until January 1996, when Henning Schulzrinne, Stephen Casner, Ron Frederick, and Van Jacobson published RTP as RFC 1889, giving every media packet a common header with a timestamp, a sequence number, and a payload type. That story is the next lesson's; this lesson explains why RTP had to run over UDP.

1.3.3The 150-millisecond budget

How much latency can a conversation tolerate? The canonical answer is ITU-T Recommendation G.114: keep one-way mouth-to-ear delay at or below 150 ms and nearly all users are satisfied; between 150 and 400 ms, interactivity degrades measurably; beyond 400 ms, G.114 declares the delay unacceptable for general network planning. The failure mode is not audio quality — the words arrive intact — but turn-taking. Conversation runs on sub-second cues: pauses, interjections, the half-breath before a reply. Push round-trip delay past roughly half a second and speakers begin talking over one another, then retreat into stilted, walkie-talkie exchanges. Anyone who has spoken over a geostationary satellite link — about 240 ms one way, just for the 71,000 km round trip to orbit and back down — knows the rhythm.

The sobering exercise is to spend the budget on paper. Consider a well-built call from San Francisco to New York:

• Capture, 20 ms. The encoder cannot compress a frame until the last sample of that frame has been spoken. One Opus frame is 20 ms; the first sample in it waits the full 20.
• Encode, ≈7 ms. Opus adds 6.5 ms of codec lookahead; the arithmetic itself is fast.
• Propagation and queueing, ≈55 ms. The great-circle distance is about 4,100 km — roughly 21 ms in straight fiber — but real routes wander, and every router adds queueing.
• Jitter buffer, ≈45 ms. Enough to absorb typical delay variation; adaptive buffers shrink and grow with measured jitter.
• Decode, ≈3 ms. Cheap.
• Playout, ≈10 ms. The operating system's mixer and the DAC's own buffer.

Total: about 140 ms — the budget nearly exhausted by a healthy call on a well-behaved path, with nothing yet spent on recovering from loss. This is the arithmetic that condemns TCP: every millisecond a transport spends being "reliable" must be paid out of a budget that is already almost gone.

1.3.4TCP's cure is the disease

TCP (RFC 793, September 1981, edited by Jon Postel) presents applications with a single, perfectly ordered, perfectly complete stream of bytes, and it maintains that abstraction with two mechanisms — retransmission and in-order delivery — that are individually sensible and jointly fatal to conversation.

Retransmission is slow by construction. A sender cannot resend what it does not know is lost. Loss is detected either by a retransmission timeout — RFC 6298 recommends a minimum RTO of a full second; Linux floors it at 200 ms — or, faster, by three duplicate acknowledgments (RFC 5681). Walk the fast path with real numbers: RTT of 60 ms, one 20 ms voice frame per segment. The segment sent at t = 0 is lost. The three following segments, paced 20 ms apart, reach the receiver and each provokes a duplicate ACK; the third arrives back at the sender near t = 120 ms. The retransmission lands another one-way trip later, around t = 150 ms — and that is the optimistic path, on a good route, with steady traffic to generate duplicate ACKs. The timeout path is worse by design.

Head-of-line blocking multiplies the damage. TCP's contract is a byte stream in order, enforced by cumulative acknowledgment. When one segment is missing, every byte behind it may be sitting in the receiver's kernel buffer — arrived, checksummed, complete — and the socket API will not release a single one until the gap is filled. One lost 20 ms frame therefore does not cost 20 ms of audio; it freezes the stream for the entire retransmission delay, then delivers a burst. The player experiences silence, then a sprint. Worse, an adaptive jitter buffer that has once observed a 200 ms stall must grow to cover the next one: TCP converts occasional loss into permanent extra latency. The buffer remembers.

Bufferbloat makes every number above worse. Router and modem buffers grew for decades because memory was cheap and drops looked bad on benchmarks. TCP's loss-based congestion control fills any buffer it meets before it ever slows down, so a single bulk upload can turn a home router into a standing queue of several seconds — through which your voice packets must also pass. Jim Gettys named the pathology bufferbloat in late 2010 after watching his own cable connection; Kathleen Nichols and Van Jacobson answered with the CoDel queue discipline in 2012 (RFC 8289, 2018). The deep lesson for this course: on the modern Internet, delay, not loss, is the early signal of congestion — which is why WebRTC's bandwidth estimator watches delay gradients rather than waiting for drops. Module 6 returns to this at length.

1.3.5UDP: the escape hatch

The escape route is almost comically small. RFC 768 — Jon Postel, August 28, 1980, three pages — defines UDP's entire contribution as an 8-byte header: source port, destination port, length, checksum. It adds exactly two things to raw IP: multiplexing among applications and an integrity check. No retransmission, no ordering, no pacing, no promises. This poverty is the point. Because UDP guarantees nothing, it never spends the latency budget without permission; every policy decision — repair this loss or conceal it? wait for that straggler or move on? — is returned to the application, the only party that knows a frame's playout deadline.

With the deadline in hand, the receiver can make per-packet choices no in-order byte stream permits. An isolated lost frame is patched by concealment. A frame that can be re-fetched before its deadline — on a short path, with time to spare — can be selectively retransmitted, a trick formalized years later in RTCP feedback. A frame that arrives after its deadline is discarded without regret, because playing it would shift the entire stream later for everyone. The exercise below makes this concrete: on the same 50-packet trace, UDP delivery loses 2% of the audio and never exceeds 75 ms of delay, while TCP-style in-order delivery holds three packets past the entire 150 ms budget to save one.

This is why the whole real-time lineage stacks on UDP: RTP in 1996, and in time the full WebRTC transport — ICE to punch a path through NATs, DTLS-SRTP to encrypt it, SCTP data channels to carry arbitrary data with tunable reliability. Even QUIC, TCP's designated successor for the web, is built on UDP for the same reason: the only transport flexible enough for the future was the one that had promised nothing since 1980. The next lesson opens the packet itself: RTP, the header that finally gave every frame a clock, and RTCP, the channel that reports what the network did to it.

Problems for § 1.3Check your understanding

One coding problem — compute the TCP-versus-UDP tradeoff yourself on a hostile packet trace — and one quiz on the latency budget.

// Fifty 20 ms audio frames on a transcontinental path.
// sendTime/recvTime in ms; recvTime === null means the packet was lost.
const trace = [
  { seq: 0,  sendTime: 0,   recvTime: 42  },
  { seq: 1,  sendTime: 20,  recvTime: 60  },
  // ...
  { seq: 23, sendTime: 460, recvTime: 535 },  // delayed: arrives after seq 24
  { seq: 24, sendTime: 480, recvTime: 521 },
  // ...
  { seq: 30, sendTime: 600, recvTime: null }, // lost
  { seq: 31, sendTime: 620, recvTime: 662 },
  // ...
];

Listing 1.3 — Shape of the canned trace used by Problem 1. The full 50-packet array is provided to your functions as their first argument.

¶Field quirks

What the specifications say, and what the browsers actually do, are two different documents. These are the divergences that cost production teams their weekends.