§ 1.5  Module 1 — When Voice Became Data

RTCP: The Feedback Channel

Feedback is the property of being able to adjust future conduct by past performance.— Norbert Wiener, The Human Use of Human Beings, 1950

1.5.1The anatomy of a report

RTCP defines five packet types. Two of them carry the actual feedback: the Sender Report (SR, packet type 200) and the Receiver Report (RR, packet type 201). The other three are bookkeeping: SDES (202) carries source descriptions, BYE (203) announces departure, and APP (204) is an escape hatch for experiments.

A Sender Report is sent by anyone who has recently transmitted media, and it opens with a block of sender information: a 64-bit NTP-format wall-clock timestamp, the RTP timestamp corresponding to that same instant, and running counts of packets and octets sent. That NTP↔RTP pairing looks unremarkable and is in fact the entire basis of lip synchronization — we return to it in §1.5.4.

Both SRs and RRs then carry zero or more report blocks, one per remote source the reporter has heard from. Each block is a 24-byte verdict on one incoming stream (RFC 3550 §6.4.1):

• SSRC of source — whose stream is being judged;
• fraction lost — an 8-bit fixed-point fraction of packets lost since the previous report;
• cumulative number of packets lost — a signed 24-bit count since the beginning of reception;
• extended highest sequence number received — the 16-bit RTP sequence number plus, in the upper 16 bits, a count of sequence-number wraparounds;
• interarrival jitter — a smoothed estimate of packet-timing variance, in RTP timestamp units;
• LSR and DLSR — the middle 32 bits of the NTP timestamp from the last SR received from that source, and the delay since receiving it, in units of 1/65,536 second.

LSR and DLSR let the original sender compute round-trip time without any clock agreement: when a report arrives at time A (in the same middle-32-bits NTP format), the RTT is A − LSR − DLSR — total elapsed time minus the time the reporter sat on the reply. This is how WebRTC's roundTripTime statistic is measured, and the SR/RR pair as a whole is how a sender with a one-way media stream learns anything at all about the far end.

1.5.2The interarrival jitter estimator

The jitter field answers a question the loss counters cannot: is the network varying the delay it imposes? A receiver's jitter buffer must hold packets long enough to smooth that variance, so a rising jitter estimate is an early warning that either the buffer must grow or quality will drop — often before a single packet is lost.

RFC 3550 §6.4.1 defines it precisely. For two packets i and j, compare the spacing at arrival against the spacing at the source, both measured in RTP timestamp ticks:

D(i,j) = (R_j − R_i) − (S_j − S_i)

// R = arrival time converted to timestamp units
// S = the packet's RTP timestamp

J += (|D(i−1, i)| − J) / 16;   // on every packet, in arrival order

Listing 1.5 — The interarrival jitter estimator of RFC 3550 §6.4.1 and Appendix A.8.

Three details carry all the weight. First, units: R is a wall-clock arrival time converted to the media clock — for 48 kHz audio, milliseconds × 48. The estimate lives entirely in timestamp ticks; a value of 720 at 48 kHz means 15 ms. Second, the absolute value: jitter is a magnitude of variation, so early and late deviations both push it up. Third, the gain of 1/16: each new sample moves the estimate one-sixteenth of the way toward the instantaneous deviation, an exponentially weighted average that RFC 3550 chose to give "a good noise reduction ratio while maintaining a reasonable rate of convergence." The estimator runs on every packet in arrival order — including out-of-order ones, whose negative timestamp deltas produce large |D| spikes that then decay by 1/16 per packet.

1.5.3Counting loss without lying

A report block carries two loss numbers with different clocks. Cumulative packets lost is a lifetime figure: the number of packets expected minus the number actually received, since reception began. Expected is computed from sequence numbers — the extended highest sequence number seen, minus the first sequence number, plus one. Fraction lost is an interval figure: the loss during the period since the previous report, expressed as an 8-bit fixed-point fraction — the lost count times 256, divided by the expected count, truncated. A value of 64 means 64/256 = 25 percent.

The reason for carrying both is diagnostic. A high cumulative count with a fraction of zero describes an outage that is already over; a low cumulative count with a fraction of 64 describes trouble that started seconds ago. Senders that adapt — and every congestion controller descended from these reports, up to and including Google Congestion Control, does — care about the fraction; operators reading logs after the fact care about the cumulative.

Reordering is where naïve implementations lie. A packet that arrives after a higher sequence number has already been seen is late, not lost: it does not raise the extended highest sequence number, but it does increment the received count, so the computed loss (expected − received) drops back down. An implementation that instead marks a gap as permanently lost the moment it appears will report phantom loss on every reordered packet — and the network reorders more than you think. Two consequences of the arithmetic are worth memorizing: duplicated packets inflate the received count, which is why cumulative lost is a signed 24-bit field that can legitimately go negative; and fraction lost is defined to clamp at zero rather than wrap when duplicates outnumber losses in an interval.

Check your intuition before implementing this in the problem set.

1.5.4NTP↔RTP mapping and lip sync

Recall from §1.4 that RTP timestamps are media-clock ticks starting from a deliberately random offset — 48,000 per second for Opus audio, 90,000 per second for video, and never comparable across streams. Your microphone track's timestamp 1,843,200 and your camera track's timestamp 2,914,560,187 may describe the same instant. Nothing inside the RTP packets themselves can align them.

The Sender Report is the missing bridge. Its sender-information block is a single calibration point: at NTP wall-clock time X, my RTP clock for this stream read Y. Given one SR per stream, a receiver maps any RTP timestamp t to wall-clock time as ntp_ref + (t − rtp_ref) / clockRate, placing audio and video on one shared timeline. Playout then delays whichever stream is ahead — almost always audio, since video pipelines are slower — until the timelines agree. One question remains: which audio SSRC belongs with which video SSRC? That binding comes from SDES in the next section, via the CNAME both streams share.

Two practical notes. The sender's NTP clock does not need to be correct — synchronization only requires that both of a participant's streams are stamped from the same clock. And until the first SR arrives, a receiver simply cannot lip-sync; this is why fresh WebRTC calls occasionally show a flash of unsynchronized media in their opening second, and why SRs are sent early and often.

1.5.5Compound packets, SDES, BYE, and the 5% budget

RTCP packets never travel alone. RFC 3550 §6.1 requires every transmission to be a compound packet: it must begin with an SR (if the participant sent media recently) or an RR, and it must contain an SDES packet carrying a CNAME — the canonical name that identifies a participant stably across its SSRCs. SDES defines other item types (NAME, EMAIL, PHONE, LOC, TOOL, NOTE), relics of the MBone's open-directory culture; WebRTC sends only CNAME, generated as an opaque random string per RFC 7022 precisely so it identifies nothing but the session. CNAME is what lets the receiver in §1.5.4 know that audio SSRC 0x9A3F17C2 and video SSRC 0x517B2E90 are the same person, and therefore must be lip-synced to each other. A participant leaving cleanly appends a BYE packet — RTCPeerConnection.close() does this for you — so the far side can tear down state immediately instead of waiting for a timeout.

How often do reports flow? RFC 3550 §6.2 fixes the control-traffic budget at 5 percent of the session bandwidth, splitting it one quarter to active senders and three quarters to receivers, on the theory that a few senders matter to many receivers. Each participant computes its share, derives a reporting interval, randomizes it between 0.5× and 1.5× to avoid synchronized bursts, and respects a 5-second minimum (halved for a participant's first report). The elegance is in what this does at scale: the rule was written for MBone multicast sessions with thousands of listeners, and the interval stretches automatically as membership grows — a session of ten thousand still spends only 5 percent of its bandwidth on gossip, with each member reporting rarely. "Timer reconsideration," added in RFC 3550, recomputes the interval when membership changes so joins and leaves cannot stampede the channel.

For a two-party WebRTC call the arithmetic is gentler, but the 5-second minimum was still far too slow for one thing the modern web demands: reacting to loss now. The extended profile AVPF (RFC 4585, July 2006 — Ott, Wenger, Sato, Burmeister, Rey) relaxed the timing rules so a receiver may send early feedback — a NACK or a picture-loss indication — almost immediately, within the same average-rate budget. Reduced-size RTCP (RFC 5506, April 2009) then dropped the compound-packet requirement for such feedback, negotiated in SDP as a=rtcp-rsize, so a two-word NACK no longer drags a full SR and SDES along with it. Every WebRTC feedback mechanism you will meet in Module 5 — NACK, PLI, FIR, TWCC — is an RTCP feedback message riding on this pair of amendments. The 1996 design took its own advice: it adjusted future conduct by past performance.

Problems for § 1.5Check your understanding

Implement the receiver-side arithmetic of a report block over a canned arrival trace, then read a decoded SR/RR pair the way a troubleshooter would.

The quiz below refers to this decoded report pair, captured mid-call (Opus audio, media clock 48,000 Hz). Alice is sending; Bob is reporting on what he receives.

SR  from Alice  (sender SSRC 0x9A3F17C2)
    NTP timestamp   = 3,992,179,200.500 s   (since 1900)
    RTP timestamp   = 1,843,200             (audio, 48 kHz)
    packets sent    = 2,450    octets sent = 392,000

    [Alice's video SR, same NTP instant: RTP = 2,914,560,187 @ 90 kHz]

RR  from Bob, report block on SSRC 0x9A3F17C2
    fraction lost   = 64 / 256
    cumulative lost = 118
    ext. highest seq = 67,890
    jitter          = 720                   (timestamp units)
    LSR = 0x5F2A8000    DLSR = 131,072      (= 2.000 s)

¶Field quirks

What the specifications say, and what the browsers actually do, are two different documents. These are the divergences that cost production teams their weekends.