§ 1.4  Module 1 — When Voice Became Data

RTP: A Clock in Every Packet

RTP does not address resource reservation and does not guarantee quality-of-service for real-time services.— RFC 1889, January 1996

The previous lesson ended in a corner: UDP is the only transport that respects a deadline, and UDP gives you nothing — no ordering, no timing, no identity. A datagram arrives as an anonymous bag of bytes. RTP's answer is not to add guarantees (the epigraph above is the protocol admitting so in its own abstract) but to add metadata: twelve bytes that let the receiver reconstruct, in software, everything the copper circuit used to provide in physics. This lesson takes those twelve bytes apart, one field at a time, because every later topic in the course — jitter buffers, bandwidth estimation, simulcast, end-to-end encryption — is built on precise knowledge of what sits where.

1.4.1Twelve bytes, read byte by byte

The RTP fixed header (RFC 3550 §5.1) is twelve bytes, big-endian throughout. The first byte packs four small fields: a 2-bit version, always 2 since 1996; a padding flag P, set when the payload is padded to a block size (SRTP ciphers use this); an extension flag X, announcing that a header-extension block follows the fixed header; and a 4-bit CSRC count CC. The second byte carries the marker bit M and a 7-bit payload type PT. Then come the three fields that do the real work: a 16-bit sequence number, a 32-bit timestamp, and the 32-bit SSRC — the synchronization source, a random identifier naming the stream itself.

The payload type is a lookup key, not a codec. Static assignments from RFC 3551 (July 2003) survive — PT 0 is still PCMU at 8 kHz, PT 8 PCMA — but modern codecs all live in the dynamic range 96–127, and the number-to-codec binding is negotiated per session in SDP through lines like a=rtpmap:111 opus/48000/2. Chrome habitually assigns Opus PT 111, but nothing on the wire requires it; a parser that hard-codes payload types is already broken.

The CSRC list exists for mixers: a server that sums three speakers into one stream sets CC=3 and lists the contributors' SSRCs so receivers can attribute the audio. Peer-to-peer WebRTC traffic almost always has CC=0 — which is precisely why parsers that forget to skip 4 × CC bytes pass every local test and then corrupt their first conference-bridge packet. Here is a real header, the same seventeen bytes you will parse in Problem 1:

80 ef 44 7a  d3 5c e3 a0  9e 1c b1 ae  78 0b e4 1f  6c
─┬ ─┬ ──┬──  ─────┬─────  ─────┬─────  ───────┬───────
 │  │   │         │            │              └─ payload (Opus TOC byte first)
 │  │   │         │            └─ SSRC = 0x9e1cb1ae
 │  │   │         └─ timestamp = 3,546,080,160 (48 kHz ticks)
 │  │   └─ sequence number = 17,530
 │  └─ 0xef: M=1 (talkspurt start), PT=111 (Opus, dynamic)
 └─ 0x80: V=2, P=0, X=0, CC=0

Listing 1.4 — An Opus packet's first seventeen bytes, annotated. Note the timestamp above 2³¹: a parser that assembles it with signed shifts will report a negative number.

1.4.2A clock in media ticks, not wall time

The timestamp is the most misread field in the header. It is not milliseconds, not Unix time, not NTP. It counts ticks of the media clock: the sampling clock of the payload it carries. For all video payload formats the rate is 90,000 ticks per second — a rate RTP inherited from MPEG's 90 kHz system clock (MPEG-1 Systems, 1993) and RFC 3551 made mandatory for video. For Opus, RFC 7587 (June 2015) fixes the RTP clock at 48,000 ticks per second regardless of the encoder's internal bandwidth, matching the 48 kHz sampling rate of the decoder's output.

The arithmetic is therefore exact and codec-specific. A 20 ms Opus packet advances the timestamp by 960 (48,000 × 0.020). A video frame at 30 fps advances it by 3,000. Two consecutive frames half a second apart differ by 45,000 ticks, always, on every machine, because the tick rate is a property of the payload format, not of anybody's wall clock. This is the design's quiet genius: the sender never has to own an accurate clock. It stamps packets with a counter derived from its sample counter, and the receiver recovers relative timing — enough to schedule playout and measure jitter — from arithmetic alone. The jitter buffer you will build in Module 5 consumes exactly these two numbers, sequence and timestamp, and nothing else.

Two consequences follow. First, timestamps from two different streams are incomparable: RFC 3550 §5.1 requires the initial timestamp to be random (as a plaintext-prediction defense, a point that matters once SRTP encrypts the payload), so an audio stream at tick 3,546,080,160 and a video stream at tick 90,000 say nothing about which was captured first. Synchronizing them — lip-sync — requires an out-of-band anchor pairing each stream's media clock with a common wall clock, and that anchor is the Sender Report of RTCP, the subject of the next lesson. Second, the timestamp and the sequence number advance independently. During Opus discontinuous transmission, silence produces no packets: the sequence number then advances by 1 per packet sent while the timestamp jumps by the duration of the silence. Conversely a large video frame split across ten packets carries ten consecutive sequence numbers and one timestamp, because all ten packets belong to the same sampling instant.

1.4.3Sixteen bits and the wrap

The sequence number increments by exactly one per packet sent, and RFC 3550 requires its initial value to be random too. Sixteen bits at 50 packets per second (typical 20 ms audio) wrap in under 22 minutes; high-bitrate video wraps in 2–3 minutes. Wraparound is not an edge case — it is a scheduled event on every call longer than a coffee break, and naive comparison fails at it: numerically, 65,535 > 0, but the packet numbered 0 comes after the one numbered 65,535.

The standard fix treats the 16-bit space as a circle. Sequence a is earlier than b when the forward distance from a to b, computed modulo 2¹⁶, is less than half the circle: in code, a !== b && ((b - a) & 0xffff) < 0x8000. On the circle, 65,535 → 0 is a forward step of 1, so the comparator gets the wrap right, at the price of ambiguity for two numbers exactly 2¹⁵ apart — which is why the comparator is only trusted within a window smaller than 32,768 packets, a safe assumption for any sanely sized jitter buffer. For bookkeeping that must span the whole call — loss statistics, retransmission indexes, SRTP's packet index — receivers instead unwrap: they count wraps and extend each 16-bit value into a monotone 32-or-64-bit one, choosing for each arrival the candidate closest to the previous extended value so that a late, reordered packet maps slightly backward instead of one full cycle forward. You will implement both in Problem 2.

1.4.4Header extensions: negotiated margins

The designers of 1996 left a door open, and WebRTC walked a warehouse through it. When the X bit is set, the fixed header (and any CSRC list) is followed by an extension block: a 16-bit profile marker, a 16-bit length in 32-bit words, then the data. RFC 8285 (October 2017, obsoleting RFC 5285 of 2008) standardized how to pack multiple small extensions into that one block. In the one-byte format — profile marker 0xBEDE, a number chosen as a pun on the Venerable Bede — each element is a single header byte holding a 4-bit ID (1–14) and a 4-bit length field encoding 1–16 data bytes as len − 1, followed by the data; zero bytes pad the block to a 32-bit boundary. The IDs mean nothing on their own: SDP binds them per session with a=extmap lines, exactly as dynamic payload types are bound with a=rtpmap.

Five extensions matter most in practice. The SSRC audio level (RFC 6464, December 2011) puts the talker's loudness, in −dBov, on every audio packet so a conference server can pick the active speaker without decrypting or decoding audio. abs-send-time, a 3-byte Google extension (URI http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time, 6.18 fixed-point seconds), stamps the true moment of transmission for delay-based bandwidth estimation. Its successor, the transport-wide sequence number (draft-holmer-rmcat-transport-wide-cc-extensions-01, 2015), adds a 16-bit counter spanning all streams on a transport, so a single feedback message can report the fate of every packet regardless of which SSRC carried it. Finally, mid (RFC 8843, January 2021) and rid (RFC 8852, January 2021) carry short ASCII tags binding a packet to its SDP media section and to its simulcast layer — identity glue for the demultiplexing that bundling many streams onto one UDP flow requires. Note what all five have in common: they annotate the transport layer of the call and are readable by intermediaries, which is why they live in the header (integrity-protected but not encrypted by SRTP) rather than in the encrypted payload.

1.4.5Payload formats: what the marker bit means

RTP deliberately stops at the payload boundary. For each codec, a separate payload format RFC defines how encoded frames map into packets — and, revealingly, what the marker bit means, because RFC 3550 left M's semantics to the profile. For Opus (RFC 7587), each packet carries exactly one Opus packet (2.5–120 ms of audio), and per the audio profile convention of RFC 3551 §4.1 the marker is set on the first packet of a talkspurt after a silence gap — a hint that lets the receiver reset its playout point. For VP8 (RFC 7741, March 2016), a payload descriptor precedes the bitstream with a start bit and an optional PictureID, and the marker is set on the last packet of a frame, telling the depacketizer the frame is complete and may be decoded. For H.264 (RFC 6184, May 2011), the payload is NAL units — aggregated several-to-a-packet with STAP-A or fragmented one-across-many with FU-A — and the marker again flags the last packet of an access unit. Same bit, three meanings; the payload type tells you which dialect of marker you are reading. The codecs themselves are Module 4's story; what matters tonight is that the twelve-byte header is codec-agnostic and everything after it is not.

Step back and look at what the Mbone generation actually built. A circuit gave three guarantees: bytes arrive in order, at a fixed rate, from a known party. RTP re-encodes each as a field — order becomes the sequence number, rate becomes the media-clock timestamp, identity becomes the SSRC — and demotes each from a guarantee to a claim that the receiver must enforce for itself. That demotion is the whole philosophy of Internet real-time media, and the rest of this course is the machinery of enforcement: RTCP to audit the claims (next lesson), jitter buffers to act on them, congestion control to keep them plausible. First, though, you should be unable to look at twelve bytes without seeing the fields. The problems below make sure of it.

Problems for § 1.4Check your understanding

Two implementations — the header parser/serializer every stack contains, and the wraparound-safe comparator every stack gets wrong once — followed by timestamp drills.

¶Field quirks

What the specifications say, and what the browsers actually do, are two different documents. These are the divergences that cost production teams their weekends.