Start published video at 90% of its target bitrate (x-google-start-bitrate)

[sergeyphi]

Swift counterpart of livekit/client-sdk-js#1987 and livekit/rust-sdks#1226, reworked from the earlier opt-in draft after the review comment, then aligned (commit 2) with the connection-level, write-once design from livekit/client-sdk-android#973 that merged in livekit/client-sdk-js#2102 and livekit/rust-sdks#1430.

Problem

libwebrtc starts every new video send stream at roughly 300 kbps and ramps up from there, regardless of the encodings' maxBitrate. On a healthy network we measured about 30 s to reach a 2.3 Mbps ceiling, with the first 10-12 s at QP 39-42 — a visibly blocky picture. The ObjC API exposes no way to set that starting point, so the handle is libwebrtc's own SDP-level x-google-start-bitrate fmtp parameter.

Change

Same behaviour as JS and Rust, no public API:

• When a video sender is added, the start bitrate is derived from its encodings: sum of the active layers' maxBitrate, × 0.9, capped at 1 Mbps unless the track is a screen share, and no hint under 300 kbps (Transport.startBitrateKbps(targetBps:isScreenShare:)).
• One connection-level value. libwebrtc reads x-google-start-bitrate per m-section but applies it to the shared Call (WebRtcVideoSendChannel::ApplyChangedParams → SetSdpBitrateParameters), where RtpBitrateConfigurator holds one config per peer connection, so per-sender values were last-writer-wins in m-section order. The offer's value is the largest hint among its sending video sections whose a=msid track id is a recorded sender (Transport.connectionStartBitrateKbps(for:kbpsBySenderId:)), and it is declared on every video codec's fmtp (VP8/VP9/AV1/H264/H265) of every sending video section. Receive-only / inactive sections neither contribute nor receive it. An fmtp line is inserted for payloads that have none (VP8); an existing value is replaced.
• Written once per peer connection. libwebrtc re-reads the hint when the send codec changes and restarts a converged estimator if the value changed, and it retains start_bitrate_bps across network route changes (RtpTransportControllerSend::OnNetworkRouteChanged), so the munge runs only until the first offer carrying the hint is accepted by setLocalDescription (_hasAppliedVideoStartBitrate). A rejected munge retries on the next offer; a full reconnect builds a new Transport. Offers before any video is published (audio / data only) find no target and leave the latch unset.
• The munge is the last, optional entry of the set(localDescription:munging:) composition from Negotiate Opus stereo in the subscriber answer so stereo publications are no longer downmixed to mono when using custom renderers #1068, in both single- and dual-PC negotiation, built on the SDP module from Encapsulate SDP munging in a tested, line-preserving SDP module #1078. Transport.addTransceiver(with:transceiverInit:startBitrateKbps:) records each sender's hint before returning, so the offer libwebrtc requests can't be created without it; remove(track:) clears it. Both LocalParticipant publish paths (primary and backup codec) pass it.
• x-google-max-bitrate is deliberately not written (same Call-level promotion would cap the connection's total send bandwidth; libwebrtc's own TODO concedes it). Per-track caps stay in maxBitrateBps.
• New SDP helper SDPMediaSection.setFmtpParameter(_:value:forPayload:) (replace / append / insert, other parameters kept verbatim).
• Default degradationPreference by source, the other half of the JS PR, is already in main (Resolve default video degradation preference by track source, including the backup codec #1083), so nothing to do there.
• Drive-by, surfaced by review: the backup-codec publish path now computes its encodings with isScreenShare, as the primary path does. It had defaulted to the camera encoding since Multi-codec v2 #275, so a screen share's backup sender got camera bitrates; the new formula's screen-share branch made that visible.

Tests

SDPTests, TransportSDPMungeTests (formula, encodings sum, connection-level max with the sending filter, the munge on every sending section: codec case, replace/append/insert, rtx/audio/recvonly untouched, no-op identity, latch condition), VideoEncodingsTests (backup codec's screen-share encodings) and, against the shipped libwebrtc in TransportMungeFallbackTests: a send-only video section's msid carries the sender id, setLocalDescription accepts the munged offer (including the inserted VP8 line), and two offers on one peer connection — answered by a second local peer connection — where only the first carries the hint.

Results

These numbers are from the earlier, answer-side version of this branch (H.264 only, hint fixed at 1000 kbps), measured on iOS with a software H.264 encoder, 800x1120@30, single layer, maxBitrate 2.3 Mbps: the estimate opened at ~1000 kbps instead of ~370, peak QP dropped from ~42 to ~28, and the 2.3 Mbps ceiling was reached in ~13 s instead of ~31 s. Zero NACKs. The offer-side variant derives the identical 1000 kbps for that track, but I haven't re-measured it on a device yet — will add a log run.

Open questions

• Camera-then-screen-share: with write-once, a screen share published after the camera gets no new seed (the connection stays at the camera's ≤1000). Same trade-off as JS/Rust/Android — the estimator has converged by then anyway.
• Only exercised end-to-end with H.264 on iOS on a healthy network. The other codecs are covered by unit tests plus the libwebrtc acceptance test, not by a live session.

[xianshijing-lk]

I fixed similar problems for JS and Rust, JS PR is livekit/client-sdk-js#1987

Can we do something similar ?

[sergeyphi]

@xianshijing-lk I'm happy to follow your pattern and make a similar implementation here!

I wanted to be more conservative and have it as an opt-in option — but since you already have this logic applied to all streams automatically in JS/Rust, I'm more than happy to do the same thing here

Let me rework my PR, I'll re-submit it later

[sergeyphi]

@xianshijing-lk I reworked the implementation to align it with the existing Rust and JS implementation. Let me know if you have any other comments or suggestions. Thanks in advance!

[xianshijing-lk]

Hi @sergeyphi, thanks for updating the PR. Over the weekend, I recalled a few more technical details around the relevant WebRTC behavior, which also came up during a fairly deep review of a related Android PR: livekit/client-sdk-android#973

One structural point emerged from that review that I think applies directly here, along with a few other details we were able to confirm. I’m sharing them since they’re fairly deep in the libwebrtc internals and took us a couple of review rounds to fully pin down.

The short version: x-google-start-bitrate is a connection level knob, not a per sender (or track) one.

libwebrtc reads it per m-section, but doesn't apply it per m-section. In

WebRtcVideoSendChannel::ApplyChangedParams:

bitrate_config_ = GetBitrateConfigForCodec(send_codec()->codec);   // webrtc_video_engine.cc:1337

call_->GetTransportControllerSend()->SetSdpBitrateParameters(bitrate_config_);  // :1358

That lands in RtpBitrateConfigurator, which holds one BitrateConstraints for the entire PeerConnection. Every video m-section writes the same slot, last writer wins — and the order is just SDP m-section order.

Two consequences for the current implementation:

1. kbpsBySenderId needs to collapse to one value. Right now each sender's section gets its own number. The camera/screen-share split is what makes this bite: camera capped at 1000, screen share uncapped at (say) 2700, both in the same offer. Whichever section applies last seeds the single estimator — so either the screen share's value silently overrides the camera cap for the whole connection, or the camera clobbers the screen share and the exemption does nothing. Which one you get depends on m-section ordering, so it'll look fine in testing and flip in the field.

2. The munge should run once per peer connection, not on every offer. It's currently in the munging: list for every negotiate(). That's mostly harmless — but not always, and the failure mode is bad.

libwebrtc guards re-application three ways:

• Only re-reads the codec's fmtp when the send codec changes (webrtc_video_engine.cc:1338-1342, otherwise it forces start_bitrate_bps = -1)
• Ignores a value equal to the stored one — rtp_bitrate_configurator.cc:66-73, with the comment "setting the same remote description twice shouldn't restart bandwidth estimation"
• Returns -1 (no change) while retaining the real value in its stored config

The gap: if the value changes between offers and the codec changed too, it does restart a converged estimator. With per-sender values, publishing a screen share mid-session changes the last-writer value — so you can reset BWE on an established connection. Writing once removes the whole class of problem.

The part that makes "once" clearly correct.

And some more webrtc implementation details:
RtpBitrateConfigurator keeps start_bitrate_bps in its stored config, and RtpTransportControllerSend::OnNetworkRouteChanged re-applies it from GetConfig() (rtp_transport_controller_send.cc:390, with an RTC_DCHECK_GT(start_bitrate_bps, 0) right after). So a WiFi→cellular handover re-seeds the estimator from your hint automatically, no renegotiation. You genuinely only need to write it once; a full reconnect builds a new PeerConnection and seeds the new estimator again.

On Android that's a one-shot latch on the transport, set only after setLocalDescription accepts the munged SDP (so a rejected munge retries on the next offer rather than silently burning the one chance).

x-google-max-bitrate : you're right to leave it out, and it's worth writing down why, because it looks like the natural companion and someone will propose adding it. Same Call-level promotion turns a per-track cap into a ceiling on total send bandwidth: a default camera publish emits x-google-max-bitrate=2310, which then starves a concurrent 3 Mbps screen share. libwebrtc has a TODO conceding the behavior is wrong — "codec max bitrate should probably not affect global call max bitrate" (webrtc_video_engine.cc:1327). Per-track and per-layer caps belong in RTCRtpEncodingParameters.maxBitrateBps, which is genuinely scoped per encoding. Android used to write it and removed it in this PR; JS and Rust never did.

On which codecs: libwebrtc only reads GetBitrateConfigForCodec(send_codec()->codec) — the selected send codec. Writing to all of VP8/VP9/AV1/H264/H265 is harmless, but note it slightly widens guard #1: if the codec later switches (backup codec), the new codec's fmtp also carries a hint, so changed_params.send_codec is true and it re-reads. Harmless with a uniform value; a genuine BWE restart with per-sender ones. Another reason the single value matters more than the codec set.

Your open question on the 300 kbps floor: we kept it, same as Rust. Below that, seeding above real capacity costs more than the ramp it saves. I'd match Rust rather than JS here.

Sorry for the long msg, the tech is more complex than what I expected. Happy to walk through any of this.

[xianshijing-lk]

I am planning to work on the rust and JS PRs to align with Android today or tomorrow

[sergeyphi]

@xianshijing-lk thank you for the context, this is very helpful. Your concern makes sense. I guess it's a good timing that you were reviewing a very similar PR for Android.

Since you're going to align JS and Rust implementation, it only makes sense to align current PR with Android as well.

Let me re-work this implementation to adopt the Android design.

[xianshijing-lk]

Thanks. FYI, I am also working on the corresponding Rust and JS PRs:
Rust livekit/rust-sdks#1430
JS livekit/client-sdk-js#2102

[devin-ai-integration]

Devin Review found 1 new potential issue.

[devin-ai-integration]

🟡 Bundled video loses start bitrate

When bundled send and receive sections share payload types, mungeVideoStartBitrate edits only send sections. Their fmtp parameters then conflict, so libwebrtc can reject the hint and retry without it.

Learn more

BUNDLE uses one RTP payload-type namespace across its media sections. Sections sharing a payload type must advertise identical codec parameters. This loop adds x-google-start-bitrate only to sending sections, while single-PC and pre-populated publisher layouts can contain receive-only video sections with the same payload types. The resulting descriptions disagree on fmtp for one payload type. The fallback in set(localDescription:munging:) then drops the start-bitrate munge if libwebrtc rejects that description, leaving the affected publication at the default ramp-up behavior.

Example: A bundled send-only camera section and recv-only placeholder both advertise VP8 as payload 96. The munge adds a=fmtp:96 x-google-start-bitrate=1000 only to the camera section. The placeholder still has no fmtp for payload 96, so the bundled payload configurations differ and the hint can be rejected.

Recommended fix: After computing the connection-level value, conform matching video codec fmtp across every non-rejected section in the BUNDLE group, including recv-only and inactive placeholders. Keep placeholder sections excluded only from choosing the bitrate. Add a real-peer-connection test containing one sending and one recv-only video section with a shared payload type.

---

Was this helpful? React with 👍 or 👎 to provide feedback.

[sergeyphi]

@xianshijing-lk Pushed a follow-up commit aligned with #973 and what merged in livekit/client-sdk-js#2102 / livekit/rust-sdks#1430:

• one connection-level value: the largest hint among the offer's sending video sections that map to a recorded sender, written to every video codec of every sending section; recvonly/inactive sections neither contribute nor receive it (same stale-section reasoning as the JS PR);
• written once per peer connection, latched only after setLocalDescription accepts the munged offer, so a rejected munge retries; audio/data-only offers before any video leave the latch unset; a full reconnect builds a new Transport;
• x-google-max-bitrate still never written, now with the reason in the doc comment;
• kept the 300 kbps floor.

Tests cover the connection-level max and the sending filter, the munge on every sending section, the latch condition, and two offers on one peer connection (answered by a second local PC) where only the first carries the hint.