Skip to content

fix: keep device-captured audio out of sink-fed audio senders - #289

Merged
devopvoid merged 4 commits into
mainfrom
fix/adm-capture-gate
Sep 19, 2026
Merged

devopvoid merged 4 commits into
mainfrom
fix/adm-capture-gate

Conversation

@devopvoid

Copy link
Copy Markdown
Owner

Fixes #217, fixes #118, fixes #213

The crash

WebRTC's AudioState starts the factory's AudioDeviceModule recording as soon as any audio send stream starts, and AudioTransportImpl fans every captured frame out to every send stream of that factory. A send stream whose track is backed by a CustomAudioSource already receives its audio through WebRTC's LocalAudioSinkAdapter on the thread that calls pushAudio(). Both feeders enter AudioSendStream::SendAudioData, and its RTC_CHECK_RUNS_SERIALIZED aborts the process:

Fatal error in: ../../audio/audio_send_stream.cc
Check failed: !race_checker.RaceDetected()

That is why single-threading pushAudio() never helped: the second feeder is WebRTC's own capture thread. Where it does not abort, microphone frames are interleaved with the pushed audio instead. Hosts without a microphone never crash, because InitRecording fails, which is why reproduction looked inconsistent. The sinks_ mutex from #285 could not address this; it protects a different list on a different path.

The fix

  • ProxyAudioDeviceModule wraps whatever module the factory is created with and gates its capture path. The gate starts closed: InitRecording() and StartRecording() fail so AudioState never opens the device, and recorded frames that still arrive are dropped before they reach WebRTC. Playout is untouched, so received audio still reaches the speakers and AudioTrack sinks.
  • One kind of audio input per factory. createAudioSource(AudioOptions) opens the gate and commits the factory to device-captured audio. createAudioTrack with a CustomAudioSource, or adding a sink-fed track (custom or forwarded remote) as a sender, commits it to pushed audio. The other kind is then rejected with an IllegalStateException that names the conflict. Applications needing both create a second factory. RTCRtpSender.replaceTrack and RTCRtpTransceiver.setDirection are not covered by the guard.
  • HeadlessAudioDeviceModule loses its capture thread, which pulled the render mix back and handed it to WebRTC as captured audio: an echo to the remote peer and a second feeder on the same send stream. Recording is now a state the module reports and nothing more. This exposed a second bug: the module handed its transport to AudioDeviceBuffer, which refuses it once playout has started, so in the documented ordering received audio never reached a sink. The render thread now pulls the transport directly, without holding the module's state lock across the pull, and the buffer is removed.
  • pushAudio() validates its arguments. It accepted any sample width although WebRTC reads 16-bit, read past the end of an array shorter than the claimed frame count, and passed on chunks too large for WebRTC's audio frame, which also aborts the process.

Docs

The custom audio source, headless audio and audio devices guides are updated: the one-input-kind-per-factory rule, 16-bit 10 ms frames from a single thread, the kDummyAudio limitations, and a note that pushed audio bypasses the audio processing module.

Testing

mvn verify is green at 158 tests. New tests cover every conflicting order of the latch, pushAudio validation, and an extended HeadlessADMIntegrationTest that pushes for two seconds while the module's recording is running. The touched test classes also pass under -Pjni-check. The concurrent-feeder abort itself cannot be reproduced in CI without a microphone; on a host with one, WebClientExample crashes before this change and must not afterwards.

WebRTC's AudioState starts the factory's AudioDeviceModule recording as soon
as any audio send stream starts, and AudioTransportImpl fans every captured
frame out to every send stream of that factory. A send stream whose track is
backed by a CustomAudioSource already receives its audio through WebRTC's
LocalAudioSinkAdapter on the thread that calls pushAudio(), so both feeders
enter AudioSendStream::SendAudioData and its RTC_CHECK_RUNS_SERIALIZED aborts
the process:

  Fatal error in: ../../audio/audio_send_stream.cc
  Check failed: !race_checker.RaceDetected()

That is why single-threading pushAudio() never helped: the second feeder is
WebRTC's own capture thread. Where it does not abort, microphone frames are
interleaved with the pushed audio instead.

PeerConnectionFactory now hands WebRTC a ProxyAudioDeviceModule that forwards
every call to the wrapped module and gates its capture path. Creating an audio
track from a CustomAudioSource switches that path off: InitRecording() and
StartRecording() fail so AudioState never opens the device, and recorded
frames that still arrive are dropped before they reach WebRTC. Playout is
untouched, so received audio still reaches the speakers and AudioTrack sinks.

Since a factory can send only one kind of audio input, it now commits to the
kind used first and rejects the other with an IllegalStateException that names
the conflict.

Refs #217, #118, #213
Builds on the previous commit, which gated the device capture path so that
audio pushed through a CustomAudioSource is the only thing feeding a sender.

The gate now starts closed. A factory can only send device-captured audio
through createAudioSource(), so gating until that call keeps captured audio
away from senders without having to recognise them one by one. A server that
forwards a track it received from one peer to another (issue #118) is fed the
same way as a CustomAudioSource, on the thread that decodes the audio, and it
is now safe without any change to the application.

A factory that does both is still a conflict, so RTCPeerConnection reports a
track to its factory before the track becomes a sender, and the factory
rejects a pushed-audio track once it sends device-captured audio. A
receive-only transceiver never sends its track and is left alone. Changing a
transceiver's direction later, or swapping a track through
RTCRtpSender.replaceTrack, is not covered.

HeadlessAudioDeviceModule loses its capture thread. It pulled the render mix
back through AudioTransport and handed it to WebRTC as captured audio, which
made a peer connection send the audio it had just received straight back to
the remote peer, and fed the send stream a second time alongside the pushed
audio. Recording is now a state the module reports and nothing more.

That capture thread was also the only thing pulling the audio mixer, which
uncovered a second bug: the module hands its AudioTransport to
AudioDeviceBuffer, which refuses it once playout has started, and the guide
tells applications to start playout before creating a peer connection, so the
render thread was pulling from nothing. Received audio then never reached an
AudioTrack sink. The render thread now pulls the transport directly, and the
integration test covers that ordering.

CustomAudioSource.pushAudio() validates its arguments. It accepted any sample
width although WebRTC reads the samples as 16-bit, read past the end of an
array shorter than the frame count claimed, and passed on chunks too large for
WebRTC's audio frame, which aborts the process.

Refs #217, #118, #213
The previous commit made the headless module's render thread call
AudioTransport::NeedMorePlayData directly, but did so with the module's
state mutex held. That serialised every 10 ms mixer pull against every
other call on the module, and any future path from inside the pull back
into the module would have deadlocked.

The render thread now decides under the state lock whether the tick pulls
and snapshots the format it needs, then releases the lock before calling
the transport. The transport pointer gets its own mutex, which is held for
the duration of a pull, so RegisterAudioCallback() still waits for a pull
in progress and no call reaches the old transport after it returns.

The AudioDeviceBuffer the module carried is gone. Since the render thread
pulls the transport directly and no capture thread delivers into it, it
only received registrations and format updates that nothing read back.
Its removal also drops the webrtc::Environment the module was constructed
with, which existed solely to create the buffer.
Pushed audio bypasses the audio processing module WebRTC runs on device
capture, and the guide said so without saying what to do instead. It now
walks through running each chunk through the standalone AudioProcessing
class before pushAudio(): noise suppression and gain control first, then
echo cancellation, which needs the application to feed the far-end audio
through processReverseStream and set the stream delay, since only device
capture gets that reference from WebRTC for free.
@devopvoid
devopvoid merged commit d181f55 into main Sep 19, 2026
11 checks passed
@devopvoid
devopvoid deleted the fix/adm-capture-gate branch September 19, 2026 21:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant