Fatal error in audio_send_stream.cc

[Charles92011]

Describe the bug
I've had my experimental project working for a while now: https://github.com/Charles92011/webrtc-java-experimental-server
It was working great, but I put it away for a couple of months until we're ready to transition to it.
When I started it up today, it gets this error upon connecting two connections together:

#
# Fatal error in: ../../audio/audio_send_stream.cc, line 387
# last system error: 0
# Check failed: !race_checker.RaceDetected()
# 

I updated my pom file to get the latest version of the library and it still fails.
This error is untrappable, so I can't recover from it

To Reproduce
Steps to reproduce the behavior:

1. clone https://github.com/Charles92011/webrtc-java-experimental-server
2. run main.java
3. browse to wsclient.html
4. click start
5. click mirror
6. Server will crash

Expected behavior
You should get two videos working fine

Desktop (please complete the following information):
Windows 10

Additional context
This was working fine when I last used it a few months ago.

[Charles92011]

This error only appears when transmitting an audio track.
Video tracks alone do not throw this error.

[naveenSaini1]

hey i am also getting this same error wheni am trying to add mediastreamtrack of another party

Map<String, RTCPeerConnection> savedMapOfConnection = list_of_connected_user
		                .get(roomUserResponse.getRoomId());
		        	savedMapOfConnection.forEach((key, connection) -> {
		        	final int receivers = connection.getReceivers().length;
		    		if (receivers == 0)
                                      {return;}
		    		try {
		    			for (int index = 0; index < receivers; index++) {
			    			
			    			final RTCRtpReceiver receiver = connection.getReceivers()[index];
			    			if (receiver != null) {
			    				
			    				final MediaStreamTrack track = receiver.getTrack();
			    				if (track != null) {
			    							    					
			    					List<String> streamIds = new ArrayList<String>();
			    					streamIds.add(receiver.getTrack().getId());
			    					
			    					@SuppressWarnings("unused")
			    					RTCRtpSender sender = connection.addTrack(track, streamIds);
			    				}
			    			}
			    		}
		    			
		    		}
		    		catch(Exception e) {
		    			System.out.println("someting went wrong "+e.getMessage());
		    			
		    		}
		    		
		            
		        });
		}

so help

[Charles92011]

@naveenSaini1 I was never able to get past this error, and in a year there's been no update that mentions it.
I had to abandon it.

[naveenSaini1]

@Charles92011 hey i saw your project and that is really amazing most of the things i learn from there and about this error i need to resolve this because i am trying to make sfu server in webrtc and now i have been stuck at this point if you want we can figure out on this

[Charles92011]

@naveenSaini1 I'm in the same situation. My goal was to build a server with this library, but I had to abandon it because of this error.

You could always download the library source, and see if you can trap the error.

From my research, the error is actually occurring in the WebRTC code itself, not the java RTC library. So you could try to modify JavaRTC to trap the exception or try to debug Google's WebRTC code to fix it. I don't have the resources available to do either one.

[naveenSaini1]

@Charles92011
hi i noticed something in this , like whenever we send the audioTrack like this

 AudioTrackSource audioSource = peerConnectionFactory.createAudioSource(new AudioOptions());
		AudioTrack audioTrack = peerConnectionFactory.createAudioTrack("audioTrack", audioSource);
			audioTrack.addSink(new AudioTrackSink() {
				@Override
				public void onData(byte[] data, int bitsPerSample, int sampleRate, int channels, int frames) {
					System.out.println(data);
				}
			});
			
			List<String> streamIds = new ArrayList<>();
			streamIds.add("stream-0");

			RTCRtpSender audioSender = peer_connection.addTrack(audioTrack, streamIds);

we will not get the error so i change this way to puting the track in diffrent way

AudioTrack audioTrack = (AudioTrack)allTheTracks.get("a");
   		audioTrack.addSink(new AudioTrackSink() {
   			@Override
   			public void onData(byte[] data, int bitsPerSample, int sampleRate, int channels, int frames) {
   				System.out.println(data);
   			}
   		});
   		
   		List<String> streamIds = new ArrayList<>();
   		streamIds.add(allTheTracks.get("a").getId());

   		RTCRtpSender audioSender = peer_connection.addTrack(audioTrack, streamIds);

but guess what same thing that same issue

 #
# Fatal error in: ../../audio/audio_send_stream.cc, line 387
# last system error: 11
# Check failed: !race_checker.RaceDetected()

@devopvoid can you please look into this , i really don't know like how to resolve this or can you gave little hint

[devopvoid]

WebRTC has detected that multiple threads are accessing the same audio send stream object concurrently without proper synchronization. Or it is improper stream lifecycle management when the stream is closed but still accessed. Try to wrap your peer-connection control flow into one managing thread.

[turbo-hammer]

I managed to resolve the issue using static mutex:

                private static final Object LOCK = new Object();


                synchronized (LOCK) {
                    customAudioSource.pushAudio(
                            audioData,
                            DEFAULT_BITS_PER_SAMPLE,
                            DEFAULT_SAMPLE_RATE,
                            DEFAULT_CHANNELS,
                            DEFAULT_FRAME_COUNT
                    );
                }

[sc-sityad]

I managed to resolve the issue using static mutex:

            private static final Object LOCK = new Object();


            synchronized (LOCK) {
                customAudioSource.pushAudio(
                        audioData,
                        DEFAULT_BITS_PER_SAMPLE,
                        DEFAULT_SAMPLE_RATE,
                        DEFAULT_CHANNELS,
                        DEFAULT_FRAME_COUNT
                );
            }

I have done that still that error persist @turbo-hammer can u give the full exxample of connection + signaling + customAudioSource usage, if possible?

[turbo-hammer]

I managed to resolve the issue using static mutex:

            private static final Object LOCK = new Object();


            synchronized (LOCK) {
                customAudioSource.pushAudio(
                        audioData,
                        DEFAULT_BITS_PER_SAMPLE,
                        DEFAULT_SAMPLE_RATE,
                        DEFAULT_CHANNELS,
                        DEFAULT_FRAME_COUNT
                );
            }

I have done that still that error persist @turbo-hammer can u give the full exxample of connection + signaling + customAudioSource usage, if possible?

This is not working for me. Some times passed but then happened again.

[sc-sityad]

Then , how u resolved it or u haven't?

[devopvoid]

The cause of this is now understood and fixed on fix/adm-capture-gate. Full write-up in #217.

Short version: this was never a threading mistake in your code. The factory's AudioDeviceModule feeds captured audio into every audio sender of that factory, so a sender whose track is backed by a CustomAudioSource, or by a track forwarded from another peer, is fed twice from two different threads. The RTC_CHECK_RUNS_SERIALIZED in audio_send_stream.cc then aborts the process.

@naveenSaini1 your SFU case is exactly that shape. Re-adding a received track creates a sender that WebRTC feeds from the thread that decodes the remote audio, while the microphone feeds the same stream from the capture thread. On the branch, the recording device is only opened once you ask for device-captured audio with createAudioSource, so forwarding received tracks works without any change on your side. If you do both on one factory you now get an IllegalStateException naming the conflict, rather than a crash minutes later.

@Charles92011 this is also why it only appeared with an audio track and never with video alone.

My earlier answer here, to wrap the peer connection control flow into one thread, was wrong: the second thread belongs to WebRTC. Sorry that it stood for so long.

[devopvoid]

Following up with the actual cause, since this is the same abort as #217.

AudioSendStream::SendAudioData is fed twice. A forwarded remote track feeds it on the decoding thread through the track's sinks. At the same time, WebRTC's AudioState starts the factory's AudioDeviceModule recording as soon as any send stream starts, and the capture thread fans every captured frame out to every send stream of that factory. The two overlap intermittently, and the function's RTC_CHECK_RUNS_SERIALIZED aborts the process.

#289 closes the capture path for factories that only forward or push audio, so the device is never opened and the send stream has a single feeder. No application change is needed for the forwarding case.