Skip to content

WIP: Fix data track saturation shutdown deadlock #1315

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
alan-george-lk wants to merge 2 commits into
base: main
Choose a base branch
from
+17 −2
Open

WIP: Fix data track saturation shutdown deadlock #1315

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
60e4ce8
Increase the data track event buffer to avoid a disconnect deadlock.
alan-george-lk Sep 24, 2026
ea2b887
Fix changeset
alan-george-lk Sep 24, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions .changeset/increase_data_track_event_buffer.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
---
livekit-datatrack: patch
livekit: patch
livekit-uniffi: patch
livekit-capture: patch
livekit-ffi: patch
---

Increase the local and remote data track event buffers so a burst of track lifecycle events cannot fill the channel and deadlock room disconnect.
5 changes: 4 additions & 1 deletion livekit-datatrack/src/local/manager.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -326,7 +326,10 @@ impl Manager {
const FRAME_BUFFER_COUNT: usize = 16;

/// Maximum number of input and output events to buffer.
const EVENT_BUFFER_COUNT: usize = 16;
///
/// Must exceed a burst of track lifecycle events. A full channel blocks
/// the manager and can deadlock room disconnect.
const EVENT_BUFFER_COUNT: usize = 512;
}

/// Task for an individual published data track.
Expand Down
5 changes: 4 additions & 1 deletion livekit-datatrack/src/remote/manager.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -443,7 +443,10 @@ impl Manager {
const PACKET_BUFFER_COUNT: usize = 16;

/// Maximum number of input and output events to buffer.
const EVENT_BUFFER_COUNT: usize = 16;
///
/// Must exceed a burst of track lifecycle events. A full channel blocks
/// the manager and can deadlock room disconnect.
const EVENT_BUFFER_COUNT: usize = 512;

@devin-ai-integration devin-ai-integration Bot Sep 24, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 Full output queue stalls room disconnect

During disconnect, Manager can block after emitting more than 512 events. The room's forward task stops reading output, so close waits forever for the blocked manager.

Learn more

Output sends await room-side consumption. On disconnect the room stops forwarding output before joining managers. More than 512 queued events block a manager inside a send, preventing it from reading Shutdown. The same failure applies to the local manager.

Example: A remote update with 513 new tracks fills the output queue after close stops forwarding. The manager blocks sending the last event and room close never finishes.

Recommended fix: Give the manager a separate shutdown signal and make output sends cancellable, or drain output until the manager has exited. Cover full output queues in shutdown tests.

Devin Review

Was this helpful? React with 👍 or 👎 to provide feedback.

}

/// Information and state for a remote data track.
Expand Down
Loading