Refactor WebM playback for asynchronous, creator-controlled movies - #361
Merged
Conversation
Open reverse movie decoder contexts without an active audio track. This allows interleaved audio packets to be discarded during demuxing instead of filling paused PCM buffers and blocking video delivery. Restore embedded audio when playback returns to a positive speed while preserving normal forward and legacy playback behavior.
Disable audio decoding during stationary or reverse playback to prevent audio queues from blocking video demuxing. Align decoded Vorbis PCM with the requested seek position. Account for codec delay, discard samples preceding the target, and insert silence when the first available audio packet begins after it. Keep the sound stream paused until the movie playback clock is anchored.
Disable audio decoding during stationary and reverse playback to prevent unconsumed audio queues from blocking video demuxing. Synchronize audio after seeking by accounting for codec delay, discarding PCM before the requested position, and inserting silence when the first available packet begins after it. Hold the sound channel paused until the movie clock is anchored. Preserve decimal playback rates through the audio period calculation instead of rounding them to integer percentages.
Add movie_draw_to_screen() so creators explicitly draw a retained movie frame to a selected screen and control its position in the composition order. Remove automatic main-screen and subscreen rendering, persistent screen ownership, movie_set_screen(), and MOVIE_PROPERTY_SCREEN. Update legacy playwebm() to clear and present only when a new movie frame arrives instead of presenting on every one-millisecond decoder poll. Preserve final-frame presentation and invalidate scaled-frame caches correctly when drawing one movie to multiple target sizes.
Add MOVIE_LOADING_AUTO as an opt-in loading mode while retaining streaming as the default behavior. Allow automatic caching when the movie fits within one-quarter of system memory and leaves a reserve of one-eighth system memory or 128 MiB, whichever is greater. Track all resident movie caches against the cumulative automatic budget without imposing a fixed upper limit. Add nonfatal movie cache allocation and bounded packfile reads. Automatic mode falls back to streaming under memory pressure, while forced cache mode returns a clean load failure.
Restore compatibility with valid WebM files that omit the optional DefaultDuration element. Playback now relies on packet timestamps and uses packet duration to space laced frames when a default duration is unavailable. Correct the Nestegg streaming callbacks to return the expected seek result, use 64-bit packfile positions, verify read boundaries, and complete partial reads safely. Add explicit diagnostics for container initialization, track discovery, allocation, and worker-thread failures instead of allowing movie playback to fail silently. These fixes apply to both the movie playback API and legacy playwebm().
Convert creator-supplied playback speed to one canonical float. Clamp supported range and round to thousandths so video timing, audio resampling, property reads, and later decoder reopens all consume the same applied rate.
- increase streaming audio reserves and improve decoder scheduling - retain underrun telemetry with playback-aware producer backoff - convert YUV directly to requested composition dimensions - allocate native-resolution RGB buffers only when required - add MOVIE_SIZE_NATIVE width and height sentinel - preserve destination-sized rendering when dimensions are zero - correct reported WebM audio sample depth
Configure VP8 decoding from source resolution and available processors while reserving capacity for engine and audio work on larger systems. Prevent prolonged active-playback stalls from back-pressuring the shared demuxer by replacing stale decoded frames only after the compressed video queue reaches its safety threshold. Preserve ordered blocking during ordinary read-ahead, startup, and paused playback. Report rate-limited underrun snapshots for PCM, compressed audio and video, decoded frames, and stale-frame replacements. Keep diagnostics off the normal playback path and preserve safe queue lifetime through audio-thread shutdown. Retain direct target-size YUV conversion, eight-buffer PCM reserves, decoder preroll, and playback-aware producer backoff.
Snapshot decoded-frame queue occupancy before polling and process only that population during the current engine update. Defer frames published concurrently until the next update. This prevents a decoder producing stale frames from continuously refilling the queue and turning nonblocking catch-up into an unbounded main-thread stall. Preserve retained-frame promotion, future-frame timing, reverse playback, and terminal-marker handling.
Preroll decoded video alongside PCM before publishing READY. Preserve one due frame per presentation opportunity, bound audio-master lateness, and report presentation or decoded-queue drops independently from compressed backlog recovery.
Treat a full decoded-frame queue as normal bounded read-ahead instead of replacing ordered frames whenever the compressed queue is populated. Keep exceptional audio-aware compressed recovery, remove obsolete decoded-drop diagnostics, and separate the initial video log block.
Rewrite playback comments as present-tense ownership, timing, and queue contracts. Remove tuning-history and compatibility-development wording, and place a blank line before WebM warnings.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Replace the unfinished WebM playback path with a reusable, asynchronous movie system designed for stable audio synchronization and creator-controlled screen composition.
The new API separates source loading, playback state, movie channels, audio routing, and drawing. Active movies advance independently, while scripts explicitly copy retained frames to the desired 32-bit screen with
movie_draw_to_screen().Movie API
playwebm()and scenevideobehavior through compatibility wrappers.Playback stability
Audio integration
Additional corrections
onblockyscript command name.onblockwscript.Compatibility and limitations
movie_draw_to_screen()calls. Playback no longer owns or retains a destination screen.Validation
Manual runtime testing was performed on Windows 11 using a Dell OptiPlex Micro 7010 with an Intel Core i5-13500T and 16 GB RAM.
Various movies played back in differing resolutions, including a 13-minute, 1920×1080, 60 FPS VP8 WebM with 48 kHz stereo Vorbis audio drawn into a 360×204 subscreen. Playback completed without a crash, lockup, PCM underrun, or sustained audio desynchronization. One guarded keyframe recovery discarded 255 compressed video packets during the complete run.
Unsupported audio encoding also produced an explicit diagnostic instead of an ambiguous playback failure.