feat: add a native extension API and an FFmpeg-based media module

.gitmodules

@@ -0,0 +1,4 @@
+[submodule "webrtc-java-media/third-party/ffmpeg"]
+	path = webrtc-java-media/third-party/ffmpeg
+	url = https://github.com/FFmpeg/FFmpeg.git
+	shallow = true

docs/.vitepress/sidebar.ts

@@ -22,6 +22,7 @@ function sidebarGuide(): DefaultTheme.SidebarItem[] {
 				{ text: 'Media Devices', link: '/media/media-devices' },
 				{ text: 'Media Constraints', link: '/media/constraints' },
 				{ text: 'Media Directionality', link: '/media/directionality' },
+				{ text: 'Media Files', link: '/media/media-files' },
 			],
 		},
 		{

docs/guide/audio/custom-audio-source.md

@@ -92,6 +92,20 @@ The call checks its arguments and throws `IllegalArgumentException` rather than
 The audio is handed to the track's senders on the thread that calls `pushAudio`, so call it from a single thread. A scheduled executor with one thread, as shown below, is the simplest way to do that.
 :::
 
+### Keeping the Source's Own Timing
+
+A chunk pushed with the call above is treated as captured at that moment. A source with timing of its own, such as a media file played alongside video, should say when the audio was captured instead:
+
+```java
+long baseUs = SyncClock.currentTimeUs();          // once, when playback starts
+long presentationUs = chunkIndex * 10_000L;       // 10 ms per chunk
+
+audioSource.pushAudio(pcm, 16, sampleRate, channels, frameCount,
+        baseUs + presentationUs);
+```
+
+Use the same base for the audio and the video of one source, and the two stay in sync on the receiving side. See [Custom Video Source](/guide/video/custom-video-source) for the video half. Chunks must still be pushed in real time; the timestamp describes the audio, it does not schedule it.
+
 ## Audio Format Considerations
 
 When pushing audio data, you need to consider the following parameters:

docs/guide/examples.md

@@ -52,6 +52,27 @@ The [`DesktopVideoExample`](https://github.com/devopvoid/webrtc-java/blob/master
 
 This example is particularly useful for applications that need to implement screen sharing or remote desktop functionality.
 
+## Media File
+
+The [`MediaFileExample`](https://github.com/devopvoid/webrtc-java/blob/master/webrtc-examples/src/media/java/dev/onvoid/webrtc/examples/MediaFileExample.java) demonstrates how to send a media file over a peer connection, in place of a camera and a microphone. See the [Media Files](/guide/media/media-files) guide for the API it uses.
+
+**Key features demonstrated:**
+- Opening a media file with a `MediaFileSource`
+- Reading what the source contains from its `MediaInfo`
+- Creating audio and video tracks from the media sources it feeds
+- Adding those tracks to a peer connection
+- Following playback through a `MediaPlayerListener`
+
+This example is useful for applications that stream pre-recorded media, or that need a dependable stand-in for a camera in testing.
+
+::: warning
+This example needs the `webrtc-java-media` module, which is opt-in while it is being brought up on every platform, so it is built and run with the `with-media-extension` profile:
+
+```bash
+mvn -Pwith-media-extension exec:java -D"exec.mainClass=dev.onvoid.webrtc.examples.MediaFileExample" -D"exec.args=movie.mp4"
+```
+:::
+
 ## Web Client
 
 The [`WebClientExample`](https://github.com/devopvoid/webrtc-java/blob/master/webrtc-examples/src/main/java/dev/onvoid/webrtc/examples/web/WebClientExample.java) demonstrates how to combine WebSocket signaling with WebRTC peer connections for real-time communication between web and Java clients.

docs/guide/media/media-files.md

@@ -0,0 +1,232 @@
+# Media Files
+
+This guide explains how to send a media file over a peer connection instead of a camera and a microphone, using the `webrtc-java-media` module. It covers:
+
+- Adding the media module to a project
+- Sending a file with `MediaFileSource`
+- Reading what a source contains with `MediaReader` and `MediaInfo`
+- Controlling playback and following it with a listener
+- Feeding your own media sources with `MediaPlayer`
+
+Sending a file is a common need: a test pattern instead of a webcam, a pre-recorded briefing, a video that has to reach several participants. Without help, an application has to bring its own decoder and push I420 frames into a `CustomVideoSource` itself. The media module removes that work by decoding with [FFmpeg](https://ffmpeg.org) inside the library.
+
+Decoding happens entirely in native code. Frames never travel through Java: the module hands decoded pictures straight to the native side of a `CustomVideoSource`, and 10 ms chunks to a `CustomAudioSource`. They are paced in real time and carry the presentation times of the file, so what a receiver gets keeps the timing of the media rather than the timing of a Java thread.
+
+::: warning Opt-in while it is being brought up
+The media module is not part of the default build yet, and its native library has so far been built for `windows-x86_64` only. Build it with the `with-media-extension` profile, as described below.
+:::
+
+## Adding the Module
+
+The module builds FFmpeg from a submodule pinned to a release tag, so the submodule has to be present:
+
+```shell
+git submodule update --init --depth 1 webrtc-java-media/third-party/ffmpeg
+mvn install -Pwith-media-extension
+```
+
+Building FFmpeg needs `make` and `nasm`. On Windows they come from MSYS2:
+
+```shell
+winget install MSYS2.MSYS2
+C:\msys64\usr\bin\bash -lc "pacman -S --needed make nasm diffutils pkgconf"
+```
+
+Maven still runs from an ordinary shell; the build enters MSYS2 and the Visual Studio environment on its own. The first build compiles FFmpeg, which takes a while; later builds reuse the install directory.
+
+Once installed, depend on it alongside `webrtc-java`:
+
+```xml
+<dependency>
+    <groupId>dev.onvoid.webrtc</groupId>
+    <artifactId>webrtc-java-media</artifactId>
+    <version>0.19.0-SNAPSHOT</version>
+</dependency>
+```
+
+The classifier jar carries the module's native library together with the FFmpeg shared libraries it uses. Applications that do not use this module never download FFmpeg.
+
+## Sending a File
+
+`MediaFileSource` is the short way to do all of it. It opens the source, creates a media source for each kind of media the file actually has, and wires a player to feed them:
+
+```java
+// Import required classes
+import dev.onvoid.webrtc.media.ffmpeg.MediaFileSource;
+import dev.onvoid.webrtc.media.audio.AudioTrack;
+import dev.onvoid.webrtc.media.video.VideoTrack;
+import java.nio.file.Path;
+import java.util.List;
+
+MediaFileSource source = new MediaFileSource(Path.of("movie.mp4"));
+
+// Create tracks from the media sources the file feeds.
+VideoTrack videoTrack = factory.createVideoTrack("video", source.getVideoSource());
+AudioTrack audioTrack = factory.createAudioTrack("audio", source.getAudioSource());
+
+peerConnection.addTrack(videoTrack, List.of("stream"));
+peerConnection.addTrack(audioTrack, List.of("stream"));
+
+// Start sending.
+source.play();
+```
+
+A file with no audio has no audio source, and likewise for video, so check before making a track:
+
+```java
+if (source.getVideoSource() != null) {
+    VideoTrack videoTrack = factory.createVideoTrack("video", source.getVideoSource());
+    peerConnection.addTrack(videoTrack, List.of("stream"));
+}
+```
+
+::: warning A factory sends either pushed audio or captured audio
+A `PeerConnectionFactory` fed from this module is sending pushed audio and cannot also send audio captured by its `AudioDeviceModule`. Give such a factory a dummy audio layer, and use a second factory if an application needs both:
+
+```java
+AudioDeviceModule audioModule = new AudioDeviceModule(AudioLayer.kDummyAudio);
+PeerConnectionFactory factory = new PeerConnectionFactory(audioModule);
+```
+:::
+
+## Reading What a Source Contains
+
+`MediaReader` opens a source and reports what is in it, without playing anything:
+
+```java
+// Import required classes
+import dev.onvoid.webrtc.media.ffmpeg.MediaInfo;
+import dev.onvoid.webrtc.media.ffmpeg.MediaReader;
+
+try (MediaReader reader = new MediaReader(Path.of("movie.mp4"))) {
+    MediaInfo info = reader.getInfo();
+
+    System.out.println("Runs " + info.getDurationUs() / 1_000_000.0 + " s");
+
+    if (info.hasVideo()) {
+        System.out.println(info.getVideoWidth() + "x" + info.getVideoHeight()
+                + " at " + info.getFrameRate() + " fps, " + info.getVideoCodec());
+    }
+    if (info.hasAudio()) {
+        System.out.println(info.getSampleRate() + " Hz, "
+                + info.getChannels() + " channels, " + info.getAudioCodec());
+    }
+}
+```
+
+A `MediaFileSource` already has this, so there is no need to open the file twice:
+
+```java
+MediaInfo info = source.getInfo();
+```
+
+::: info
+Duration is `0` for a source whose container does not say how long it runs, which is the case for live streams. Opening fails if a source holds nothing that can be played.
+:::
+
+## Controlling Playback
+
+```java
+source.play();                  // start, or resume after a pause
+source.pause();                 // hold playback where it is
+source.seek(30_000_000);        // move to 30 seconds
+source.setLooping(true);        // start over instead of ending
+source.getPositionUs();         // where playback has got to
+source.getState();              // IDLE, PLAYING, PAUSED, ENDED or CLOSED
+```
+
+Looping keeps the timing across the seam, so a looping file works as a stand-in for a camera that never stops. A looping source never reports an end of stream.
+
+A seek lands on the keyframe at or before the position asked for, which is how far back the decoder has to go to produce a picture at all. How close that is to the position asked for depends on how often the media was encoded with keyframes.
+
+## Following Playback
+
+```java
+// Import required classes
+import dev.onvoid.webrtc.media.ffmpeg.MediaPlayerListener;
+import dev.onvoid.webrtc.media.ffmpeg.MediaPlayerState;
+
+source.setListener(new MediaPlayerListener() {
+
+    @Override
+    public void onStateChanged(MediaPlayerState state) {
+        System.out.println("Player state: " + state);
+    }
+
+    @Override
+    public void onEndOfStream() {
+        System.out.println("The source ran out.");
+    }
+
+    @Override
+    public void onError(String message) {
+        System.out.println("Playback failed: " + message);
+    }
+});
+```
+
+::: warning
+Every call arrives on the player's own thread, and that thread is the one decoding the media. A listener must return promptly, and must not wait on the player.
+:::
+
+## Feeding Your Own Media Sources
+
+`MediaFileSource` creates the media sources for you. An application that needs its own — to feed a source it already created, or to send only the video of a file — uses `MediaPlayer` directly:
+
+```java
+// Import required classes
+import dev.onvoid.webrtc.media.ffmpeg.MediaPlayer;
+import dev.onvoid.webrtc.media.ffmpeg.MediaReader;
+import dev.onvoid.webrtc.media.video.CustomVideoSource;
+
+CustomVideoSource videoSource = new CustomVideoSource();
+
+// Video only: passing null for the audio source decodes and drops the audio.
+MediaPlayer player = new MediaPlayer(new MediaReader(path), videoSource, null);
+
+VideoTrack videoTrack = factory.createVideoTrack("video", videoSource);
+
+player.play();
+```
+
+::: info
+The player takes over the reader it is given. That reader must not be used or closed afterwards; closing the player releases it.
+:::
+
+## Closing
+
+Closing a `MediaFileSource` stops playback and releases the player along with both media sources. Release the senders a peer connection handed out, and the tracks, before that:
+
+```java
+videoSender.dispose();
+audioSender.dispose();
+peerConnection.close();
+
+source.close();
+```
+
+## What Can Be Played
+
+The FFmpeg build is deliberately small, and carries only what this module plays:
+
+| | |
+| --- | --- |
+| **Containers** | MP4 and MOV, Matroska and WebM, AVI, MPEG-TS, FLV, WAV, MP3, Ogg, FLAC, AAC |
+| **Video** | H.264, H.265/HEVC, VP8, VP9, MPEG-4, MJPEG |
+| **Audio** | AAC, MP3, Opus, Vorbis, FLAC, PCM |
+
+Audio of any rate or layout is resampled to what WebRTC takes, which is 48 kHz 16-bit PCM in mono or stereo. Video that decodes to I420 — almost all 8-bit H.264, VP8, VP9 and MPEG-4 — reaches the encoder without being copied; anything else is converted first.
+
+Only local files play today. FFmpeg demuxes network sources just as well, so the same code will cover http, rtsp and rtmp once those protocols are turned on in the build.
+
+## Licensing
+
+The module uses FFmpeg under the LGPL version 2.1 or later. It is configured without `--enable-gpl` and without `--enable-nonfree`, and FFmpeg is linked dynamically and shipped as separate files inside the platform jar, so its libraries may be replaced with your own build, as the LGPL requires. The wrapper code is licensed under the Apache License 2.0 like the rest of webrtc-java.
+
+## Complete Example
+
+See `MediaFileExample` in the `webrtc-examples` module, which opens a file, reports what it contains, creates tracks, adds them to a peer connection and follows playback to the end.
+
+```shell
+mvn -Pwith-media-extension -pl webrtc-examples compile
+```

docs/guide/video/custom-video-source.md

@@ -142,6 +142,32 @@ public class VideoStreamer {
 }
 ```
 
+### Keeping the Source's Own Timing
+
+`pushFrame(VideoFrame)` treats a frame as captured at the moment of the call, so the timing that reaches the other peer is the timing of your pushing thread. For a live source such as a camera that is exactly right. For a source that has timing of its own, such as a video file, it is not: every hiccup of the executor above ends up in the stream, and audio pushed alongside drifts out of sync with the video.
+
+Pass the capture time instead:
+
+```java
+// The clock capture timestamps are interpreted in.
+long baseUs = SyncClock.currentTimeUs();
+
+// For each frame, its own presentation time within the source.
+long presentationUs = frameIndex * 1_000_000L / frameRate;
+
+videoSource.pushFrame(frame, baseUs + presentationUs);
+```
+
+Map your source's timeline onto the clock once, when playback starts, and add each frame's presentation time to that base. The frame rate that reaches the encoder then follows your timestamps rather than the jitter of the thread.
+
+::: warning
+Frames must still be pushed in real time. A frame is encoded and sent when it arrives, so a timestamp in the future does not delay it; it only describes when the frame was meant to be shown.
+
+Timestamps must also advance by at least one millisecond from frame to frame. WebRTC drops a frame whose capture time does not move forward.
+:::
+
+Push audio with timestamps from the same clock, as described in [Custom Audio Source](/guide/audio/custom-audio-source), and the two stay in sync on the receiving side.
+
 ## Integration with Video Tracks
 
 ### Adding Sinks to Monitor Video

pom.xml

@@ -354,6 +354,19 @@
 				<platform.module>webrtc.macos.aarch64</platform.module>
 			</properties>
 		</profile>
+		<!--
+			The FFmpeg based media extension. Opt-in while it is being brought
+			up on every platform: it builds FFmpeg from the
+			webrtc-java-media/third-party/ffmpeg submodule, which has build
+			requirements of its own (make, nasm, pkg-config) that the rest of
+			the project does not have.
+		-->
+		<profile>
+			<id>with-media-extension</id>
+			<modules>
+				<module>webrtc-java-media</module>
+			</modules>
+		</profile>
 	</profiles>
 
 	<dependencies>