diff --git a/.github/RELEASE_SETUP.md b/.github/RELEASE_SETUP.md index 3201001a2..e3a4d6fc2 100644 --- a/.github/RELEASE_SETUP.md +++ b/.github/RELEASE_SETUP.md @@ -67,6 +67,7 @@ Once all six secrets are set: 1. Make sure a release exists (use the existing `2.0.0` or create a draft tag). 2. Trigger the workflow manually: [Actions](https://github.com/superuser404notfound/AetherEngine/actions) → `Release .dmg` → `Run workflow` → enter the tag → Run. + The job checks out that tag, whatever branch the "Use workflow from" dropdown says, so a tag that predates `Examples/DemoPlayerMac/Scripts/build-dmg.sh` fails loudly instead of building `main`. The log's "Building commit" line names the built SHA. 3. Watch the job. It should reach "Upload .dmg to release" and finish green in 3-6 minutes. 4. Confirm the `.dmg` appears under the release's Assets (or got `--clobber`'d over the existing one if you pointed at `2.0.0`). diff --git a/.github/dependabot.yml b/.github/dependabot.yml new file mode 100644 index 000000000..d457c480a --- /dev/null +++ b/.github/dependabot.yml @@ -0,0 +1,9 @@ +version: 2 +updates: + # Third-party actions are pinned by commit SHA (audit OPS-2); this keeps the pins moving. + - package-ecosystem: github-actions + directory: / + schedule: + interval: weekly + commit-message: + prefix: "chore(ci)" diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index cd6d81930..dda2ed164 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -12,18 +12,24 @@ concurrency: cancel-in-progress: true jobs: + # One toolchain: GitHub's `xcode-27` image (macOS 27.0, default Xcode 27.0 27A266a), the same + # Xcode the engine is consumed and released from. The earlier macos-15 lanes resolved + # `latest-stable` to Xcode 26.3, whose Swift 6.2 disagreed with 6.4 in both directions (6.4 + # rejects isolation shapes 6.3 accepted, 2026-09-16; 6.2 timed out type-checking a literal sum + # inside #expect, 2026-09-30), and nothing downstream builds with 26.x any more, so the README's + # requirement is Xcode 27 and CI tests exactly that. `xcode-27` is still a GitHub preview image. + # Its flaky `swift test` runs of mid-September were on a beta macOS 27 base with Xcode 27 + # beta 6; the image moved to macOS 27 on 2026-09-16. If it turns unstable again, that shows as + # a lane red for its image, which is the signal to wait for the GA label, not to lower the bar. + # All three macOS jobs use it, which stays within the account's concurrent macOS runners. test-macos: name: swift test (macOS) - runs-on: macos-15 + runs-on: xcode-27 timeout-minutes: 30 env: AUTHORIZATION_TEST_SUITES: 'RefreshableHLSAuthorizationTests|RefreshableSubtitleAuthorizationTests|LiveTrustEvaluatorTests' steps: - - uses: actions/checkout@v4 - - - uses: maxim-lobanov/setup-xcode@v1 - with: - xcode-version: latest-stable + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - name: Show toolchain run: | @@ -72,23 +78,11 @@ jobs: run: swift build --build-tests --package-path Probes/TransportProbe build-tvos: - name: xcodebuild (tvOS Simulator) - runs-on: macos-15 + name: xcodebuild (tvOS + visionOS Simulator) + runs-on: xcode-27 timeout-minutes: 30 steps: - - uses: actions/checkout@v4 - - - uses: maxim-lobanov/setup-xcode@v1 - with: - xcode-version: latest-stable - - - name: Build for tvOS Simulator - run: | - xcodebuild build \ - -scheme AetherEngine \ - -destination 'generic/platform=tvOS Simulator' \ - -derivedDataPath .build/xcode-tvos \ - CODE_SIGNING_ALLOWED=NO + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 # The test target is the only place a platform gate in TEST code is ever checked. `swift test` # on macOS compiles the macOS side of every `#if`, so a gate put around a helper instead of @@ -98,12 +92,16 @@ jobs: # # EXCLUDED_ARCHS because a generic simulator destination builds both slices and `libdovi.a` # ships arm64 only, so the x86_64 link fails and says nothing about this repo. + # + # This step also compiles the library, so no separate `xcodebuild build` of the AetherEngine + # scheme runs first: that doubled every simulator job's compile, and the x86_64 slice it added + # is one no app can link anyway. - name: Build the tests for tvOS Simulator run: | xcodebuild build-for-testing \ -scheme AetherEngineTests \ -destination 'generic/platform=tvOS Simulator' \ - -derivedDataPath .build/xcode-tvos-tests \ + -derivedDataPath .build/xcode-tvos \ EXCLUDED_ARCHS=x86_64 \ CODE_SIGNING_ALLOWED=NO @@ -120,52 +118,12 @@ jobs: -derivedDataPath .build/xcode-probe \ CODE_SIGNING_ALLOWED=NO - build-ios: - name: xcodebuild (iOS Simulator) - runs-on: macos-15 - timeout-minutes: 30 - steps: - - uses: actions/checkout@v4 - - - uses: maxim-lobanov/setup-xcode@v1 - with: - xcode-version: latest-stable - - - name: Build for iOS Simulator - run: | - xcodebuild build \ - -scheme AetherEngine \ - -destination 'generic/platform=iOS Simulator' \ - -derivedDataPath .build/xcode-ios \ - CODE_SIGNING_ALLOWED=NO - - # The other half of the gate check above; see the tvOS job for why the test target needs its - # own build and why x86_64 is excluded. - - name: Build the tests for iOS Simulator - run: | - xcodebuild build-for-testing \ - -scheme AetherEngineTests \ - -destination 'generic/platform=iOS Simulator' \ - -derivedDataPath .build/xcode-ios-tests \ - EXCLUDED_ARCHS=x86_64 \ - CODE_SIGNING_ALLOWED=NO - - # #344: the package has declared visionOS since 6.0.0 and nothing here compiled for it, so an - # availability list that named tvOS/iOS/macOS and then fell through to `*` broke the visionOS - # build silently. `*` resolves to the declared visionOS floor (1.0), so any API newer than that - # is a compile error on visionOS and on no other platform. Nobody here owns a Vision Pro, which - # makes this job the only place the platform is exercised at all. - build-visionos: - name: xcodebuild (visionOS Simulator) - runs-on: macos-15 - timeout-minutes: 30 - steps: - - uses: actions/checkout@v4 - - - uses: maxim-lobanov/setup-xcode@v1 - with: - xcode-version: latest-stable - + # #344: the package has declared visionOS since 6.0.0 and nothing here compiled for it, so an + # availability list that named tvOS/iOS/macOS and then fell through to `*` broke the visionOS + # build silently. `*` resolves to the declared visionOS floor (1.0), so any API newer than that + # is a compile error on visionOS and on no other platform. Nobody here owns a Vision Pro, which + # makes this step the only place the platform is exercised at all. It rides in this job so CI + # stays within the account's concurrent macOS runners instead of queueing a sixth job. - name: Build for visionOS Simulator run: | xcodebuild build \ @@ -174,61 +132,30 @@ jobs: -derivedDataPath .build/xcode-visionos \ CODE_SIGNING_ALLOWED=NO - # The Xcode 27 lane. Every job above resolves `latest-stable` on macos-15, which is Xcode 26.3, - # while the engine is consumed and released from Xcode 27, and Swift 6.4 rejects isolation shapes - # 6.3 accepted (the Sodalite test target stopped compiling on exactly that, 2026-09-16). These two - # jobs compile the package and both test targets against the 27 toolchain, for macOS and for the - # tvOS 27 SDK. - # - # They build and do not run the tests, on purpose. `xcode-27` is GitHub's preview image and runs a - # BETA macOS 27 (26A5406e) under Xcode 27 beta 6: measured on two attempts of the first run, Vision - # text recognition returns nil there and the loopback-origin tests starve (a global-queue block not - # scheduled within 30 s, a TLS relay request timing out), a different set of network tests each - # time, while all 2983 pass on macOS 27 GA on a Mac. A lane that is red for its image teaches - # everyone to ignore red. Run `swift test` here once a GA macOS 27 image ships, and that is also - # the point to move to its label; the preview label disappearing fails these jobs loudly first. - # This lane replaces the weekly beta SDK watcher, whose one subject (#351) the 7.0 platform floor - # closed. - build-macos-xcode27: - name: swift build --build-tests (macOS, Xcode 27) - runs-on: xcode-27 - timeout-minutes: 30 - steps: - - uses: actions/checkout@v4 - - - name: Show toolchain - run: | - swift --version - xcodebuild -version - - - name: swift build --build-tests - run: swift build --build-tests - - build-tvos-xcode27: - name: xcodebuild (tvOS Simulator, Xcode 27) + build-ios: + name: xcodebuild (iOS Simulator) + release aetherctl runs-on: xcode-27 timeout-minutes: 30 steps: - - uses: actions/checkout@v4 - - - name: Build for tvOS Simulator - run: | - xcodebuild build \ - -scheme AetherEngine \ - -destination 'generic/platform=tvOS Simulator' \ - -derivedDataPath .build/xcode-tvos \ - CODE_SIGNING_ALLOWED=NO + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - # See the tvOS job above for why the test target needs its own build and why x86_64 is excluded. - - name: Build the tests for tvOS Simulator + # The other half of the gate check above; see the tvOS job for why the test target needs its + # own build and why x86_64 is excluded. + - name: Build the tests for iOS Simulator run: | xcodebuild build-for-testing \ -scheme AetherEngineTests \ - -destination 'generic/platform=tvOS Simulator' \ - -derivedDataPath .build/xcode-tvos-tests \ + -destination 'generic/platform=iOS Simulator' \ + -derivedDataPath .build/xcode-ios \ EXCLUDED_ARCHS=x86_64 \ CODE_SIGNING_ALLOWED=NO + # Every other Swift step in CI is Debug, and aetherctl is not a product, so no consumer compiles + # it either: a DEBUG-only engine hook called without a guard kept Release broken from 7.6.0 to + # 7.14.0 without anything noticing (AE#610). Off the `swift test` job's critical path. + - name: swift build -c release (aetherctl) + run: swift build -c release --product aetherctl + # The docs site (aetherengine-website) validates every heading anchor at build time, but it # lives in another repo and only builds AFTER a push to main, so a broken anchor ships first # and is found second: `docs/api.md` linked `formats.md#dolby-vision` against a heading that @@ -240,7 +167,7 @@ jobs: runs-on: ubuntu-latest timeout-minutes: 5 steps: - - uses: actions/checkout@v4 + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - name: Validate relative links and heading anchors in the published docs run: python3 Scripts/check-doc-links.py diff --git a/.github/workflows/release-dmg.yml b/.github/workflows/release-dmg.yml index 9bec23dff..a0d03371c 100644 --- a/.github/workflows/release-dmg.yml +++ b/.github/workflows/release-dmg.yml @@ -49,9 +49,14 @@ jobs: runs-on: macos-15 timeout-minutes: 30 steps: - - uses: actions/checkout@v4 + # workflow_dispatch checks out the ref picked in "Use workflow from" + # (normally main), so the tag is pinned explicitly: the DMG is named, + # versioned and uploaded for the tag and must be built from it (audit OPS-101). + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + ref: refs/tags/${{ github.event.release.tag_name || inputs.tag }} - - uses: maxim-lobanov/setup-xcode@v1 + - uses: maxim-lobanov/setup-xcode@ed7a3b1fda3918c0306d1b724322adc0b8cc0a90 # v1.7.0 with: xcode-version: latest-stable @@ -65,6 +70,7 @@ jobs: fi echo "tag=$TAG" >> "$GITHUB_OUTPUT" echo "Building for tag: $TAG" + echo "Building commit: $(git rev-parse HEAD)" - name: Import Developer ID certificate into temporary keychain env: diff --git a/.github/workflows/used-by-add.yml b/.github/workflows/used-by-add.yml index 17eec145f..79386b3ec 100644 --- a/.github/workflows/used-by-add.yml +++ b/.github/workflows/used-by-add.yml @@ -26,7 +26,7 @@ jobs: runs-on: ubuntu-latest steps: - name: Checkout - uses: actions/checkout@v4 + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - name: Normalise the issue body id: normalise @@ -62,7 +62,7 @@ jobs: - name: Parse issue form id: parse - uses: stefanbuck/github-issue-parser@v3 + uses: stefanbuck/github-issue-parser@cb6e97157cbf851e3a393ff8d57c93a484cc323f # v3.2.5 with: template-path: .github/ISSUE_TEMPLATE/used-by-submission.yml issue-body: ${{ steps.normalise.outputs.body }} @@ -151,7 +151,7 @@ jobs: - name: Open pull request if: steps.build.outputs.skip == 'false' id: pr - uses: peter-evans/create-pull-request@v6 + uses: peter-evans/create-pull-request@5f6978faf089d4d20b00c7766989d076bb2fc7f1 # v8.1.1 with: # Only the README, so no stray runner file can ride along. add-paths: README.md diff --git a/CHANGELOG.md b/CHANGELOG.md index c91fd1c81..7df7ea50d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -22,22 +22,832 @@ the public-API contract. - Live HTTPS-to-HTTP redirect tests cover provider refusal, anonymous redirects, static-header stripping and the downgrade credential filter. - `ExternalSubtitleTrack.httpRequestAuthorization` supplies refreshable headers for primary/secondary sidecars and native subtitle stores without changing registered track IDs or rendition mappings. Authorized container decoding retains AVIO streaming and range access. - `HTTPRequestAuthorization.data(from:maximumBytes:)` fetches raw auxiliary resources such as font bundles with a caller-supplied byte limit and a whole-transfer deadline, reusing the relay's redirect, authorization, retry, cancellation and TLS policy. - - `LoadOptions.httpRequestAuthorization` accepts an async `HTTPRequestAuthorization` resolver for native HLS. The engine resolves headers before requests and redirects, and retries a rejected request once when the bearer changes, preserving the active player item across token rotation. +- `setSoftwareSubtitleDelay(_:)` applies a persistent, caller-selected subtitle offset to both channels in software PiP without changing the playback clock. Native renditions remain unchanged. +- `needsForegroundVideoRestore` exposes actual background video teardown so a host can rebuild through the existing reload API before resuming. ### Fixed - The software video decoder no longer runs more than 16 frame threads. It used one per core, and each frame thread holds back one decoded frame, so a 32-core Mac waited for 31 frames before showing the first one after a load or seek (about 3 s at 10 fps). FFmpeg also warns above 16 threads. Hosts with 16 or fewer cores keep their current thread count. -- A paused video no longer starts playing by itself. When the player item died while paused (`failedToPlayToEndTime`), the recovery reload bypassed the pause guard and called `play()` on the fresh item. The reload now keeps a pause made before the item died, whether it came through the engine, AVKit, Control Center or PiP, and mounts the item paused at the same position. -- A dead item's recovery no longer restarts the title from where the session was first opened. When AVPlayer refused the recovery item's master (`-11868`), the media fallback reloaded at the first mount's start position, so a title opened from its beginning restarted at 0:00. The fallback now reloads where the refused item was placed. Upstream #621. -- The media fallback no longer starts a paused title. When the recovery item was refused, the fallback called `play()` unconditionally, so a title paused behind the tvOS screensaver started itself. It now plays only when the refused item was playing, or was told to play, and the viewer had not paused it. A Play or Pause from AVKit, Control Center or PiP counts as well as one through the engine. - TrueHD Atmos rendered to APAC (`LoadOptions.objectAudioRendering`) no longer plays 42.7 ms ahead of the video. The bridge dropped the encoder's 2048 frames of priming and stamped the first content packet on the source position, but AVFoundation presents an APAC packet's audio 2048 frames before its timestamp, so every session ran early, at load and after every seek. The priming packets now stay in the stream and take the source position's timestamp. - Authorized native HLS uses the engine relay from the initial load, without forwarding origin credentials to the loopback asset. Optional subtitle playlist preparation shares the authorizer and has a bounded deadline across redirects and refreshes. - Static-header HLS redirects apply the shared credential policy, including Emby and MediaBrowser token headers, before contacting another origin. - Session option corrections recognize `httpRequestAuthorization` and external subtitle provider replacements by identity. - Sidecar decoding runs on a dispatch worker instead of a cooperative-pool thread, so concurrent authorized decodes cannot starve the async resolvers their requests wait on. +- A pause made through the engine while the premature-end recovery re-seeks (AE#287) is kept. The recovery called `play()` after its seek whatever the play intent said, so the title started again. +- An external ASS sidecar the host draws itself (`preserveASSMarkup`) on the `nativeRemoteHLS` bypass keeps its plain rendition selected behind a suppressing legible output instead of deselecting it. AE#616 measures `sourceTime` off the selected injected rendition, so the deselect left the clock a styled-ASS overlay draws from on item time, 1 to 8 s ahead of the picture after a seek on a transcode restarted at an earlier keyframe. The hidden rendition is selected only once its whole-program WebVTT has been extracted and playback is running, because AVPlayer fetches and buffers a selected rendition and AE#616 measures only presented lines. Turning subtitles off takes it down immediately. +- TrueHD Atmos rendered to APAC reports a decoder that produces nothing (`.audioBridgeProducedNoOutput` on VOD), as the channel bridges do since AE#641, instead of playing the picture over silence. +- Playlist preflight through a refreshable `HTTPRequestAuthorization` keeps the 2 MB playlist cap the static-header preflight has (audit NAT-5), and relayed fetches use upstream's per-fetch pending cap (audit NET-107) in place of the fork's high-water wait. +- A 206 whose `Content-Range` total is zero or negative no longer crashes an HTTP source as it opens. `Int64` parses a signed total, so `bytes 0-0/-9223372036854775808` reached `total - 1` with `Int64.min` and trapped. +- Below 1x, or after a pause in the last quarter second, the audio-only host parks its clock when it reaches the last sample. It parked after the tail's length in wall time, which cut up to 0.125 s at 0.5x. +- A delayed revive of a dead live producer is checked again when its restart is admitted, so a seek restart that began in between no longer gets the stale position queued behind it. +- The I-frame payload cache refuses a payload larger than its byte limit instead of holding it over the cap. The payload is still served. +- Closing the DVR packet ring releases the writer's open chunk instead of keeping it until the ring is freed. +- Remote-HLS cue normalization strips markup in one pass. Removing one tag at a time and searching again from the start took seconds on the main actor for a crafted 64 KiB cue. +- Log redaction covers an Xtream password containing `,` `'` or `)`, which used to end the path segment early, and a registered secret that starts a credential name such as `token` no longer hides the value after it. +- `aetherctl`: `--served-url` and the served-VTT readout redact origin URLs, a localhost origin included (the engine's own loopback URLs still print in full), the readout resolves rendition and segment URIs against the playlist that lists them, and `smb-test`'s sequential pass no longer scans every probe for every chunk inside its timed window. +- `Scripts/hls-burst-origin.py` rejects a `--prefill` or `--freeze-at` that the folder's segments cannot satisfy instead of failing with an `IndexError`. +- `reloadAtCurrentPosition(applying:)` refuses a `sharedOutputRole` change with `loadIdentityNotCorrectable`. The role was listed as load identity but never compared, so a correction rebuilt a multiview engine under the other role. +- Seven upstream tests no longer turn a timeout into a hang, rely on main-executor ordering after `Task.yield()`, race a closed socket, or write a fake's state from another queue without a lock. - Native subtitle renditions wait for complete extraction instead of caching a partial whole-track response. While extraction is pending, the local server returns a retryable HTTP 503. - External subtitles can declare `nativeTimelineOffsetSeconds` for media reanchored upstream. Native HLS/PiP/AirPlay renditions use the adjusted timeline while host overlay cues retain source timestamps. +- Video routing consults the AV1 hardware decoder only for AV1 sources, avoiding supplemental decoder registration on unrelated playback starts. +- Rapid audio-track selections coalesce into serialized rebuilds. Stop/load invalidate queued work, and play/pause commands received during a rebuild supply its final transport intent. Native handover retains the old item until replacement unless media services were reset. +- A `play()` or `pause()` that arrives while any session-preserving rebuild runs (an audio or disc-title switch, `reloadAtCurrentPosition()`, an engine-raised rebuild) now decides the transport it comes back in; before, only an audio pick honoured it and the others came back in the state they started from. + +## [7.28.3] - 2026-10-07 + +### Fixed + +- **A resume or seek onto an open-GOP keyframe keeps the #409 repair (#699).** Two defects on H.264/HEVC MP4s without composition offsets. The repair verdict was reached at the first read, and the software host seeks to its resume position before reading, so the sample started on a CRA (picture order not 0) and was declined: the whole session played in decode order. A seek before the first read now settles the verdict at the head first. Separately, every post-seek re-anchor treated the landing keyframe as the first picture displayed; a CRA with leading pictures is displayed that many slots later, so the axis sat early and B pictures that fell below their decode time went out untouched. The session now holds from the landing to the first trailing picture and anchors with the leading-picture count. Covered by an x265 open-GOP twin (CRA plus four RASL pictures) for a resume-shaped seek and a seek during playback. Reported by cmcpherson274. No API change. + +## [7.28.2] - 2026-10-06 + +### Changed + +- **A bounded live start on a source the engine cuts itself no longer waits a second grace (#686).** AVPlayer opens with two plain `/media.m3u8` requests, and on raw MPEG-TS under `.fastZap` each one waited out the bounded-start grace; #684's first-serve latch spared only ingest sessions. It now covers engine-cut sources too, the behaviour 7.26.3 put behind `AETHER_FIRST_SERVE_LATCH_ALL=1`. Measured by cmcpherson274 on an Apple TV 4K (1080p59.94 H.264 + AAC, 1.001 s segments, three runs per cell): every bounded start reached its picture 1.003 to 1.039 s sooner, the session sat 0.5 to 1.5 s nearer the edge after 60 s, and 18 latched launches logged no stall, `-16832` or `-12888`. Rebuilds from a backlog served a full cushion either way. `AETHER_FIRST_SERVE_LATCH_ALL=0` restores the old gate. No API change. + +## [7.28.1] - 2026-10-06 + +### Fixed + +- **`[SWDiag]` reports `aLead` on live DVR sessions as well (#395).** 7.27.2 added the marker to the combined demux loop only. A live session loaded with `dvrWindowSeconds` feeds its audio from the ring pump, which never wrote it, so its line still read `aLead=-` throughout. The line now reads the pump's own fed PTS there, the value the pump paces on. Measured with `aetherctl dvr --path sw`: `aLead=-` before, `aLead=4.00`, `3.12`, `2.12` after. Diagnostic only, pacing unchanged, no API change. +- **HEVC with a parameter-set change mid-title keeps its picture.** On an Annex-B HEVC source (Blu-ray M2TS, broadcast TS) movenc converted the samples itself and, under the `hvc1` sample entry, dropped every in-band VPS/SPS/PPS, so a stream that sends a new PPS mid-title had every later slice decoded against the stale one: sound continued over a frozen or black picture. The session muxer now gets a length-prefixed record and converts the samples itself with their parameter sets kept, the shape a Matroska remux of the same stream already had; `init.mp4` and `CODECS` are unchanged. The I-frame rendition muxer takes the same path. Measured on a fixture whose PPS changes at 20 s: 463 presented frames in 42 s before (none after the change), 936 after. Diagnosed by yipengfei329 (#703). +- **A keyframe index that stops minutes before the end is no longer trusted.** A partial scan (an MKV whose Cues are missing or point past EOF) passed both the gap and the coverage check, and the keyframe planner then cut a final segment from the last scanned keyframe to the end of the title, which the producer can never finish. The index now also has to reach within 60 s of the source duration, otherwise the session takes the uniform plan. Diagnosed by yipengfei329 (#703). + +## [7.28.0] - 2026-10-06 + +### Added + +- **HDR Vivid (CUVA) detection in the opt-in probe (#699).** `ProbeDetail.hdrVivid` scans HEVC packets for the CUVA T/UWA 005.1 T.35 SEI and sets the new `SourceProbe.carriesHDRVividMetadata`. It runs in the same packet pass as `.hdr10Plus` and under the same `HDR10PlusDetectionOptions` budget; asking for both still opens one connection, and a cap that ends the pass after one of the two was confirmed keeps that confirmation. libavcodec's CUVA parser is internal, so `HDRVividMetadataScan` walks the body with the same field widths and only counts a complete message with a defined `system_start_code` and zero bits after its last field. `videoFormat` does not move (Vivid rides an HLG or PQ base, which stays the label), and playback is unchanged: the SEI is stream-copied as before and no Apple platform applies it. `aetherctl probe --detect-hdr-vivid` prints the finding, and `Scripts/make-hdr10plus-fixture.py --vivid` builds a fixture ffprobe parses as Vivid. Requested by ijuniorfu. + +## [7.27.3] - 2026-10-05 + +### Fixed + +- **An HEVC MP4 without composition offsets is presented in display order (#699).** The #409 repair only armed on H.264, so an HEVC file whose writer dropped `ctts` while the bitstream reorders pictures went to AVPlayer and the software decoder in decode order, which shows as flicker and back-and-forth motion. The repair now reads the picture order count with libavcodec's HEVC parser as well, and its window check also accepts a hierarchical mini-GOP longer than the reorder delay (five pictures at delay 2 on the reporting asset). Measured on that asset (4K50 Main 10, 1500 frames): 0 backward steps in the served stream, against 35 of 88 in the first frames before. The partial-region repair stays H.264-only. Reported by ijuniorfu. + +## [7.27.2] - 2026-10-05 + +### Added + +- **The software path names its audio route, its audio lead on live, and its renderer state (#395).** A live software session that was silent on an AirPlay 2 receiver and audible over HDMI logged the same on both. The software host now writes `[SoftwarePlaybackHost] audioRoute ...` at session start with the route's output latency (same fields as the native host's line), the engine writes `[AetherEngine] audioRoute changed reason=N ...` on every route change once per process, `[SWDiag]` reports `aLead` on live sessions too (diagnostic marker only, pacing unchanged), and it carries `aRend=status/sufficient/error` for the audio renderer, whose error was only logged in DEBUG builds before. No API change. + +### Fixed + +- **An end-of-media park that runs late still stops the clock on the last sample (#694).** The park is deferred by the queued audio tail and ran wherever the clock stood when its task got the main actor, so a busy main actor parked it past the end (measured: 1.149 s on a 1.0 s source). Both the audio-only and the software host now put the clock back on the point the deferral aimed at. + +## [7.27.1] - 2026-10-04 + +### Fixed + +- **The audio-only path's clock parks at end of media instead of free-running past it (#694).** `AudioPlaybackHost` reported `.ended` while its synchronizer kept rate 1, so `currentTime` grew past `duration` for as long as the session stood. It now gets the AE#374 park the software host has: the queued tail plays out, then the clock stops on the last sample, and `play()` no longer restarts a clock the source stopped. Reported by ijuniorfu. + +## [7.27.0] - 2026-10-04 + +### Fixed + +- **One refusal at byte 0 no longer settles a range-capable VOD forward-only for the whole session (#693).** An Xtream origin answered the ranged open of an episode with a 403 and the unranged GET right after it with a 200. Since #378 every 401/403/404/410 at byte 0 was read as the answer to the resource, so the source was served as a sequential origin: `startPosition` dropped and every seek past the downloaded window snapped back. The open now asks the same range once more before it settles: served, the source stays seekable; refused again, the #378 path runs unchanged. A source that refuses everything costs one request more and still fails typed with its status. + +### Added + +- **`isSequentialOrigin` (#693).** Published, true while a VOD session is served as a sequential origin, declared or found by the reader. Such a session plays from the start (a `startPosition` was not honoured) and a seek lands only inside `$residentRanges`, so a host can reopen the source itself or tell the user instead of reading log lines. + +## [7.26.3] - 2026-10-03 + +### Added + +- **A measurement arm for the first-serve latch on engine-cut sources (#686).** On raw MPEG-TS under `.fastZap` a bounded start held AVPlayer's second plain `/media.m3u8` request for a second grace (1.0 to 1.4 s on the reporter's Apple TV). `AETHER_FIRST_SERVE_LATCH_ALL=1` applies #684's latch to those sources too; read once per process, off by default, so default behaviour is unchanged. Both arms now log how long that second request waited (`repeat live manifest request held Xs`), which was silent before. No API change. + +## [7.26.2] - 2026-10-03 + +### Fixed + +- **One frame more bought a whole second of `.fastZap` TARGETDURATION (#670 round 2).** The GOP headroom term took `ceil(1.5 x max EXTINF)`, so on a 59.94 fps source with 1.001 s GOPs a 79-frame segment (1.318 s) sealed 2 and an 80-frame one (1.335 s) sealed 3: a 9 s holdback, and a 6 s rebuild backlog then started bounded and presented 2.5 s later. The term now asks what it means, whether a GOP 1.5 x the longest seen would list legally (RFC 8216 rounds EXTINF to the nearest integer) and finish inside AVPlayer's `1.5 x TD` patience with 0.5 s to spare, so segments up to 1.666 s seal 2. Never above the old value; 1 s GOPs keep TARGETDURATION 2. The seal line names what the term needs (`1.5 x max EXTINF 2.002s needs 2s`). No API change. + +## [7.26.1] - 2026-10-03 + +### Fixed + +- **A software-path rescue replayed the gap a host `load()` had skipped (#629).** A native VOD item mounted past the head reads, until its mount seek lands, the start of the segment AVPlayer decodes up from (12.00 s under 15.90 s on the reporter's Apple TV), and the session published that reading. Once the host's `load()` had returned nothing else remembered 15.90 s, so an item AVPlayer refused in that window was rebuilt at 12.00 s and the film replayed 3.9 s after the rescue. The clock hold #646 gave the in-place swap now covers that mount as well: the session reads the position it was mounted at until the mount seek lands or the item plays, and the rescue resumes there. No API change. + +## [7.26.0] - 2026-10-02 + +### Added + +- **Scrub thumbnails in the stock player (#682).** `LoadOptions.serveIFramePlaylist` makes a native VOD session list an I-frame rendition in its master, which is the one input `AVPlayerViewController` takes thumbnails from: a host that presents the system transport bar gets them, and I-frame fast forward and rewind, without writing any UI. Each entry is the source's own keyframe at full resolution, read by a second reader and stamped on the main rendition's timeline. Off by default. The rendition is left out entirely, never half-served, where the session cannot answer every keyframe (live, MPEG-TS, single-connection origins, discs, media-playlist routing); the log names the reason. + +## [7.25.3] - 2026-10-02 + +### Fixed + +- **Engines running side by side each claimed a quarter of the same free space (#687).** A session sizes its disk allowance once, at its start, as `min(2 GiB, a quarter of the tmp volume's free space)`. An allowance another session had claimed but not yet written was still free at that moment, so four engines started together (a multiview host) could claim four quarters: on 4 GiB free, the whole volume. A session is now sized from the free space minus what the other running sessions may still write. One session alone gets exactly what it got; four on 4 GiB free take 2.7 GiB instead of 4, and n sessions always leave at least `(3/4)^n` of the volume. The live window already follows its allowance (#443), so a smaller share means a shorter timeshift depth on the later tiles, and the session log says how much was held back. Covers the native session, the software live ring and the software VOD read-ahead; no API change. + +## [7.25.2] - 2026-10-02 + +### Fixed + +- **A live HLS ingest sealed its TARGETDURATION under the upstream's own segment length (#684).** The engine re-cuts each upstream segment (at its GOPs under `.fastZap`), so the upstream's segment entered the seal only as `ceil(segment / 1.5)`: a 6 s provider sealed 4, a client patience of exactly one delivery and a 12 s holdback against deliveries that arrive 6 to 9 s apart. AVPlayer drew `-12888`, skipped its next playlist reload and stalled with the content already listed. The seal now asks for the longest upstream segment served (up to 10 s) and takes as much of it as the join pays for: + - uniform 6 s segments under `.fastZap` seal 5 (holdback 15 s), uniform 10 s segments under either profile seal 9 (27 s), with the join unchanged (three segments); + - alternating lengths: the join is equalised over the phases of one upstream, so a tune never loads more than the deepest tune of the same channel already did. The reporting channel alternates 6 s and 4 s; its join used to be three or four segments depending on which was newest, and only the four-segment phase got more than 7.25.1's value. Both phases now join four and seal 6 (18 s). Shapes whose phases already joined alike keep their join and what it pays: 10 s / 8 s seals 8 (7 before), 6 s / 6 s / 4 s keeps 4, and a shape like 6 s / 5 s keeps a seal that follows the phase (5 or 4); + - 2 s providers keep 2, 3 s providers with 1 s GOPs keep 2 (their join pays for no more), and `.standard` is unchanged up to 6 s segments. + + Measured on a uniform 6 s origin with jittered deliveries over 180 s: 5 `-12888` and a stall before, 3 and no stall after; over nine 60 s runs 7 stalls before and none after. On the alternating 6 s / 4 s shape: 0 to 1 `-12888` per run before, none after, in both phases. On 10 s segments the window close and item rebuild that 7.25.1 produced in 4 of 9 `.fastZap` runs and 6 of 6 `.standard` runs did not occur. Expect far fewer stalls and fewer `-12888`, not none: the reporting capture had 10 of 110 delivery gaps above 9 s. + + Costs. Standing latency: 3 s more behind the upstream's edge on uniform 6 s segments and on 10 s / 8 s, 6 s on alternating 6 s / 4 s and on 10 s ones. Join: unchanged on uniform upstreams and on every shape whose phases joined alike; on alternating 6 s / 4 s about four tunes in ten load one additional short segment (3.47 s to 4.23 s to first picture behind an 8 Mbit/s link; the other six tunes already took 4.27 s). A host that allows fewer parallel requests than the join has segments (`maxConcurrentSourceRequests` of 3 or less) sees its first picture later, because the gate waits for the join to be handed over where it used to serve at 12 s of cut content: 2.79 to 3.08 s before and 3.27 to 3.49 s after, one request at a time behind 14 Mbit/s on the 6 s / 4 s shape. A join slower than the `.fastZap` grace is still served by the bounded start and then seals the full value over a window under its holdback (a `-16832` warning at the start). +- **On an ingest the first-serve gate held the second playlist request too (#684).** On a `.fastZap` bounded start AVPlayer's second opening request waited out a second grace: 4.47 s to first picture where one grace is 2.24 s. Sources the engine cuts itself (raw MPEG-TS) are unchanged. +- **The AE#440 forced start is no longer taken on an item that cannot play yet (#684).** On a rejoin placed by `EXT-X-START`, AVPlayer fetches from about 6 s below the target first, and the guard read that lookback as its cushion (`buffer ahead 4.00s` from a playhead the item never played from) and called `playImmediately` 36 ms before the item was ready. The decision now waits for `readyToPlay` and is asked again there. Whether this is what put sound and picture apart on the reporting device is not confirmed. + +### Added + +- **Each native live item says what it started on (#684):** `#684 item #N starts at its own Xs: segment K + Ys, whose picture opens at ... and whose sound runs ...`, and every `live seg-N finalized` line carries `sound=first..last`. The audio route line adds the route's output latency. +- **`aetherctl play --served-url`** prints the loopback URL of each native item, and **`Scripts/hls-burst-origin.py`** is a live HLS origin whose delivery rhythm is scripted (jitter cycle, outage, window depth, mixed and inflated EXTINF, shared link rate). + +### Changed + +- `HLSVideoEngine`'s public initializer gained two defaulted parameters, `liveJoinBacklogObservation` and `liveJoinSpentObservation`. Source compatible; a host that constructs the session itself passes nothing new. + +## [7.25.1] - 2026-10-01 + +### Fixed + +- **Cancelling the task that awaits `load()` now ends the load at once (Sodalite#173, #683).** A `.custom(HLSLiveIngestReader)` load against an unreachable provider kept running for 19 s (`open_input 18987ms` while the ingest retried its playlist), and a host that cancelled it to tune the next channel had to wait it out. The load now throws `CancellationError` within tens of milliseconds for URL and `.custom` sources, and leaves the engine the way a newer load would: `.idle`, native host and AVPlayer kept for the next load, display criteria and audio-session membership untouched; a host that leaves on cancel calls `stop()`. Generation-guarded, so a cancel never touches a session a newer `load()` or `stop()` started. A cancelled AE#629 follower also ends the software-path rebuild, nested reroutes included. +- **A superseded custom-source `reloadAtCurrentPosition()` throws `CancellationError`** like the URL branch already did, instead of returning normally and logging a false "rebuilt on the software path". +- **A closed `AVIOReader` no longer waits out the 0.75 s probe-fallback delay** (about 0.03 s now), which also shortens `stop()` during a URL open. + +## [7.25.0] - 2026-10-01 + +### Added + +- **Several engines in one process share the audio session, its channel preference, the panel and Now Playing correctly (Sodalite#175, #681).** Measured with two live tiles on an Apple TV: every stop that released the session paused the OTHER engine within 10 ms, picture frozen, 4 of 4. An internal coordinator now tracks which engines are active (join on `load()`, leave on `stop()`): the session is released only by the last engine out and only when a release is owed in that round, the preferred output channel count is the widest playing source, display-criteria resets (on stop and the two load-time clears) wait for the last engine, and activation and release run on one process-wide queue (AE#538 ordering across engines). Owners are held weakly, so an engine released without `stop()` cannot pin the session. A host with one engine sees no change. See `docs/api.md`, "Running several engines at once". +- **`LoadOptions.sharedOutputRole`** (`SharedOutputRole.primary` default, `.secondary`). A secondary never writes display criteria, skips the play-gate wait for a criteria write nobody will make, and never claims Now Playing, on the video and the audio-only path. Identity field. +- **`AetherEngine.logTag`**: a short instance name carried by the `[SharedOutput]` lines and a `[AetherEngine:] state=` line per transition. Set it before `load()`. +- **`aetherctl play` checks the served `.vtt` against the window its playlist declares (#680):** segments are picked at the playhead by walking EXTINF, `MISPLACED=N` names cues outside their segment's window, and an empty served `.vtt` is now a visible log line. + +## [7.24.0] - 2026-09-30 + +### Fixed + +- **`.fastZap` on an HLS upstream with long segments always ended at the bounded start (#678).** The served TARGETDURATION on an ingest is floored by the longest upstream segment (`ceil(longest / 1.5)`, AE#447), so a 10 s provider seals 7 and the first serve wants 21 s, but the join covered `1.5 x` the advertised target duration: two 10 s segments, 19 s once the last GOP stays open. Short by construction for every segment above ~5 s, so every zap waited out the grace on content the origin already held. The join now covers that holdback plus one GOP (at most 4 s). Measured on 10 s segments: first picture t+2.2 to 2.5 s -> t+0.22 s on loopback, t+4.1 to 4.4 s -> t+2.9 s behind a 40 Mbit/s link. +- **The HLS live ingest ignored `maxConcurrentSourceRequests` (#678).** It charged nothing to the origin request budget and fetched four segments in parallel, so a host declaring `1` for a single-connection provider still opened four requests to it. Playlist, segment and key fetches now take a budget ticket, report refusals and redirects, and the prefetch window narrows to the limit; a direct `.custom(HLSLiveIngestReader(...))` load registers the ceiling on its playlist URL. +- **A same-format load kept the panel in its UI mode after the host cleared the display criteria itself (#678).** The unchanged skip compared against the engine's own record only; it now also reads `preferredDisplayCriteria` back and writes again when it finds nothing set. + +### Changed + +- **A live URL known to be an HLS playlist skips the raw probe (#678).** A `.m3u8` / `.m3u` path, or a URL that already took the AE#363 reroute in this process, goes straight onto the live ingest instead of spending one discarded request per zap. + +### Added + +- **`[Demuxer] open timings`** on every playback, probe and restart open: connect, `open_input` and `find_stream_info` separately, the stream count and the probe budget, also when the stream analysis fails (#678). + +## [7.23.2] - 2026-09-30 + +### Fixed + +- **A Dolby Vision master was refused when the source stated HEVC level 5.2 (#674).** Encoders stamp level 5.2 (156) on 1080p24 streams, the level went into the master's `CODECS` verbatim, and an Apple TV 4K screens a declared level against the device: the DV master and the reduced HDR master both failed with -11848 / CoreMedia -15517, and the session ended on the media playlist with Dolby Vision dropped. The declared level is now capped at 5.1 (`L153`, which covers 2160p60), for the hvcC-derived string and the fallback alike; the init's hvcC keeps the stated level. The DV routes (P8.1, P8.4, P7) also derive their `CODECS` from the hvcC now, so they carry its constraint bytes like the plain HEVC route has since AE#187. Contributed by @gonkowonko (#677). + +## [7.23.1] - 2026-09-30 + +### Fixed + +- **`.fastZap` live froze on sources with irregular GOPs (#670).** The engine's sub-second cut makes every segment one whole GOP, and TARGETDURATION was sealed from the first few with no headroom over them: a broadcast whose GOPs run 1.0 to 2.4 s sealed 1 on three 1.000 s ones, then broke `EXTINF <= TD` on every longer GOP and held the playlist unchanged past AVPlayer's patience (-12888, #524 on 1.5 s of runway, and on macOS the playlist refused outright). Self-cut fastZap segments now seal `ceil(1.5 x max EXTINF)`, the headroom `.standard` always had from its cut-target floor, so 1 s GOPs serve TARGETDURATION 2 and a 6 s holdback. Ingested segments and `.standard` are unchanged. + +## [7.23.0] - 2026-09-30 + +Second full audit of the engine (2026-09-29, against 7.22.2): about 107 verified defects (3 High, 35 Medium), including regressions and incomplete fixes the first audit's patches left behind. All but three performance items are fixed here, each with a regression test that failed before it; the embedded-subtitle double download (PERF-103), the byte-bounded fragment flush (PERF-105) and the audio tap readers' cost (DEC-108) need a device round or a larger redesign and are deferred. + +### Added + +- **`EngineLog.redacted(_:)`** returns a line exactly as the log handler would receive it, so a host or tool can print a URL through the same redaction funnel (audit OPS-106). + +### Changed + +- **Durations past a week are refused or clamped.** An HLS `#EXTINF` / `#EXT-X-TARGETDURATION` that is non-finite or over 7 days, or a VOD playlist summing past 7 days, fails the load with `playlistInvalid`; a container duration over 7 days is reported as 7 days (audit NET-101, HLS-102). +- **A server that ignores HTTP Range plays forward-only.** It is detected at the response head (confirmed once with a one byte range) and played like `sequentialOrigin`: no seeking, `AVSEEK_SIZE` still answers the Content-Length. A remote disc image on such a server fails at load with `originIgnoresRange` (audit DMX-101, NET-102). +- **Remote HLS bodies are size-capped at the response head.** Segments may weigh 20 MB per second of their `#EXTINF` (between 32 and 256 MiB), keys 64 bytes, live subtitle playlists 8 MiB and their segments 1 MiB; a non-2xx status ends the transfer (audit NET-112, FEA-107). +- **Credential headers stay on the host's origin.** Cross-origin redirects and playlist-named hosts on the ingest, disc, relay, subtitle proxy, audio tap and live-subtitle routes no longer receive `LoadOptions.httpHeaders`. On the native bypass AVFoundation still sends them everywhere, which `docs/api.md` now states (audit NET-108, NET-109, Vcred-104). +- **Log lines changed shape.** The local server's session token reads `//`, request lines omit it, tokenless requests collapse into one throttled line, and ingest / carriage-probe URL errors log `NSURLError from ` instead of the failing URL (audit SUB-107, NET-111, NET-114). +- **`load()` returns normally when the app backgrounds mid-load, and an audio-track switch keeps a paused session paused.** Hosts that treated a `CancellationError` from `load()` as a supersede now get a normal return followed by the background teardown (audit LIF-101, LIF-102). + +### Security + +- **A hostile HLS playlist no longer crashes the app with an infinite or huge duration.** `inf`, `nan` or values past a week in `#EXTINF` / `#EXT-X-TARGETDURATION` are refused at the parser, and a VOD playlist summing past a week is refused. Before, a live join, a live subtitle rendition or a HEVC-in-TS VOD ingest trapped on it (audit NET-101, FEA-102, HLS-102). +- **A corrupt container duration or keyframe index can no longer crash segment planning or ask for terabytes.** Durations are clamped to a week, implausible index entries are dropped, and the plan builders fail empty instead of trapping or reserving trillions of segments (audit HLS-102). +- **A crafted DVD subpicture no longer reserves up to 4 GiB.** Units above 1 MiB are dropped, the up-front reservation is at most 64 KiB, and fragments join in place instead of copying the partial unit each time (audit NET-104, DMX-106). +- **Tiny laced audio packets can no longer pin gigabytes while the video gate waits.** The pre-gate audio buffer is also capped at 65,536 entries (audit SEG-102). +- **Credential headers no longer follow a redirect to another origin on any engine session** (audit NET-108). URLSession's own redirect copies every custom header except `Authorization`, so `X-Emby-Token` and `Cookie` followed a cross-host or https-to-http 302 on the HLS ingest readers, the carriage probe, the remote disc reader, the origin relay, the subtitle proxy, the audio tap and the live rendition fetch. Every engine session now applies the #126 rule AVIOReader already applied. +- **Allowing a relay or proxy fetch no longer grants it the host's credentials** (audit NET-109, NAT-105, DEC-103, Vcred-101). After a cross-host redirect on the #316 master, the relay anchored its grant on the edge, so the token went there and the host's own origin lost it. The #316 variant fetch, the remote-HLS audio tap and the live subtitle rendition fetch sent every host header to every host a playlist named. All of them now send credentials only to the URL the host handed over. +- **The log redactor reads through percent escapes** (audit SUB-104, NET-114). An upstream URL carried percent-encoded inside a proxy or debrid wrapper kept its Xtream password, userinfo password or base64 token in `load url=`, `asset.url=` and URL error dumps. The ingest and carriage-probe errors now log the error code and host instead of the failing URL. +- **A query password holding `:` `;` `,` `)` or `>` is redacted whole** (audit SUB-108). Previously the text after the first such character stayed in the log. +- **More credential shapes are redacted** (audit SUB-109). This covers `Bearer` and `Basic` credentials, any cookie, `api-key` / `X-Api-Key`, `pwd`, `passwd`, session ids, auth, refresh and session tokens, and quoted JSON or dictionary keys. `EngineLog.registerSecret` now counts registrations. +- **The local server's session token no longer appears in the log** (audit SUB-107). Request lines, `load url=` and `asset.url=` used to hand a live LAN capability to anyone reading a shared log. +- **A LAN peer without the session token can no longer flood or forge the log** (audit NET-111). Its lines are throttled with a tally, and its request text is logged with control characters escaped. +- **The first-request header dump prints header names, not values** (audit Vcred-102). On the stand-in route AVPlayer sends the host's credentials to the loopback server too. +- **aetherctl banners print the redacted source URL** (audit OPS-106). New public `EngineLog.redacted(_:)` returns a line exactly as the log handler would receive it. +- **The local server's exposure is documented accurately.** `SECURITY.md`, `architecture.md` and `cli.md` no longer say it is loopback-only: it binds all interfaces for AirPlay behind a per-session 128-bit path token, a 32-connection cap (24 for non-loopback peers) and a 10 s head deadline. `docs/api.md` also states that on the native bypass AVFoundation sends `LoadOptions.httpHeaders` to every host and redirect target (audit OPS-102, Vcred-104). +- **The bridge memory telemetry no longer reads freed FFmpeg contexts** (audit DEC-104). `liveBytes` and `fifoSampleCount` read a snapshot published under the bridge lock, instead of calling `swr_get_delay` on a resampler or encoder that feed or restart may have just freed. +- **A crafted Blu-ray or DVD image can no longer stall or exhaust memory during disc recognition** (audit NET-103). Playlists are capped at 999 PlayItems, at most 4000 `.mpls` entries and 32 MiB of playlist data are read, retained PlayItems are budgeted, and a title is cut at 65 536 extents. Clips and IFOs are found through one name index instead of a scan per item, and a DVD only groups title sets 1 to 99. +- **The Dolby Vision record audit can no longer read a whole source looking for a frame** (audit BIT-103). The RPU walk now arms the same input byte budget the HDR10+ and Atmos probes use. +- **HLS segments, AES keys and audio tap downloads are bounded as they arrive, not after they are fully held** (audit NET-112, NET-113, DEC-103). A non-2xx status or a declared length over the cap ends the transfer at the response head; the body accumulates in chunks and is cancelled at the cap (a segment may weigh 20 MB per second of its duration, between 32 and 256 MiB; a key 64 bytes). +- **The live subtitle rendition loop no longer fetches, parses and merges on the main actor, and is bounded** (audit FEA-104, FEA-107, SUB-106). Ephemeral session with a 30 s resource timeout, status checked, playlists capped at 8 MiB, segments at 1 MiB and 4096 cues, non-finite or negative timestamps dropped, published list capped at 2000 cues. +- **Hostile subtitle and playlist text no longer costs quadratic time** (audit SUB-101, SUB-110, NET-105, NAT-102). An unclosed-brace ASS cue took 2.6 s for 20k braces on the pump thread, and a 64 KiB HLS tag line with unclosed quotes took 9.6 s. Override blocks strip in one forward pass and a cue is cut at 64 KiB; attributes scan a tag line once and a line over 64 KiB is refused. + +### Fixed + +- **Seeking, thumbnails and resume positions reject NaN, infinity and huge values instead of crashing.** The Demuxer converts seconds through a checked helper and the seek fails cleanly. `FrameExtractor` returns nil, the SW and audio hosts ignore a non-finite start position, `seek(to:)` ignores a non-finite target, and the byte-estimate seek saturates at an inflated `Content-Range` total (audit DMX-113, BIT-105, DMX-103). +- **`dvrWindowSeconds: .infinity` no longer crashes the first live manifest request.** The window is clamped to a day inside the live window sizing (audit SEG-107). +- **A non-finite or absurd audio sample rate on the remote-HLS bypass reads as 0 instead of crashing at readyToPlay** (audit NAT-106). +- **ASS `\pos(nan,nan)` or `PlayResX: inf` no longer publish a NaN placement.** The cue falls back to its alignment (audit SUB-112). +- **aetherctl rejects huge or negative numeric flags with exit 64, and `pktdump` survives overflowing dts deltas** (audit OPS-110, SUB-112). +- **Going to the background during a load no longer makes `load()` throw `CancellationError`** (audit LIF-101, LIF-105). The teardown owed since `didEnterBackground` ran when the native load passed item readiness, seconds before it returned, so the host read a supersede and left its spinner up without a start report. It now runs after `load()` returns. A PiP window closed in the background during a load or seek is torn down once it settles, instead of never. +- **An audio-track switch on a paused session stays paused** (audit LIF-102). The same goes for every custom-source `reloadAtCurrentPosition` (audio-delay nudge, option correction), including a mount with `autoplay = false`. +- **An audio or disc-title switch after a media services reset builds a fresh AVPlayer** (audit LIF-104). +- **A callback from an ended session no longer reaches the next one** (audit LIF-103, Vcore-101, Vcore-102). No `.error` on a stopped engine from a late escalation, no reader stall or scrub state carried into a successor, and no stale master rejection replayed by a reused host. +- **Cancelled or superseded loads stop waiting on a display mode switch at the next tick, and the #667 wait and the play gate share one 6 s cap** (audit LIF-106, DEC-105). +- **An in-place item swap releases its clock hold only on the fresh item's own playback** (audit NAT-103). +- **A load superseded during its Dolby Vision RPU audit no longer writes its geometry over its successor's** (audit LIF-107). +- **A live channel the engine reconnects keeps playing instead of freezing on its last segment** (audit HLS-101). Only the session's first producer reported its segments to the playlist. A live reopen, a muxer-failure rebuild or the AE#222 rebuild cut new segments that the playlist never listed, so AVPlayer froze after "live reopen succeeded" until the host retuned (Sodalite) or the viewer acted (AetherPlayer). Every producer the session builds now reports into the playlist. +- **A rate-limited source's delayed revive no longer drags a recovered session back to where it failed** (audit HLS-104). The #377 revive waited 3 to 45 s and then re-anchored whichever producer was running, usually the working one a seek or segment request had already put in. It now runs only while its dead producer is still installed and the session is unchanged. +- **A seek that re-cuts a keyframe drought keeps the clock and captions on the picture when the re-cut opens late** (audit HLS-105). If the AE#412 re-cut took longer than the seek's 2 s wait, its mark was dropped and its backoff went into the published axis, up to one GOP off until the AE#418 reconcile. The mark now belongs to the restart that performs it. +- **A seek burst no longer leaves a stale timeline entry or a truncated segment behind** (audit SEG-104). A producer replaced while a read was in flight could still record its gate-open epoch, or adopt its half-written segment when the aborted read came back as end of file, so a later backward seek hit a short segment. Replaced producers are now ignored, and a stopped pump drops the read it had in flight. +- **A sequential archive with bridged audio plays to its real end** (audit SEG-103). When the last GOP crossed the last plan boundary, the audio tail flushed at end of file opened a segment the video never reached, and ENDLIST landed one segment early. The flush now follows the video cutter like the rest of the audio. +- **A damaged keyframe at a seek or restart point is repaired like the same frame played through** (audit BIT-104). The framing verdict that lets the AE#561 sanitizer cut an overlong NAL length reset with every muxer, so the sample a restart landed on reached movenc uncut and failed with `-19602` on every retry. The verdict is now kept for the whole session. +- **Manual DMG runs build the tag they upload to.** `release-dmg.yml` dispatched by hand checked out the dispatching branch and then signed, notarized and `--clobber`-uploaded it under the tag's name; the checkout is now pinned to the tag and the log names the built commit (audit OPS-101). +- **`aetherctl dovitest` reads Annex-B sources and fails loudly.** The probe resolves the NAL framing once and uses it for the converter and the writer, so a disc remux is no longer reported as converted while writing one bogus NAL per packet. A full disk or a read error now ends the run with an error instead of an abort or a silent EOF, and the output is opened without following symlinks (audit BIT-106). +- **`aetherctl` no longer leaves a new debug directory per run.** Debug artefacts go to one private per-user directory that the next run overwrites, and `dovitest` takes `--out` (audit OPS-107). +- **`aetherctl seektest` and `smbtest` can fail.** `seektest` reads its counters under the log lock and exits 1 when a probe prints FAIL or a wedge is left unrecovered. `smbtest`'s random-seek check compares against the bytes the sequential pass saw and no longer passes when reads fail (audit OPS-104, OPS-105). +- **The demo DMG drops two hardened-runtime exceptions.** `disable-library-validation` and `allow-unsigned-executable-memory` are gone from the notarized demo, effective at the next release build (audit OPS-112). +- **Audio-only files and radio streams whose timestamps start late play instead of staying silent** (audit DEC-102). The audio-only host anchored its clock at the load position and ignored the first sample, so a source starting an hour in (an Ogg Icecast stream, a mid-broadcast audio TS) stamped every buffer far ahead of a clock at 0. It now anchors at the first sample, publishes position relative to it, and seeks on the source axis. +- **Dolby Vision Profile 5 scrub thumbnails lose the green and magenta cast again** (audit BIT-102). The cache-backed still path took VideoToolbox for every title, and its P010 output skips the Dolby Vision converter. No-base-layer Dolby Vision (P5, AV1 P10.0) stays on software for stills. +- **A second undecodable audio track on a live channel no longer re-hangs the session silently** (audit FEA-103). The AE#641 mark held one stream; it is a set now, and a repeat report is logged. +- **A stop during a failing recording teardown reports the failure, and a start that a stop or zap overtook is cancelled** (audit FEA-105, FEA-106). `stopRecording()` used to publish `.ended` while the file was still draining and lose the `.writeTooSlow` failure. `startRecording(to:)` now throws `CancellationError` when a stop or a new load lands during its wait for the previous file. +- **Full-range and 10-bit SDR software-decoded video show correctly** (audit DEC-107). The software decoder now reads `color_range` and the frame's bit depth, so full-range AV1 / VP9 / HEVC is no longer crushed and 10-bit SDR AV1 / VP9 is no longer dithered to 8 bit. +- **A seek, a DVR rewind or a renderer self-flush no longer leaves audio stuck** (audit DEC-106). `AudioOutput.flush()` retires an epoch, and the FFmpeg hosts, including the DVR ring path, enqueue only if it is unchanged. The AE#549 auto-flush runs off the posting thread, so a renderer that posts from inside `enqueue` cannot hang the feed. +- **An injected subtitle rendition is matched by its playlist identity** (audit NAT-104). An origin rendition whose localized display name equalled the sidecar's NAME could be selected instead. +- **The Atmos confirmation pass closes the reader fork it only tests for nil** (audit DEC-109). +- **A server that ignores HTTP Range now plays to the end instead of stopping at 32 MB, including when the first request was refused or slow** (audit DMX-101). The response head is judged on the cold open and again on the connection that follows the size probe, so a 429/503 or slow first answer no longer hides the origin. A range-ignoring 200 is confirmed once with a one byte range, then plays forward-only (no seeking), still reporting its Content-Length so MPEG-TS duration estimation works. Origins that honour Range, and small files answered with a 200, are unchanged. A CDN that only fills its cache while a client downloads it can read as range-ignoring and then plays forward-only. +- **A remote disc image on a server that ignores Range now fails at once with a named cause** (audit NET-102). The disc reader judged range answers only after downloading the whole body. It now decides at the response head and hangs up, and the load throws `originIgnoresRange`. A 206 wider than asked is cut where the range ends. +- **A forward-only source whose transport ignores the pause now ends with a named cause** (audit DMX-107). Reaching the 128 MB hard cap reports `AVIOReaderError.originIgnoresFlowControl` through `lastReadFailure` and the log, instead of a bare EIO that looked like a dropped connection. +- **A detour block fetch no longer takes two origin slots** (audit DMX-105). On an origin limited to two requests with the pump holding one, each detour block waited 4 s for its own second slot. +- **A malformed chunked response on the held connection ends the connection instead of growing without bound** (audit DMX-111). A size or trailer line over 4 KiB, or 64 KiB of framing with no body byte, now throws. Opt-in `heldSourceConnection` path. +- **A forward-only source keeps its opening GOP** (audit HLS-103). The NAL framing probe and the in-band hvcC scan used to seek to 0 and consume packets; libavformat flushes its queue before it tries a seek, so a range-less archive lost its first GOP. The demuxer now holds the packets those probes look at and hands them to the producer, and refuses repositions on a source that cannot rewind. +- **A remote disc image session is released when the demuxer closes** (audit NET-110, DMX-110). The disc reader the demuxer built for a remote ISO stayed open on close, on a failed open and on an unparseable structure. +- **Closing a demuxer cancels an open that is still connecting, a read on a local path, and a pump waiting for its origin slot** (audit DMX-102, DMX-109, DMX-112). `markClosed()` now reaches those stages instead of waiting out the origin's slot or pacer timeout. +- **A stop no longer looks like the end of the source** (audit SEG-104). A read aborted by the demuxer close now reports `AVERROR_EXIT` and the hosts treat it as the stop arriving; tail adoption, bridge flush and `onSequentialSourceEnded` no longer fire from a teardown. +- **Track and stream accessors no longer read libavformat's live streams array** (audit DMX-108). They answer from a snapshot, and `close()` waits for callers inside `withStream(at:)`, closing a use-after-free window during a live reopen. +- **Unfinished PGS display sets count toward the subtitle store's memory cap** (audit SUB-111). A stream that never sends an END segment can no longer escape the aggregate cap; one set is capped at 4 MiB instead of 16. +- **A full volume no longer freezes a software live channel** (audit VPERF-101). The ring used to evict only after a successful write; it now frees its oldest chunk on a failed write (full disk or quota) and retries. +- **The segment cache's stale sweep no longer deletes a live DVR ring an hour into a session** (audit SEG-105). The ring holds the same flock marker as the cache. +- **One stalled relay consumer no longer blocks every other relayed fetch** (audit NET-107). A fetch that falls more than 32 MiB behind is cancelled and re-requested. + +### Performance + +- **`playlistShiftSeconds` is published only when it changes**, so the engine's `objectWillChange` no longer fires at 10 Hz (audit PERF-106). +- **Forward-only playback no longer copies the whole read-ahead buffer on every read** (audit PERF-102, DMX-104). The streaming buffer is chunked, so a read copies only what it returns. 200 reads against a 60 MB buffer went from 329 ms to 5.7 ms on a debug build. +- **Idle scrub detour blocks are released after 30 s** (audit PERF-108). Previously they stayed resident until close, up to 32 MB per reader. +- **The subtitle packet store appends in place** (audit SUB-103, PERF-107). 40k appends took 10.9 s because every append copied each retained packet. +- **HLS program dates parse without building two formatters per line** (audit NET-106). +- **Playlist bodies accumulate in chunks** (audit NET-113). 32 MiB took 5 s in a debug build and takes 0.14 s now. +- **The software live DVR ring no longer costs a file and a heap object per packet** (audit PERF-101, SEG-106). It spools into ~4 MB chunk files behind a flat 24-byte index (266 MB of index heap at 100k packets is now 4 MB, appends 314 us to 28-45 us) and is bounded by bytes as well as time (min(2 GiB, a quarter of free space), at least 64 MiB). The software live seekable range follows what the ring still holds. +- **Software video holds one frame for reordering instead of four** (audit PERF-104). Only the VideoToolbox HEVC decoder emits out of presentation order, so libavcodec and dav1d output skips three dead frames (75 MB of IOSurface at 4K P010). A late frame is dropped, not shown out of order, and raises the depth to 4 for that renderer. +- **The software demux loop no longer wakes 200 times a second while parked** (audit PERF-110). It waits on its condition for 5 to 20 ms. +- **The software VOD packet spool writes a record with one `writev` and reads it without intermediate copies** (audit PERF-109). A 208 KB packet round trip went from about 800 us to about 380 us on the dev Mac. +- **Dolby Vision profile 7 to 8.1 rebuilds a packet with one copy instead of two** (audit PERF-111). + +## [7.22.2] - 2026-09-29 + +### Fixed + +- **An unproven HDR master is served after the display mode switch, not during it** (#667). The + pre-flight releases the load at its 2 s cap while an HDR switch is still running, which is what keeps + the load overlapping the switch (#348). On a panel whose EDR headroom never proves HDR (#459) the + engine then offers the master and lets AVPlayer's acceptance answer, and mid-switch AVPlayer answers + for the mode the panel is leaving: measured on tvOS 27, `-11868` 80 ms after the serve and 660 ms + before the switch ended. The refusal latched (#588), so every HDR title went media-direct and lost its + subtitle and audio renditions. That route now waits for a switch it saw start to end, re-reads the + panel, and routes then; the play gate held the first frame until that end anyway. A refusal raised + while a switch is in progress still earns the item its media fallback but no longer sets the latch. + +## [7.22.1] - 2026-09-29 + +### Fixed + +- **`LiveTelemetry`'s two bitrate fields measure the media played, not the bytes transferred** (#514). + Both were metered from the reader's transfer counter, which parts from playback on every route that + reads ahead: a 20 Mbps VOD stream read about 35 Mbps (prefetch and seek re-fetches counted as they + arrived), and a paused live session kept draining its origin into the DVR window while the divisor + stood still, so its average climbed for as long as the pause ran. The pumps now record the played + video and audio packets by presentation time, and the sampler charges what the playhead crossed: + `instantBitrateMbps` over about the last 10 s of playback, `averageBitrateMbps` over the session. + Both stand still through a pause on every route, live included, and a seek charges nothing for the + span it jumps. The transfer remains `networkThroughputMbps`. The remote-HLS bypass is unchanged (it + reports the variant's declared rates). + +## [7.22.0] - 2026-09-28 + +### Changed + +- **The remote-HLS bypass publishes what a stats panel reads.** On `nativeRemoteHLS` (route + `.remoteBypass`) the engine published no `liveTelemetry`, an empty `audioTracks` and no delivered + video format, so a host fell back to its own metadata for the original file: a 1280x720 H.264 Jellyfin + transcode showed as "3840x2160, Main 10, 17 Mbps". Everything is now read from AVPlayer's own item, with + no second connection to the origin (#664): + - `diagnostics.liveTelemetry` runs on the bypass. The two bitrate fields are what the playing variant + declares (BANDWIDTH, and AVERAGE-BANDWIDTH or BANDWIDTH where the master omits it), because what + AVPlayer transferred is buffer fill at link speed. Network throughput, transferred bytes, dropped + frames and forward buffer come from the access log and the loaded ranges. The loopback counters read 0. + - `audioTracks` carries one `TrackInfo` per audio track AVPlayer built (codec, channels, sample rate, + profile, Atmos, language), ids from 400000. `selectAudioTrack` is informational on this route and + logs instead of acting. + - `sourceVideoWidth`, `sourceVideoHeight` and `sourceVideoStreamFormat` describe the stream AVPlayer + plays. Under a server-side transcode that is the transcode, not the library's file. + +## [7.21.1] - 2026-09-27 + +### Fixed + +- **A software session keeps the volume the host app set.** The engine applies its stored volume to a + new host before `load()`, and `SoftwarePlaybackHost` and `AudioPlaybackHost` forwarded that only to + the `AudioOutput` that `load()` builds later, so the write was dropped and every software session + started at full volume, with `volume` reading 1.0 until then. Both hosts now hold the volume and + hand it to each output they build (#660). + +## [7.21.0] - 2026-09-27 + +### Added + +- **The stream format a stats panel needs, from the engine (#658).** `sourceVideoStreamFormat` (and + `SourceProbe.videoStreamFormat`) is a `VideoStreamFormat`: pixel format, bit depth, colour primaries, + transfer, matrix, range and profile in libav's names, with viewer labels alongside ("BT.2020", + "PQ (SMPTE ST 2084)", "Limited"). A field the stream leaves unspecified stays nil rather than reading + as BT.709. `decodedVideoFormat` is what the engine's own decoder produced and the CoreVideo buffer it + was displayed from ("P010 (x420)"), republished on change; it is nil on the native routes, where + AVPlayer decodes and no frame passes through the engine. `TrackInfo` gains `sampleRate`, + `bitsPerSample`, `sampleFormat`, `channelLayout` and `profile`, the last being where DTS:X and + TrueHD Atmos show up. `aetherctl probe` and `aetherctl play` print all of it. + +## [7.20.1] - 2026-09-27 + +### Fixed + +- **An in-place load on a reused native host no longer publishes a pause at the seam.** The host + stopped observing `timeControlStatus` before pausing the outgoing item, so it kept publishing + `.playing`; the next load replayed that on subscribe, treated the transport as already rolled, and + let its own pre-roll `.paused` through as a real pause (`loading, playing, paused, playing` within a + millisecond). A host raising its transport on an external pause showed it over every + auto-advanced episode. `aetherctl play --host-calls reloadnext` reproduces the seam (#661). + +## [7.20.0] - 2026-09-27 + +### Added + +- **`airPlayPictureStaysLocal`.** iOS publishes true while a wireless AirPlay receiver holds the audio + route and the session runs on the software host, whose picture stays on the device while its sound + goes to the TV. Nothing fails there, so a host had no way to tell the viewer why the receiver showed + no picture. + +### Fixed + +- **A URL source that can only be read front to back plays on the native path, and AirPlays with a + picture.** An origin that ignores `Range` and names no length (a remote MKV on filesamples.com) + was forced onto the software host, which cannot seek on it either and never hands a receiver its + picture: AirPlay played the sound only. When the container states a duration the engine now serves + such a source as a sequential origin, and an AirPlay hop swaps the item onto the LAN address rather + than reopening a source that could only restart from byte 0. +- **A forward-only source no longer loses its opening.** The cursor reset after the segment plan + seeked a source that cannot rewind, which drops the packets the probe had buffered and leaves the + Matroska demuxer resyncing wherever the stream had got to: the first GOP of a 30 s clip, 30 s into + a remote MKV on the software path. +- **A sequential origin whose GOP is longer than the segment stride lists all of its media.** Audio + opened segments by time while the playlist is built from the video keyframe cuts, so the video + after an audio-opened boundary landed in a file the playlist never listed (4 to 11 s of an 11 s + first GOP), and AVPlayer stalled at the end of seg0. Audio now follows the video cut there, as on + live. The finalize reports are also anchored on the pump's first segment, so a skipped index known + before seg0 is captured no longer holds back every later one. +- **A backward jump on a sequential origin is served from the cache.** The residency scan read the + holes its cutter leaves as a gap and asked for a restart the origin cannot give, which published + "Source cannot be repositioned" over a session that held every segment it needed (an AirPlay hop + back to the device, 22 s in). + +- **A `FrameExtractor` still carries the colour space playback shows the picture in.** SDR stills + were tagged sRGB while their pixels are in the source's own primaries and video transfer. They now + carry the space CoreVideo builds from the tags the displayed buffer carries + (`kCGColorSpaceCoreMedia709` for BT.709). Against VideoToolbox's own conversion of the same frame, + max channel error 9 -> 2 on BT.709, 52 -> 2 on NTSC SMPTE-C, 57 -> 2 on SDR BT.2020. + +## [7.19.0] - 2026-09-26 + +### Fixed + +- **A software-decoded picture carries the colour tags VideoToolbox would give it (AE#654).** The + software decoder attached a tag only where the frame declared one and CoreVideo had a mapping, so + an untagged source reached the display layer with no primaries, transfer or matrix, while the same + file through VideoToolbox arrives tagged BT.709. It now fills the gaps the way VideoToolbox does, + measured on its output: nothing declared means BT.709 in all three at any size and codec, and a + lone BT.601 matrix gets SMPTE-C primaries. Affects the software route (MPEG-4 ASP, MPEG-2, VC-1, + VP9, AV1 without hardware decode). `ColorDescription` keeps its gaps, so the HDR gate and the + tone mapper still see an untagged stream as untagged. +- **Tagged SD sources keep their tags on the software path.** `ColorAttachments` had no mapping for + BT.601 matrices, SMPTE-C or EBU 3213 primaries, so a correctly tagged PAL or NTSC source lost all + three. Added with DCI-P3, SMPTE 240M, sRGB and linear transfer. +- **A `FrameExtractor` still is converted with the picture's own matrix and range.** The SDR still + path never called `sws_setColorspaceDetails`, so every still went through swscale's BT.601 + default and a range read from the pixel format alone: HD colour bars were up to 33 levels off, and + a 10-bit full-range picture (no `yuvj` variant exists) was read as limited. The matrix now follows + the rule the displayed buffer is tagged by, so an untagged still resolves exactly as playback. + +### Added + +- **`SoftwareDecodeProbeResult.firstFrameColor`**: the colour tags the first picture reaches the + display layer with, as `primaries / transfer / matrix`. Printed by `aetherctl swdecode`. + +## [7.18.2] - 2026-09-26 + +### Fixed + +- **A software-path escalation reports the position its rebuild resumes at (AE#629).** Under a + mount raised by a host `load()`, AVPlayer's clock reads the start of the segment it decodes up + from until the mount seek lands, and a refusal in that window put that reading on the `#561` line + and on `SoftwarePathEscalationEvent.positionSeconds`, while the rebuild itself resumed at the + position the load was handed: 12.00 s against 15.90 s on the reporter's Apple TV. Both now carry + the rebuild's position. + +## [7.18.1] - 2026-09-26 + +### Fixed + +- **A VOD consumer that goes quiet while it keeps playing no longer trips the #65 wedge breaker.** + AVPlayer on a cellular iPhone fetches in bursts, several segments at once and then nothing for + half a minute to almost two while it plays out a forward buffer of up to 113 s. The slow path + counted that silence alone, broke the park every few minutes and nudge-seeked a healthy session, + which flushed the buffer on screen: a half-second freeze of picture and sound each time, and a + re-anchor that fetched the source it already held a second time. The slow path now counts only + seconds in which the consumer neither fetched nor rendered; one that really stopped fetching + still breaks on the fast path once its buffer runs dry. The `PARK` line carries the new count as + `idle=` and names a quiet but playing consumer as such (AE#649, thanks @tschuegy). + +## [7.18.0] - 2026-09-26 + +### Changed + +- **A remote disc image adopts a warm (#647).** `AetherEngine.prewarm` on an `.iso` / `.img` / + `.udf` URL was fetched and then never used: the disc reader kept its own buffer, probed the size + with `bytes=0-0` and fetched the head again, and the warm stayed in the store. The disc reader now + takes the size from the warm, serves the disc structure and the title's opening extents out of it, + and starts its first request at the warm frontier; its forks share the same bytes without a copy + and no longer re-probe the size. A disc-image URL that turns out not to be a disc hands the warm + on to the streaming reader. Measured on an authored DVD over a 20 Mbit/s, 100 ms origin: 10 + requests after the warm instead of 19, first frame 5.7 s sooner. + +### Fixed + +- **A DVD title opens without reading 50 MB first, and lists every subtitle it has (#651).** + MPEG-PS never lets `find_stream_info` finish early, so every DVD open read the whole playback + probe budget (50 MB / 60 s) and still missed any subtitle stream whose first packet came later. + The subpicture streams the title's VTS IFO declares are now created before the probe, their + fragments are joined in the engine the way libavformat's `dvdsub` parser joins them for streams + it creates itself, and a title with a readable IFO probes 8 MB / 5 s. Measured on an authored + DVD over a 20 Mbit/s, 100 ms origin: first frame at 6.5 s instead of 29.6 s, 0.5 s after a + prewarm. A subtitle whose first packet sits at 100 s is a track from the first frame and + decodes. Discs without a readable VTS IFO, and plain `.mpg` / `.vob` URLs, keep the old budget. + +## [7.17.1] - 2026-09-26 + +### Fixed + +- **A session revived by the #93 item swap keeps its playhead until the fresh item lands there.** + Until then AVPlayer reports the start of the segment it decodes up from, so the playhead stepped + back by up to a segment, and a software-path rebuild raised in that window resumed there and + replayed the gap (3.97 s on a 15.97 s revive) (AE#629). +- **A software-path rebuild that fails after its teardown surfaces one `.error`, its own.** The + rung then published the failure it had absorbed on top, a second `.error` that contradicted what + a `load()` following the rebuild threw. The absorbed failure stays on `softwarePathEscalations` + (AE#629). +- **A live H.264 join opens on an intra recovery point, not only on an IDR.** Feeds whose encoder + never sends an IDR (entry points are I-pictures behind a recovery point SEI with + `recovery_frame_cnt` 0) never started on the native route; the join gate refused every entry + point and gave up after three 15 s reopen cycles (AE#627). +- **A live join that finds no entry point at all goes to the software path after one wait**, instead + of three reopen cycles into the same bitstream, or to `liveSourceReset` when the host declined + that rung (AE#627). +- **A live channel whose audio the bridge cannot decode plays video-only instead of stalling.** A + FLAC bridge builds its sample entry from the encoder's extradata, so a bridge that decoded nothing + left segments with an audio track that never carried a sample, and AVPlayer showed the first + picture and waited on it for the rest of the session. The engine now rebuilds the session without + that track and `audioDelivery` reads `.droppedNoPipeline` (AE#641). +- **A VOD audio track the bridge cannot decode surfaces `.audioBridgeProducedNoOutput` on the FLAC + route too.** Only the E-AC-3 route reached that verdict, through its failed first cut; a FLAC + bridge builds its sample entry from the encoder's extradata, so the session played silently while + `audioDelivery` read `.bridged`, and a host ladder never heard of it. The verdict reaches the host + once, whichever arm finds it first (AE#641). +- **`audioDelivery` reads `.droppedNoPipeline` for a source whose audio stream could not be picked**, + not `.noAudioInSource`. `av_find_best_stream` passes over a stream whose parameters the probe left + empty and only live fell back to it, so on VOD (and on the software path) a source with an + undecodable track read as one without audio (AE#641). +- **FFmpegBuild 3.6.0: an MPEG-TS audio PID is identified by its payload.** The raw `dts`, `truehd` + and `loas` demuxers were missing, so the mpegts content probe could never confirm DTS, TrueHD or + LATM and the lenient mp3 probe named the track; a PID labelled 0x03 (MPEG-1 audio) was not probed + at all. A DTS-HD IPTV channel opened as mp3, and every packet failed with "Header missing" + (AE#641). +- **The keyframe wait log counts the keyframes it dropped** (`keyframes=N`), so a gate refusing + every entry point no longer reads as a feed without keyframes (AE#627). + +## [7.17.0] - 2026-09-25 + +### Added + +- **`LoadOptions.escalatesToSoftwarePath` (default `true`) lets a host decline the AE#561 rebuild** + onto the software path and get the native failure as `.error`, for hosts that re-plan a failing + title with a ladder of their own (AE#629). +- **`softwarePathEscalations` publishes that rebuild when it is taken**, with the failure it + absorbed, where a host used to see only `videoRoute` change (AE#629). + +### Security + +- **The origin relay no longer writes the media server's access token into its local URLs.** They + name the origin by a per-session sealed reference, so the token no longer reaches the request + log, `load url=` lines, rewritten playlists or an AirPlay receiver (audit NET-1). +- **Log redaction also catches credentials that are percent-encoded, or encoded twice, inside + another URL.** `api%5Fkey%3D...` is now stripped the same as `api_key=...` (audit NET-1). +- **Credential headers stay with the host's own origin when a playlist names another host or drops + to http.** Other headers still go everywhere, the same rule redirects already follow (audit NET-7). +- **A LAN peer can no longer hold the loopback server's connection slots.** Connections without the + session token must send a request head within 10 s, and LAN peers are held to 24 of the 32 slots + (audit NET-6). +- **A crafted Matroska SeekHead no longer crashes the app during prewarm.** A SeekPosition near the + top of the 64-bit range is skipped instead of overflowing (audit DMX-2). +- **The held source connection no longer sends media-server credentials to a redirect target on + another origin**, and writes the request path as the URL spells it, so an encoded line break can + no longer inject header lines and IPv6 hosts get a valid Host header (audit DMX-3, DMX-4). +- **A sidecar subtitle name can no longer add lines to the served HLS master.** Line breaks and other + control characters in a name or language are neutralised (audit NAT-6). +- **A crafted or broken source timestamp no longer crashes the app.** Demuxed pts/dts beyond + plus or minus 2^60 are treated as unset, and the segment pump's timestamp arithmetic saturates + instead of trapping (audit SEG-1). +- **The Dolby Vision profile check can no longer read a whole file while playback is starting.** It + stops after a fixed number of packets from any stream, or 64 MiB (audit BIT-2). +- **Hostile playlists and disc images no longer crash the app.** An `EXT-X-MEDIA-SEQUENCE` near + `Int.max`, a crafted UDF partition map or descriptor sequence, a 4 GB ISO9660 directory length and a + `#EXTINF:inf` behind an injected subtitle sidecar all trapped; each is now rejected or bounded at + parse, and the sidecar playlist fetch is size-capped (audit NET-3, NET-4, NET-5, NET-11, NAT-1, + NAT-5). +- **SECURITY.md names the current supported release line** instead of 2.1.x, CI pins third-party + GitHub Actions to commit SHAs with Dependabot proposing updates, and aetherctl writes its debug + files into a private per-run temporary directory instead of fixed names in `/tmp` (audit OPS-1, + OPS-2, OPS-3). +- **The HDR10+ and Atmos probes no longer read a crafted file to its end.** Blocks of the streams + they ignore are now counted against a byte budget below the demuxer, so one target packet followed + by gigabytes of another stream stops the probe instead of downloading it. + +### Fixed + +- **A `load()` still waiting when the engine rebuilds its session on the software path now returns + with the rebuilt session** instead of throwing the `CancellationError` a host's own supersession + throws, which a host read as a failed load and answered by stopping the session that had just come + back (AE#629). +- **Stopping playback during a restart after a stalled read aborts the replacement connection**, + instead of letting it finish its connect and stream probe after the session was gone. +- **A non-finite segment or cadence duration can no longer crash the live playlist writer.** +- **Live AV1 High and Professional streams play on hardware-AV1 devices**, through the software host + like VOD. +- **A seek on the software path no longer lets one pre-seek frame wedge the picture.** A seek that + landed while the decoder was retrying a full queue, or while the hardware decoder was preparing a + packet, still let that packet through after the flush (audit DEC-1, DEC-4, AE#492). +- **Seeking in audio-only playback no longer leaves stale audio queued ahead of the new position** + (audit DEC-2). +- **Subtitles in Picture in Picture no longer squash anamorphic video or turn HDR to SDR while a line + is showing.** Composited frames keep the source's pixel aspect ratio, colour tags and colour space, + and the compositor's buffer pool is no longer raced at stop (audit DEC-3, DEC-5). +- **A bridged audio track no longer goes silent for the rest of the session after one frame the + resampler rejects.** The frame is dropped and logged, and the previous resampler keeps running + (audit DEC-6). +- **Stopping a live recording no longer blocks the main thread while the backlog is written.** + `recordingState` turns `.ended` once the file is closed, a moment after the stop (audit REC-1). +- **CRLF-terminated HLS playlists parse.** In Swift `"\r\n"` is one Character, so splitting on `"\n"` + read the whole playlist as one line and rejected it on the live/VOD ingest, the audio tap and the + subtitle proxy (audit NET-2). +- **A remote disc image only accepts a 206 that starts at the requested offset**, instead of placing + wrong bytes or buffering a whole-file 200. The UDF descriptor scan also stops at its Terminating + Descriptor again (audit NET-9, NET-11). +- **Closing the player during a scrub restart no longer spins segment requests at full CPU** + (audit SEG-3). +- **A re-cut segment can no longer be dropped by a request that is reading it at the same moment**, + and a replaced pump no longer stores a cut-off segment (audit SEG-4, SEG-5). +- **Segments with a 256 to 511 byte first NAL are covered by the AE#561 repair**, instead of being + mistaken for Annex B and failing with -19602 (audit BIT-1). +- **Matroska H.264 timing repair no longer gives two frames the same time at a GOP boundary** + (audit BIT-3). +- **A slow source no longer ends a healthy session with "Source read failed" after a few scrubs.** + When the engine replaced a producer stuck in a read, the old producer's aborted read was treated as + the session failing: it spent the read-error revive budget, forced an extra reconnect and re-seeked + to a stale position. Exits from replaced producers and reads aborted by a stop are now ignored + (audit HLS-1). +- **Changing channel during a live reconnect no longer opens the old channel again**, which on + single-connection tuners took the slot the new channel needed. A stop that lands during an open + now cancels it (audit HLS-2). +- **8-bit HEVC in MPEG-TS is no longer advertised as Main10.** The CODECS string for Annex-B HEVC is + read from the stream's SPS, so the master playlist matches the init segment (audit HLS-3). +- **Seeks that need a re-cut stay within their time limit** on slow sources (audit HLS-4). +- **AV1 4:4:4, 4:2:2 and 12-bit play on devices with hardware AV1.** Apple's decoder handles AV1 Main + profile only; other profiles now take the dav1d software path for VOD (audit HLS-5). +- **Scrub previews on large segments use far less memory.** The segment file is memory-mapped instead + of copied twice per preview, and nothing is read when a cached preview decoder already covers the + segment (audit SEG-2). +- **Leaving a session while its software rebuild is running no longer leaves an error behind.** A + `stop()` or new `load()` during the AE#561 rebuild used to put the engine into `.error`, including + onto the next title's startup. The rebuild also runs in its own task now, so its waits no longer + spin inside the cancelled item-death task (audit CORE-1). +- **A title started just before pressing the TV button no longer plays on in the background.** A + load or scrub still in flight when the app backgrounds gets the background teardown once it + settles, instead of carrying a live pipeline through the tvOS suspension (audit CORE-2, AE#597). +- **Track selections survive a second background teardown on iOS** (audit CORE-3). +- **Music plays again after a media services reset.** The reset rebuilds the audio-only AVPlayer too, + not only the video player (audit CORE-4, AE#597). +- **A recording that fails while zapping no longer ends the next channel's recording**, and the live + edge snap no longer writes the previous channel's position after a zap (audit CORE-5, CORE-7). +- **Load-time sidecar subtitles on the remote-HLS path select the right track** when names repeat, + match an origin rendition, or contain a quote (audit NAT-2). +- **A superseded load or live-join probe no longer acts on the session that replaced it** (audit + CORE-6, NAT-4). +- **An origin that ignores Range no longer streams the whole file into memory.** Bytes past the + requested range are dropped and the connection is ended. Such an origin could never seek, and a + file larger than memory got the app killed; it now ends in a read error instead, and + `LoadOptions.sequentialOrigin` stays the way to play one (audit DMX-1). +- **A 206 that starts somewhere other than the requested offset is refused** instead of shifting every + later read (audit DMX-5). +- **A length-less stream that drops or stalls reports an error** instead of ending playback as if the + title had finished, and its buffer has a hard 128 MB bound for a transport that ignores the suspend + (audit DMX-6, DMX-7). +- **A failed chunk fetch during a probe or still extraction reports an error** instead of a truncated + file (audit DMX-10). +- **Tail prefetch checks its span against the file size and tolerates over-delivered bytes**, instead + of giving up suffix ranges for that origin until restart (audit DMX-8, DMX-9). +- **Two demuxer races closed:** the provider reference during a cross-thread abort, and stream lookups + while an MPEG-TS read adds streams (audit DMX-11, NAT-7). +- **Playlist and held relay bodies are capped while they download, not after.** A runaway origin can + no longer grow one until the app is killed (audit NET-10). +- **The accept loop backs off when the process runs out of file descriptors**, instead of spinning a + core and flooding the log (audit NET-13). +- **The server's stop line names the port it released** (audit NET-12). + +### Performance + +- **Faster first segment after a seek on TrueHD sources, and less CPU for HDR10+ detection on sources + without HDR10+** (audit SEG-6, BIT-4). +- **The live ingest FIFO no longer re-copies its whole buffer on every read** (audit NET-8). +- **Stripping ASS override tags from a cue is linear.** A cue of 200k tags took 525 s and now takes + 0.15 s on the PiP and AirPlay subtitle track (audit SUB-1). +- **PGS and other bitmap subtitles decode off the main thread.** The overlay drain and the PiP/AirPlay + OCR worker used to expand every display set on the MainActor, including the whole window a + selection or seek backfills at once. Each bitmap is also about 4x cheaper to convert (a full + 1080p set measured 5 ms before, 1.3 ms after, in a release build on an M1) (AE#628). + +## [7.16.1] - 2026-09-24 + +### Fixed + +- **The media fallback comes back where the rejected item was placed, not where the session + started (#98).** The fallback replayed the start position of the session's first mount. The + #93/#65 stage-2 recovery swaps a fresh item in at the position playback held, so when THAT item + was refused at startup (-11868), the session rewound to wherever it had first been loaded. Field + log, Apple TV 4K 3rd gen, tvOS 27.0, HDR10+ HEVC Matroska opened with a resume at 1844 s: + paused at 2099.69 s, the item died behind the screensaver, the recovery item was refused, and + playback resumed at 1834.79 s, the keyframe before the session's first mount and four minutes + behind the pause. A title started from its beginning resumes at its first frame. `NativeAVPlayerHost` + now records where every mount places its item, in-place swaps included, and the fallback reads + that. +- **A recovery reload leaves a paused viewer paused (#93, #98).** Item death parks AVPlayer at + `.paused` whatever the viewer wanted, so the #93/#65 stage-2 reload runs for a paused consumer + too, and it and the #98 media fallback then called `play()` on the fresh item unconditionally. + Field log, Apple TV 4K 3rd gen, tvOS 27.0, HDR10+ HEVC Matroska: paused, the tvOS screensaver + took the display two minutes later, the item died with -11868, and the recovery started the + title and dismissed the screensaver. Both now resume only when the host's durable intent (#122), + which the in-place swap keeps, says the viewer was playing; a playing viewer is resumed as before. + `aetherctl play --host-calls pausereload,playreload,extplayreload` drills it headless: before, + a paused session came back from a forced stage-2 reload at `+7.20 s, state=playing`; after, it + holds at `+0.00 s, state=paused`, and a session resumed past the engine through AVKit still + comes back playing. + +## [7.16.0] - 2026-09-24 + +### Added +- `clock.sourceTimeFollowsPicture` (and the `sourceTimeFollowsPicture` mirror): false on `nativeRemoteHLS` from a time jump until an injected rendition line has re-measured the lead, so a host can hold its overlay across that window instead of detecting seeks itself (AE#616 follow-up). +- `EngineLog.registerSecret(_:)` / `unregisterSecret(_:)`: a host names a value that must never be logged, and every line has it replaced, raw or percent-encoded, before it reaches os_log or the handler. + +### Security +- **Log redaction covers credentials carried as plain path segments.** IPTV panels speaking the Xtream Codes API put the account password in the path of every stream URL (`/live/{user}/{password}/{id}.ts`, likewise `/movie/`, `/series/`, `/timeshift/`, and the `/hls/` and `/hlsr/` redirect targets), where no named parameter, userinfo or encoded payload points at it, so `load url=` lines carried it in clear text. The layout is now matched, the user name stays readable, and ordinary paths such as `/live/master.m3u8` are untouched. The short form without a prefix has no layout to match, which is what `registerSecret(_:)` is for. + +## [7.15.2] - 2026-09-24 + +### Fixed + +- **`sourceTime` on the `nativeRemoteHLS` bypass follows the presented frame (AE#616).** It was + AVPlayer's item time. An origin whose playlist places a segment at its slot while the segment + starts at the keyframe before it (a Jellyfin transcode restarted with `-noaccurate_seek -copyts`) + makes item time lead the picture, by 1.1 to 8.3 s in the report. A legible output on the bypass + item now matches presented lines of the injected #316 WebVTT renditions back to the cues the + engine wrote, and `sourceTime` is item time less that measured lead. `currentTime` and + `seek(to:)` stay on item time. With no injected rendition selected nothing is measured and + `sourceTime` stays item time, which the API docs now say. + +### Performance + +- **The persistent HTTP reader no longer copies its read window on every trim (AE#619).** + `AVIOReader` dropped the consumed head of its window with `subdata(in:)`, copying up to ~18 MB + once per 4 MB read, and grew the fresh buffer again on every append. The window now keeps the + chunks as delivered and a trim only advances an offset. CPU is unchanged within noise. Peak + memory footprint on a 91.6 Mbit/s 4K HEVC session over HTTP fell from 828 to 911 MB to 551 to + 553 MB on the native route and from 604 to 651 MB to 193 to 197 MB on the software route + (M1, `/usr/bin/time -l`, three runs per arm). Local files are read by `FileIOReader` and are + not affected. +- **The software VOD packet spool no longer serializes through a property list (AE#592).** The + binary plist encoder uniqued every object through a `Set`, hashing the whole payload of every + packet written to the read-ahead spool. The envelope is now a fixed little-endian header and the + raw bytes. Encode fell from 375 to 43 us per 425 KB packet, and process CPU on the same session + from 0.220 to 0.223 to 0.205 to 0.208 cores. The spool is per session, so there is no format + compatibility to keep. + +## [7.15.1] - 2026-09-23 + +### Fixed + +- **The software VOD buffer frontier on HEVC, and after a long session (AE#613).** HEVC kept the + strict `pts + duration` packet coverage, so a Matroska file muxed with 41 ms durations against + 41/42 ms deltas split it at every 42 ms step: `bufferedPosition` read about half a second ahead + of the playhead over a twenty-second reservoir, and cached seeks saw the same short frontier. + HEVC now takes the successor-timestamp model H.264 already used, under FFmpeg's own reorder + bound. Separately, a coverage that reached its 4096-range cap stopped describing new packets + (or, on the successor model, invalidated itself), so every frontier after the cap was nil; it + now forgets the ranges behind the playhead instead. +- **A master refused while the display is ineligible for HDR no longer latches (AE#535).** An + audio route death right after a display mode switch opens a window in which the criteria readout + reads `hdrEligible=no`. A session-preserving reload that landed in it got -11868 from AVPlayer + and latched `panelRefusedHDRMaster`, so every later HDR title in the session went media-direct. + The latch now also needs `AVPlayer.eligibleForHDRPlayback` to read true at the refusal; the item + still takes its media fallback either way, and a refusal that is not latched says so in the log. +- **An audio-delay rebuild raised the moment one returned keeps the playhead (AE#464).** + `rebuildPosition` honoured the parked position only while `state == .loading`, but `load()`'s + autostart writes `.playing` before the new host publishes a position, so for about 50 ms the + clock still read zero and a correction in that window rebuilt the session at its head. A + playable session whose clock reads exactly the reset zero now answers with the parked position; + an accepted seek retires it, so a genuine seek to 0 is never overridden. +- **`aetherctl` builds in Release again (AE#610).** The stallclock drill called two DEBUG-only test + hooks unguarded, so `swift build -c release` failed from 7.6.0 through 7.15.0. The library + products were never affected. A Release binary now reports that the drill needs a DEBUG build, + and CI builds `aetherctl` in Release. + +## [7.15.0] - 2026-09-23 + +### Added + +- **Scrub stills on the software VOD path, with no second connection (AE#605).** A VOD session the + device cannot hardware-decode (MPEG-4 Part 2, MPEG-2, interlaced H.264, everything on the iOS + Simulator) already spools its packets to a disk cache its seeks land in, but `scrubThumbnail` + had no arm for it, so every such session scrubbed blind on a source that refuses a second + request. It now decodes the still out of that cache, keyframe to target, through the same + extractor, queue and newest-wins ticket the live software path uses (#544), and the consumer + cursor is never moved, so playback reads on undisturbed. A target past what is retained answers + nil rather than the frame before it. `supportsCacheBackedStills` is true for such a session, and + now also for a software live session, which already served stills but reported false. + +## [7.14.0] - 2026-09-22 + +### Fixed + +- **A seek could leave the software VP9 decoder pointing at frames it had already released + (FFmpegBuild 3.5.0, FFmpeg n8.1.3).** `SoftwareVideoDecoder` opens VP9 with frame and slice + threading and flushes the codec context on every reposition, which is exactly the shape upstream's + `vp9_decode_flush()` mishandled: it released `frames[]`, `refs[]` and `ref_frames[]` but left + `next_refs[]` referenced, and a worker seeds its own references from another worker's `next_refs[]`, + so the pre-flush set survived and the first inter frame after a seek could decode against buffers + that were gone. A heap out-of-bounds read and write, found by Mozilla's bugmon automation. + +- **A Dolby Vision RPU with trailing zero padding keeps its dynamic metadata.** The RPU parser + stopped trimming zero padding in `ef167512ab`, which shipped in n8.1.2, so a heavily padded RPU + could fail the extension block size check and the file played without its dynamic metadata. Both + the HEVC and the AV1 decoder pull in the RPU parser, so this was on the path of every Dolby Vision + source, and a second fix bounds the RPU partition counts that were read without one. + +- **A VC-1 picture no longer loses its last macroblock.** The per-macroblock guards upstream added + in n8.1.2 bail out to error concealment when one bit is left, but a skipped P or B macroblock can + legitimately cost exactly one bit, and a Simple or Main profile picture is byte-aligned with no + stop bit, so a picture ending on such a macroblock lost it. + +- **The DTS core bitstream filter no longer passes on a profile it has just stripped.** It removes + the extension substreams, so a declared DTS-HD MA profile no longer described the bitstream that + came out of it. + +- **A channel layout read from the container survives a decoder that fails to open.** Probing copies + codec parameters back from the decoder context even after a failed `avcodec_open2()`, and the + failure zeroes the layout, so what the demuxer had read from the container was overwritten with + nothing. + +### Changed + +- **FFmpegBuild 3.5.0, built on FFmpeg n8.1.3.** 268 upstream commits over n8.1.2, of which the five + above touch code this engine ships. dav1d 1.5.4, zimg 3.0.6 and libzvbi 0.2.45 are unchanged (they + are the newest upstream releases), the component set and the six local FFmpeg patches are + unchanged, and the soname majors are the same, so nothing in the public API moves. ## [7.13.0] - 2026-09-22 diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index dacf79664..3c6f2761d 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -33,7 +33,7 @@ swift test --skip "$AUTHORIZATION_TEST_SUITES" swift test --skip-build --filter "$AUTHORIZATION_TEST_SUITES" ``` -For iterative work, open `Package.swift` in Xcode 26+ and pick the `AetherEngine` scheme. `FFmpegBuild` is a transitive dependency that supplies the bundled FFmpeg / dav1d binaries; you do not build it yourself. +For iterative work, open `Package.swift` in Xcode 27+ and pick the `AetherEngine` scheme. `FFmpegBuild` is a transitive dependency that supplies the bundled FFmpeg / dav1d binaries; you do not build it yourself. The `aetherctl` command-line target is macOS-only (it uses `Foundation.Process`) and is excluded from the iOS / tvOS library build. diff --git a/Examples/DemoPlayerMac/Scripts/build-dmg.sh b/Examples/DemoPlayerMac/Scripts/build-dmg.sh index 86bf85b47..96dd1fe23 100755 --- a/Examples/DemoPlayerMac/Scripts/build-dmg.sh +++ b/Examples/DemoPlayerMac/Scripts/build-dmg.sh @@ -183,22 +183,18 @@ cat > "$APP_DIR/Contents/Info.plist" < PLIST -# Hardened Runtime entitlements. Both flags are conservative defaults -# for a SwiftUI app that ships static FFmpeg libraries: the unsigned -# executable memory entitlement covers Swift / SwiftUI runtime JIT -# paths; the library-validation disable is a safety net in case -# FFmpegBuild ever introduces a dynamic load. Drop them once a clean -# notarization run confirms they're unnecessary. +# Hardened Runtime with no exceptions. The two this script used to grant +# (allow-unsigned-executable-memory, disable-library-validation) were +# conservative defaults waiting on a clean notarization run; the frameworks +# are embedded and re-signed below with the same Developer ID as the app, so +# library validation has nothing to exempt, and nothing here needs JIT memory +# (audit OPS-112). If a launch ever fails with a dyld or code-signing error, +# the error names the framework that needs its own fix, not a blanket key. cat > "$ENTITLEMENTS" < - - com.apple.security.cs.allow-unsigned-executable-memory - - com.apple.security.cs.disable-library-validation - - + ENT diff --git a/Examples/README.md b/Examples/README.md index d8bfc6dad..78c7c9545 100644 --- a/Examples/README.md +++ b/Examples/README.md @@ -18,7 +18,7 @@ Three samples covering different audiences: ``` https://github.com/superuser404notfound/AetherEngine ``` - Dependency Rule: Up to Next Major Version, starting from `7.13.0`. Add the `AetherEngine` library product to your app target. + Dependency Rule: Up to Next Major Version, starting from `7.28.3`. Add the `AetherEngine` library product to your app target. 3. **Drop the file in.** Replace the Xcode template's default `App.swift` (or whatever the generated `@main` file is called) with the contents of [`MinimalPlayerApp.swift`](MinimalPlayer/MinimalPlayerApp.swift). The file is self-contained: it defines both the `@main App` struct and the `ContentView`. diff --git a/Package.resolved b/Package.resolved index ba1ce5398..0b21bea8a 100644 --- a/Package.resolved +++ b/Package.resolved @@ -1,13 +1,13 @@ { - "originHash" : "72b666c842f7117ec649c4d1dd1b310cf6f2a819a0e59b4ad5cd97a3e913f6b1", + "originHash" : "aaad2851e63fbb5d8035f6ae9c6abb71e8e010beeeb30d2f9dbbe60ccd7022f6", "pins" : [ { "identity" : "ffmpegbuild", "kind" : "remoteSourceControl", "location" : "https://github.com/superuser404notfound/FFmpegBuild", "state" : { - "revision" : "9ee46ba4fb533e35efa6492eb1a903ca4f8058fc", - "version" : "3.4.0" + "revision" : "fda08325455bc12c112b5b82d014b98fa0fee0be", + "version" : "3.6.0" } }, { diff --git a/Package.swift b/Package.swift index 3f58e40d1..a80e5e326 100644 --- a/Package.swift +++ b/Package.swift @@ -38,7 +38,7 @@ let package = Package( // rebuilds still reach existing tags, which is where a pure rebuild // belongs; anything that adds slices or enables a component is a minor // and reaches consumers through an engine release. - .package(url: "https://github.com/superuser404notfound/FFmpegBuild", .upToNextMinor(from: "3.4.0")), // 3.4.0: FFmpeg learns the `dav1` sample entry in both directions, the AV1 variant of `av01` that signals Dolby Vision, so a Profile 10.0 source can be read at all and can be remuxed with the tag its packaging requires; stock FFmpeg has it in neither of its two mp4 tag tables, so the read probes as "unknown codec" and the write fails EINVAL (#547); same n8.1.2 binaries as 3.3.0 otherwise, compiled against the Xcode 27 SDKs with unchanged deployment targets; 3.3.0: every slice a shipped app can embed carries its dSYM inside the xcframework, so Xcode copies it into `.xcarchive/dSYMs` and a crash inside FFmpeg symbolicates instead of staying a list of addresses (FFmpegBuild#4); same n8.1.2 binaries as 3.2.1 apart from their UUID, the libraries just compile with `-gline-tables-only` now and the debug map is harvested before the shipped binary is stripped; 3.2.1: vc1_parser seeds its parse context from extradata, so a seek landing on an entry point alone keeps its picture size instead of falling back to 0x0 or to a size read out of the following payload (#490, FFmpeg PR 24458); 3.2.0: the legacy Flash tail, video and audio together (flv1 / vp6 / vp6f / vp6a, nellymoser, adpcm_swf, speex, and FLV's pcm_s16be / pcm_u8 / pcm_alaw / pcm_mulaw), so a pre-2008 .flv plays with sound where before only H.264-in-FLV did; flashsv stays out, it needs zlib and would have been dropped silently; 3.1.0: native .wmv / .asf, the asf demuxer plus every WMA decoder (Standard, Pro, Lossless, Voice), which is the container half of the legacy Microsoft support 2.4.3 started (FFmpegBuild#3); 3.0.0: every target, product, framework bundle and install name carries an `Aether` prefix, so this build can sit in an app that already has an FFmpeg (FFmpegKit, MobileVLCKit, mpv); same n8.1.2 binaries as 2.5.0, only names changed; 2.5.0: libzvbi 0.2.45 for GHSA-86rm-g7qf-j2fh (OOB read/write + integer underflow, reachable through the libzvbi_teletext decoder), dav1d 1.5.4, zimg 3.0.6, and the concat demuxer removed (a script demuxer selectable by probing alone, which made any stream a potential file-open primitive); 2.4.3: the legacy Microsoft video decoders (msmpeg4v1/v2/v3, wmv1/wmv2/wmv3; FFmpegBuild#3); 2.4.2: pgssubdec missing-palette recovery, replaces the 2.1.1 Epoch-Continue retain (#142); 2.4.1: qtrle decoder; 2.4.0: visionOS (xros) device + simulator slices; 2.3.0: webvtt demuxer (standalone .vtt sidecars, plus the cue settings as packet side data); 2.2.0: matroska TTS warn-only per RFC 9559 (#145 rework); 2.1.3: sup demuxer (raw PGS sidecars); 2.1.2: matroska TrackTimestampScale clamp (#145, dropped in 2.2.0); 2.1.1: pgssubdec Epoch-Continue retain (#142); 2.1.0: yadif_videotoolbox + hwupload (Metal GPU deinterlace); 2.0.0: dynamic frameworks (LGPL), zvbi GPL excision + .package(url: "https://github.com/superuser404notfound/FFmpegBuild", .upToNextMinor(from: "3.6.0")), // 3.6.0: an MPEG-TS audio PID is identified by its payload when its label does not name the codec: the raw dts / truehd / loas demuxers are in, so the mpegts content probe can confirm DTS, TrueHD and LATM at all (without them the lenient mp3 probe named the track, whatever the label), and a PID labelled 0x03 (MPEG-1 audio) gets the same content probe as 0x04; a DTS-HD IPTV channel opened as mp3 and never produced a frame (#641); same n8.1.3 otherwise; 3.5.0: FFmpeg n8.1.3, 268 upstream commits over n8.1.2, of which six touch code this engine ships: VP9 kept `next_refs[]` referenced across `avcodec_flush_buffers()`, so a worker could reseed its references from the pre-flush set and the first inter frame after a seek decoded against released buffers (heap OOB read and write, and this engine decodes VP9 with frame threading and flushes on every reposition); the Dolby Vision RPU parser trims trailing zero padding again, which n8.1.2 had stopped doing, so a padded RPU no longer fails the ext-block size check and loses its dynamic metadata, and the RPU partition counts are bounded; a VC-1 picture ending on a one-bit skipped macroblock keeps that macroblock instead of falling into error concealment; the `dca_core` bsf clears the profile it has just stripped; and a container channel layout survives a failed decoder open during probing. Same dav1d 1.5.4, zimg 3.0.6 and libzvbi 0.2.45 (all current upstream), same component set, same six local patches, unchanged soname majors; 3.4.0: FFmpeg learns the `dav1` sample entry in both directions, the AV1 variant of `av01` that signals Dolby Vision, so a Profile 10.0 source can be read at all and can be remuxed with the tag its packaging requires; stock FFmpeg has it in neither of its two mp4 tag tables, so the read probes as "unknown codec" and the write fails EINVAL (#547); same n8.1.2 binaries as 3.3.0 otherwise, compiled against the Xcode 27 SDKs with unchanged deployment targets; 3.3.0: every slice a shipped app can embed carries its dSYM inside the xcframework, so Xcode copies it into `.xcarchive/dSYMs` and a crash inside FFmpeg symbolicates instead of staying a list of addresses (FFmpegBuild#4); same n8.1.2 binaries as 3.2.1 apart from their UUID, the libraries just compile with `-gline-tables-only` now and the debug map is harvested before the shipped binary is stripped; 3.2.1: vc1_parser seeds its parse context from extradata, so a seek landing on an entry point alone keeps its picture size instead of falling back to 0x0 or to a size read out of the following payload (#490, FFmpeg PR 24458); 3.2.0: the legacy Flash tail, video and audio together (flv1 / vp6 / vp6f / vp6a, nellymoser, adpcm_swf, speex, and FLV's pcm_s16be / pcm_u8 / pcm_alaw / pcm_mulaw), so a pre-2008 .flv plays with sound where before only H.264-in-FLV did; flashsv stays out, it needs zlib and would have been dropped silently; 3.1.0: native .wmv / .asf, the asf demuxer plus every WMA decoder (Standard, Pro, Lossless, Voice), which is the container half of the legacy Microsoft support 2.4.3 started (FFmpegBuild#3); 3.0.0: every target, product, framework bundle and install name carries an `Aether` prefix, so this build can sit in an app that already has an FFmpeg (FFmpegKit, MobileVLCKit, mpv); same n8.1.2 binaries as 2.5.0, only names changed; 2.5.0: libzvbi 0.2.45 for GHSA-86rm-g7qf-j2fh (OOB read/write + integer underflow, reachable through the libzvbi_teletext decoder), dav1d 1.5.4, zimg 3.0.6, and the concat demuxer removed (a script demuxer selectable by probing alone, which made any stream a potential file-open primitive); 2.4.3: the legacy Microsoft video decoders (msmpeg4v1/v2/v3, wmv1/wmv2/wmv3; FFmpegBuild#3); 2.4.2: pgssubdec missing-palette recovery, replaces the 2.1.1 Epoch-Continue retain (#142); 2.4.1: qtrle decoder; 2.4.0: visionOS (xros) device + simulator slices; 2.3.0: webvtt demuxer (standalone .vtt sidecars, plus the cue settings as packet side data); 2.2.0: matroska TTS warn-only per RFC 9559 (#145 rework); 2.1.3: sup demuxer (raw PGS sidecars); 2.1.2: matroska TrackTimestampScale clamp (#145, dropped in 2.2.0); 2.1.1: pgssubdec Epoch-Continue retain (#142); 2.1.0: yadif_videotoolbox + hwupload (Metal GPU deinterlace); 2.0.0: dynamic frameworks (LGPL), zvbi GPL excision // Pure-Swift SMB2 client (MIT) that speaks the protocol over // NWConnection. Replaces AMSMB2/libsmb2, which EPERMs on tvOS/iOS. // Pinned to the 0.3.x minor: SMBClient is pre-1.0 with an actively diff --git a/README.md b/README.md index f927ec788..7fd3b8b0a 100644 --- a/README.md +++ b/README.md @@ -42,10 +42,14 @@ You provide the transport bar. You provide the dropdowns. You provide the pretty - [Syravo](https://syravo.app): Xtream Codes, Jellyfin and radio client for iPhone, iPad and Apple TV. - [KIPTV](https://kiptv.app): Premium, cross-platform IPTV Player. - [Silo](https://github.com/Silo-Server/silo-apple): native iOS, tvOS and macOS client for the Silo self-hosted media server. -- [File Box](https://apps.apple.com/app/id6765818194): File Box is a simple and practical local file manager that makes it easy to manage, view, organize, and process your files on iPhone and iPad. +- [File Box](https://apps.apple.com/app/id6765818194): File Box is a simple and practical local file manager on iPhone and iPad. - [Moonfin](https://github.com/Moonfin-Client/Moonfin-Core): A multi-platform third party Jellyfin client. - [Vivid](https://github.com/blurbery/vivid): open-source media app for iPhone, iPad and Apple TV. - [Snappier IPTV](https://apps.apple.com/gb/app/snappier-iptv/id1579702567): IPTV App for iOS/iPadOS and tvOS. +- [stashy](https://stashy.shelf.am/): iOS/tvOS native player for stash. +- [Gflix](https://gflixhub.app/): A flawless media app for your media. +- [Melon Video](https://apps.apple.com/ca/app/melon-video/id6811750997): Video Player for Vision Pro. +- [WUW IPTV Player](https://apps.apple.com/tr/app/wuw-iptv-player-smart-m3u-tv/id6759220800): Watch U Want — or simply WUW — your IPTV player for iPhone, iPad, Mac, Apple TV and Apple Vision Pro. Shipping something on AetherEngine? [Submit it](https://github.com/superuser404notfound/AetherEngine/issues/new?template=used-by-submission.yml) to get listed here and on [aetherengine.superuser404.de](https://aetherengine.superuser404.de). @@ -58,9 +62,9 @@ A scannable summary; the depth for each row lives in **[docs/formats.md](docs/fo | --- | --- | | Containers | MKV, MP4, WebM, MPEG-TS, AVI, ASF / WMV, OGG, FLV | | Disc | DVD-Video and Blu-ray ISO (decrypted): selectable titles and chapters, demuxed through the normal path | -| Video (HW) | H.264, HEVC, HEVC Main10 via VideoToolbox; AV1 where HW AV1 exists | -| Video (SW) | AV1 (dav1d) without HW, VP9 / VP8, MPEG-4 Part 2 / MPEG-2 / VC-1, QuickTime RLE, the Flash tail (Sorenson Spark, On2 VP6) and anything else the FFmpeg build carries a decoder for (software is the default route; only HEVC, H.264 and HW-decodable AV1 go native), H.264 High 4:2:2 / 4:4:4 / 10 and HEVC Rext where VideoToolbox has no HW decoder (Intel Macs, older chips), interlaced H.264 (AVPlayer does not deinterlace; on VOD the declared field order is verified against decoded frames, so progressive-in-interlaced-carriage keeps hardware decode); GPU deinterlace (yadif_videotoolbox, Metal, field-rate by default) with a CPU bwdif fallback | -| HDR | HDR10, HDR10+ (per-frame ST 2094-40), Dolby Vision (P5, P7 as single-layer 8.1, P8.1, P8.4, AV1 P10.x), HLG | +| Video (HW) | H.264, HEVC, HEVC Main10 via VideoToolbox; AV1 Main profile where HW AV1 exists | +| Video (SW) | AV1 (dav1d) without HW, and AV1 High / Professional (4:4:4, 4:2:2, 12-bit) everywhere, VP9 / VP8, MPEG-4 Part 2 / MPEG-2 / VC-1, QuickTime RLE, the Flash tail (Sorenson Spark, On2 VP6) and anything else the FFmpeg build carries a decoder for (software is the default route; only HEVC, H.264 and HW-decodable AV1 go native), H.264 High 4:2:2 / 4:4:4 / 10 and HEVC Rext where VideoToolbox has no HW decoder (Intel Macs, older chips), interlaced H.264 (AVPlayer does not deinterlace; on VOD the declared field order is verified against decoded frames, so progressive-in-interlaced-carriage keeps hardware decode); GPU deinterlace (yadif_videotoolbox, Metal, field-rate by default) with a CPU bwdif fallback | +| HDR | HDR10, HDR10+ (per-frame ST 2094-40), Dolby Vision (P5, P7 as single-layer 8.1, P8.1, P8.4, AV1 P10.x), HLG, HDR Vivid (played on its HLG / PQ base, CUVA metadata reported by `probe`) | | Audio | AAC, AC3, EAC3, FLAC, ALAC stream-copy lossless; TrueHD / MLP / DTS / DTS-HD MA / MP3 / MP2 / Opus / Vorbis / LPCM (incl. Blu-ray, G.711) / WMA (Standard, Pro, Lossless, Voice) / Nellymoser / ADPCM-SWF / Speex bridge to EAC3 5.1 (default) or lossless FLAC | | Dolby Atmos | EAC3+JOC stream-copied on every route (HDMI MAT 2.0, AirPods spatial, BT downmix). No container reliably declares JOC pre-decode, so an honest `TrackInfo.isAtmos` needs a bounded decode: `AetherEngine.probeDetectingAtmos(url:/source:)` answers for a details screen without starting playback, and `LoadOptions.confirmAtmos` has the running session confirm its own tracks in the background and republish `audioTracks`. Both are opt-in and neither sits on the playback-start path | | Surround | 5.1 / 7.1 with correct `AudioChannelLayout` | @@ -98,33 +102,72 @@ The engine leans on the platform where the platform is best (hardware decode, Do ### Measured -The table above is qualitative. The numbers below are measured, on a 4K HDR HEVC file in Matroska, the container media servers actually serve. They are produced by [aetherengine-bench](https://github.com/superuser404notfound/aetherengine-bench), which is run by this project's author, so its method, its raw data and the cases where AetherEngine does not win are all in that repository. +The table above is qualitative. The numbers below are measured on 4K HDR HEVC in Matroska, the container media servers actually serve, at 38 Mbit/s and at 90 Mbit/s, the rate of a UHD remux. Every engine plays each file twice: from local disk, and over HTTP from a local range origin with a fixed latency and a shared link rate, because media servers deliver over HTTP and an engine's network reader is not its file reader. They are produced by [aetherengine-bench](https://github.com/superuser404notfound/aetherengine-bench), which is run by this project's author, so its method, its raw data and the cases where AetherEngine does not win are all in that repository. AVPlayer does not open Matroska at all; on MP4, the one container it plays, it uses less than every engine here, see that repository's README. -Measured on Apple-M1 (MacBookAir10,1), macOS 26.5.2, hevc-4k-hdr10.mkv, windowed, 3840x2160 px rendered, 60 s, median of 2. +Measured on Apple-M1 (MacBookAir10,1), macOS 27.0, hevc-4k-hdr10.mkv (38 Mbit/s), hevc-4k-hdr10-90m.mkv (90 Mbit/s), each from local disk and over HTTP from a local range origin at 1000 Mbit/s shared, 20 ms latency per request, windowed, 3840x2160 px rendered, 60 s, median of 2. MacBookAir10,1 is fanless: sustained decode can reach thermal pressure, which is why the protocol has cooldowns between runs and discards throttled windows. -| | GPU power | CPU load | RSS | Plays | -| --- | --- | --- | --- | --- | -| **AetherEngine** | 63 mW | 5.2% of a core | 325 MB | Yes | -| **KSPlayer** | 149 mW | 6.6% of a core | 331 MB | Yes | -| **AVPlayer** | | | | n/a (refuses hevc-4k-hdr10.mkv: This media format is not supported.) | -| **VLCKit** | 362 mW | 14.3% of a core | 102 MB | Yes | -| **libmpv** | 887 mW | 18.1% of a core | 373 MB | Yes | - -Launch failures on hevc-4k-hdr10.mkv (crashes that were retried, not refusals, see the README): KSPlayer 1. +**hevc-4k-hdr10.mkv (38 Mbit/s)** + +| | Source | GPU power | CPU load | RSS | Peak footprint | Plays | +| --- | --- | --- | --- | --- | --- | --- | +| **AetherEngine** | disk | 41 mW | 4.4% of a core | 342 MB | 323 MB | Yes | +| | HTTP | 40 mW | 5.4% of a core | 336 MB | 350 MB | Yes | +| **KSPlayer** | disk | 43 mW | 5.8% of a core | 333 MB | 772 MB | Yes | +| | HTTP | 41 mW | 6.4% of a core | 357 MB | 496 MB | Yes | +| **AVPlayer** | disk | | | | | n/a (refuses hevc-4k-hdr10.mkv: This media format is not supported.) | +| | HTTP | | | | | n/a (refuses hevc-4k-hdr10.mkv: This media format is not supported.) | +| **VLCKit** | disk | 266 mW | 15.0% of a core | 111 MB | 281 MB | Yes | +| | HTTP | 244 mW | 17.0% of a core | 122 MB | 300 MB | Yes | +| **libmpv** | disk | 887 mW | 20.7% of a core | 475 MB | 933 MB | Yes | +| | HTTP | 892 mW | 21.5% of a core | 629 MB | 1191 MB | Yes | + +Launch failures on hevc-4k-hdr10.mkv (crashes that were retried, not refusals, see the README): KSPlayer 1 (HTTP). + +**hevc-4k-hdr10-90m.mkv (90 Mbit/s)** + +| | Source | GPU power | CPU load | RSS | Peak footprint | Plays | +| --- | --- | --- | --- | --- | --- | --- | +| **AetherEngine** | disk | 40 mW | 6.6% of a core | 577 MB | 681 MB | Yes | +| | HTTP | 40 mW | 9.3% of a core | 557 MB | 700 MB | Yes | +| **KSPlayer** | disk | 41 mW | 6.7% of a core | 589 MB | 1040 MB | Yes | +| | HTTP | 41 mW | 7.6% of a core | 578 MB | 703 MB | Yes | +| **AVPlayer** | disk | | | | | n/a (refuses hevc-4k-hdr10-90m.mkv: This media format is not supported.) | +| | HTTP | | | | | n/a (refuses hevc-4k-hdr10-90m.mkv: This media format is not supported.) | +| **VLCKit** | disk | 235 mW | 16.2% of a core | 117 MB | 314 MB | Yes | +| | HTTP | 235 mW | 18.4% of a core | 135 MB | 325 MB | Yes | +| **libmpv** | disk | 953 mW | 21.5% of a core | 501 MB | 980 MB | Yes | +| | HTTP | 962 mW | 22.3% of a core | 684 MB | 1188 MB | Yes | + +Launch failures on hevc-4k-hdr10-90m.mkv (crashes that were retried, not refusals, see the README): KSPlayer 4 (disk). Frame delivery and output (median; resolution, bit depth and HDR transfer are informational per engine only, shown as 'varies across repeats' when they disagree, see the README): -- **AetherEngine**: 1440/1438 delivered/expected, dropped 0, 3840x1714, 10-bit, hdr10, rendered into 3840x2160. -- **KSPlayer**: 1440/1440 delivered/expected, dropped 0, 3840x1714, 10-bit, SMPTE_ST_2084_PQ, rendered into 3840x2160. Served via **KSMEPlayer**. -- **VLCKit**: 1442/1440 delivered/expected, dropped 0, 3840x1714, bit depth and color transfer not reported by this engine, rendered into 3840x2160. -- **libmpv**: 1442/1440 delivered/expected, dropped 0, 3840x1714, 10-bit, pq, rendered into 3840x2160. - -Versions: AetherEngine 6.26.0, KSPlayer 2.3.4, AVPlayer macOS 26.5.2, VLCKit 4.0.0-alpha.21, libmpv mpv v0.41.0. +*hevc-4k-hdr10.mkv* +- **AetherEngine, disk**: 1438/1438 delivered/expected, dropped 0, 3840x1714, 10-bit, hdr10, rendered into 3840x2160. +- **AetherEngine, HTTP**: 1441/1438 delivered/expected, dropped 0, 3840x1714, 10-bit, hdr10, rendered into 3840x2160. +- **KSPlayer, disk**: 1440/1440 delivered/expected, dropped 0, 3840x1714, 10-bit, SMPTE_ST_2084_PQ, rendered into 3840x2160. Served via **KSMEPlayer**. +- **KSPlayer, HTTP**: 1440/1440 delivered/expected, dropped 0, 3840x1714, 10-bit, SMPTE_ST_2084_PQ, rendered into 3840x2160. Served via **KSMEPlayer**. +- **VLCKit, disk**: 1451/1440 delivered/expected, dropped 0, 3840x1714, bit depth and color transfer not reported by this engine, rendered into 3840x2160. +- **VLCKit, HTTP**: 1454/1440 delivered/expected, dropped 0, 3840x1714, bit depth and color transfer not reported by this engine, rendered into 3840x2160. +- **libmpv, disk**: 1443/1440 delivered/expected, dropped 0, 3840x1714, 10-bit, pq, rendered into 3840x2160. +- **libmpv, HTTP**: 1443/1440 delivered/expected, dropped 0, 3840x1714, 10-bit, pq, rendered into 3840x2160. +*hevc-4k-hdr10-90m.mkv* +- **AetherEngine, disk**: 1436/1438 delivered/expected, dropped 0, 3840x1714, 10-bit, hdr10, rendered into 3840x2160. +- **AetherEngine, HTTP**: 1439/1438 delivered/expected, dropped 0, 3840x1714, 10-bit, hdr10, rendered into 3840x2160. +- **KSPlayer, disk**: 1440/1440 delivered/expected, dropped 0, 3840x1714, 10-bit, SMPTE_ST_2084_PQ, rendered into 3840x2160. Served via **KSMEPlayer**. +- **KSPlayer, HTTP**: 1440/1440 delivered/expected, dropped 0, 3840x1714, 10-bit, SMPTE_ST_2084_PQ, rendered into 3840x2160. Served via **KSMEPlayer**. +- **VLCKit, disk**: 1454/1440 delivered/expected, dropped 0, 3840x1714, bit depth and color transfer not reported by this engine, rendered into 3840x2160. +- **VLCKit, HTTP**: 1452/1440 delivered/expected, dropped 0, 3840x1714, bit depth and color transfer not reported by this engine, rendered into 3840x2160. +- **libmpv, disk**: 1442/1440 delivered/expected, dropped 0, 3840x1714, 10-bit, pq, rendered into 3840x2160. +- **libmpv, HTTP**: 1443/1440 delivered/expected, dropped 0, 3840x1714, 10-bit, pq, rendered into 3840x2160. + +Versions: AetherEngine 7.15.2, KSPlayer 2.3.4, AVPlayer macOS 27.0, VLCKit 4.0.0-alpha.21, libmpv mpv v0.41.0. Power figures are package power with an idle baseline subtracted, so they are attributable to the run and not to the machine. -CPU package power was measured for every run but is not published above: the idle baseline drifts with load enough that one engine's own repeats can disagree more than the column would be used to show between engines. For example, on hevc-4k-hdr10.mkv libmpv measured 100 to 372 mW of CPU package power across its own repeats, a 3.7x range. Recorded for every run in Results/; see "Known gaps" in the README for the full reasoning. +CPU package power was measured for every run but is not published above: the idle baseline drifts with load enough that one engine's own repeats can disagree more than the column would be used to show between engines. For example, on hevc-4k-hdr10.mkv KSPlayer measured 76 to 267 mW of CPU package power across its own repeats, a 3.5x range. Recorded for every run in Results/; see "Known gaps" in the README for the full reasoning. libmpv is measured with --hwdec=auto-safe (hardware decode), not mpv's own software-decode default, see "Fairness decisions" in the README. -libmpv's rows come from a separate run of the same protocol on the same day and the same machine. Its rows in the main run were lost to a defect in the mpv report writer (a missing import), not to anything about mpv itself. The GPU idle baseline on this machine is under 1 mW, and CPU load and RSS are not baseline-subtracted at all, so a separate baseline does not move the three published columns. +Peak footprint is the kernel's lifetime maximum of the player process's physical footprint (what the system's memory limit acts on), load phase included; RSS is the mean over the measured window. Decoding that happens in a system service (VideoToolbox) is in neither. +The hevc-4k-hdr10-90m.mkv rows come from a second run of the same protocol, build and origin on the same day and machine. A tvOS Simulator had been left running in the background for the first half of the main run, which raised idle package power to 188 to 214 mW (19 to 26 mW once it was shut down). Both hevc-4k-hdr10-90m.mkv baselines failed validation under it, so that fixture was measured again. The hevc-4k-hdr10.mkv baselines were taken under it too and passed; the GPU idle baseline stayed under 0.05 mW throughout, and CPU load, RSS and footprint are not baseline-subtracted, so none of the published columns moved with it. Method and raw results: https://github.com/superuser404notfound/aetherengine-bench ## Quick start @@ -342,7 +385,7 @@ Subtitle cues land in raw source PTS; render the overlay against `player.sourceT Install via Swift Package Manager: ```swift -.package(url: "https://github.com/superuser404notfound/AetherEngine", from: "7.13.0") +.package(url: "https://github.com/superuser404notfound/AetherEngine", from: "7.28.3") ``` Three samples ship in `Examples/`: @@ -435,7 +478,7 @@ player.$recordingState // .idle / .recording / .ended / .failed `startRecording(to:)` (AetherEngine#560) writes the live source to an MPEG-TS file fed from the connection the session already holds. That is the point of it rather than a detail: an IPTV plan commonly caps an account at 1 to 3 simultaneous connections, so a host that opens its own connection to record either fails outright or knocks the viewer off the channel. It is a stream copy of the source packets taken before any audio bridging, so a TrueHD or DTS channel records its original audio while playback listens to the bridged rendition, and a file cut short by a crash is still playable up to the cut. Recording follows the source rather than the playhead, so pausing or scrubbing back inside the DVR window does not interrupt it, and a `liveSourceReset` ends it cleanly rather than writing past a seam the codecs may not survive. The one route that cannot record is `nativeRemoteHLS` (`.remoteBypass`), where AVFoundation holds the source connection and the engine never sees a byte; `startRecording` throws `.unsupportedRoute` there instead of producing an empty file. Full contract in [docs/api.md](docs/api.md#recording-a-live-stream). -`liveJoinProfile: .fastZap` (AetherEngine#195/#208) cuts live segments at every keyframe past 0.5 s instead of the standard ~4 s, so the served `TARGETDURATION` collapses to the source GOP length and its live-edge holdback (`HOLD-BACK` = 3 x `TARGETDURATION`, the RFC 8216bis floor; AetherEngine#189) shrinks with it. The first manifest still prefers the full holdback. After two finalized segments, a strict-realtime source gets one observed-segment grace clamped to 0.5...2.0 s, then a shallow first window may be served so startup stays bounded. This can produce one early `-16832` or a short rebuffer. `.standard` retains the full-holdback guarantee. The smaller `TARGETDURATION` also tightens AVPlayer's unchanged-playlist patience and live-edge buffer, so origins that stall or burst mid-stream rebuffer more readily; opt in for zapping UX, keep `.standard` for lean-back viewing. +`liveJoinProfile: .fastZap` (AetherEngine#195/#208) cuts live segments at every keyframe past 0.5 s instead of the standard ~4 s, so the served `TARGETDURATION` collapses to 1.5 x the source GOP length (1 s GOPs serve 2, and so do segments up to 1.666 s; the headroom covers a broadcast's irregular GOPs, AetherEngine#670) and its live-edge holdback (`HOLD-BACK` = 3 x `TARGETDURATION`, the RFC 8216bis floor; AetherEngine#189) shrinks with it. The first manifest still prefers the full holdback. After two finalized segments, a strict-realtime source gets one observed-segment grace clamped to 0.5...2.0 s, then a shallow first window may be served so startup stays bounded. This can produce one early `-16832` or a short rebuffer. `.standard` retains the full-holdback guarantee. The smaller `TARGETDURATION` also tightens AVPlayer's unchanged-playlist patience and live-edge buffer, so origins that stall or burst mid-stream rebuffer more readily; opt in for zapping UX, keep `.standard` for lean-back viewing. `liveSourceReset` is live's counterpart to a terminal `state = .error`, and a host that plays live has to subscribe to it. It fires where the session cannot be revived from inside the engine and only a new URL can: a source that restarted from byte 0 (a Jellyfin transcode respawn), a playlist still frozen after the stall ladder's last reload rung (#65), or an in-engine reopen transport whose budget is spent (#199). Each of those halts production first, so a dead provider stops advertising blocking reloads behind the host's back. Answer it by negotiating a fresh URL and calling `load` again, and guard that answer (one retune in flight, a minimum spacing, a bounded count per session) or a permanently dead upstream turns into a retune loop. Spend that bound out loud: a retune ladder that ends on a silent `return` leaves the same dead channel behind a counter, so surface the exhaustion the way a terminal `.error` would be surfaced. @@ -451,7 +494,7 @@ For an upstream AVPlayer can play natively (a standard remote `master.m3u8`, e.g A live channel whose master advertises HEVC (or Dolby Vision / AV1) while delivering MPEG-TS segments is carriage AVFoundation builds no video track for, so the bypass would play it as audio over black. The engine recognizes that signature and reroutes the session onto the live ingest above (#168). The recognition runs alongside the mount: the playlist and the head of one segment are read while AVPlayer starts, so the reroute does not wait out a grace window, and a media playlist URL with no master to judge is covered too (#293). Only a codec the HLS Authoring Spec sanctions in fMP4 alone reaches that read, so an H.264 channel spends no extra request on it. `LoadOptions.nativeRemoteHLSIngestFallback = false` turns the whole recovery off. -A LIVE remote `m3u8` handed to the default (loopback) path routes the other way, onto the live ingest above (#363). The raw live path reads bytes, not playlists, so it used to reject an `.m3u8` with a typed `hlsPlaylistOnRawLivePath` error naming the reader the host should have built. It builds that reader itself now, with `LoadOptions.httpHeaders` on the playlist, on every segment and on every AES key, which is what a tokenized IPTV origin enforces per request. A custom `IOReader` carrying the same misroute still gets the typed error: it has no playlist URL to ingest from. +A LIVE remote `m3u8` handed to the default (loopback) path routes the other way, onto the live ingest above (#363). The raw live path reads bytes, not playlists, so it used to reject an `.m3u8` with a typed `hlsPlaylistOnRawLivePath` error naming the reader the host should have built. It builds that reader itself now, with `LoadOptions.httpHeaders` on the playlist, on every segment and on every AES key, which is what a tokenized IPTV origin enforces per request. Credential headers stay with the playlist's own origin: a segment or key on another host, or on http:// behind an https:// playlist, gets the rest of the headers but not the token. A custom `IOReader` carrying the same misroute still gets the typed error: it has no playlist URL to ingest from. If a live `nativeRemoteHLS` bypass is refused by the origin outright (HTTP 401 or 403, which reach the item as `NSURLError` -1013 / -1102), the engine hands that session to the same ingest instead of failing the load (#363). The ingest fetcher is a different client at that origin: it carries the configured headers on every request, caps itself at four concurrent fetches, and sends no AVFoundation user agent, so a UA filter or a per-token connection cap that turned AVPlayer away can still serve it. The refusal is not remembered for the next load the way a carriage verdict is; a token expires, a cap frees up. `LoadOptions.nativeRemoteHLSIngestFallback = false` turns this off along with the carriage recovery. @@ -589,10 +632,10 @@ Browse all of this as a searchable site at **[aetherengine.superuser404.de](http AetherEngine uses [Semantic Versioning](https://semver.org). The public API surface, every `public` declaration in `Sources/AetherEngine/`, is the stability contract. **Major** removes / renames public symbols or breaks adopters; **Minor** adds public API or codec / format support; **Patch** fixes bugs with no public API change. `internal` types are not part of the contract. ```swift -.package(url: "https://github.com/superuser404notfound/AetherEngine", from: "7.13.0") +.package(url: "https://github.com/superuser404notfound/AetherEngine", from: "7.28.3") ``` -Pin to `.upToNextMinor(from: "7.13.0")` for stricter teams that prefer to opt into minor bumps explicitly. +Pin to `.upToNextMinor(from: "7.28.3")` for stricter teams that prefer to opt into minor bumps explicitly. ## Requirements @@ -602,8 +645,10 @@ Pin to `.upToNextMinor(from: "7.13.0")` for stricter teams that prefer to opt in | tvOS | 18.0 | | macOS | 15.0 | | visionOS | 1.0 | -| Swift | 6.0 | -| Xcode | 16.0 | +| Swift | 6.4 | +| Xcode | 27.0 | + +The Swift and Xcode rows are the toolchain CI builds and tests with (GitHub's `xcode-27` image); older toolchains are not tested. The source needs at least the 26.2 SDK in any case (`VTRegisterSupplementalVideoDecoderIfAvailable` is iOS / tvOS 26.2 API, and an unguarded C symbol has to exist at compile time). The deployment floor above stays iOS / tvOS 18 and macOS 15, with the newer calls behind `#available`. The 7.x line raised the floor from iOS 16, tvOS 17 and macOS 14. A project that still supports those pins the 6.x line with `.upToNextMajor(from: "6.89.1")`. diff --git a/SECURITY.md b/SECURITY.md index 11453089d..fb6f25c84 100644 --- a/SECURITY.md +++ b/SECURITY.md @@ -4,10 +4,10 @@ AetherEngine ships fixes on the latest released minor line. Security fixes land there first; older lines are not back-patched. Host apps pin the engine by commit SHA, so picking up a fix means bumping the pin to the patched release. -| Version | Supported | -| ------- | ------------------ | -| 2.1.x | :white_check_mark: | -| < 2.1 | :x: | +| Version | Supported | +| ------- | --------- | +| The minor line of the [latest release](https://github.com/superuser404notfound/AetherEngine/releases/latest) | :white_check_mark: | +| Every earlier minor line | :x: | ## Reporting a vulnerability @@ -28,7 +28,7 @@ You can expect an initial acknowledgement within a few days. Once a fix is ready AetherEngine plays media from servers and local sources and parses untrusted container and codec data. Areas most relevant to security: - **Media parsing.** Demuxing and decoding of untrusted containers and bitstreams (the FFmpeg / dav1d surface). -- **Network handling.** The engine's HTTP range reading and its on-device loopback server used to bridge sources into AVPlayer. Nothing is exposed off the device, and there is no external analytics or session reporting. +- **Network handling.** The engine's HTTP range reading and its local HTTP server used to bridge sources into AVPlayer. The server listens on all interfaces, not only loopback, so that an AirPlay receiver can fetch the stream over the LAN, on an ephemeral port and for the current stream only. Every path it answers starts with a per-session 128-bit random token, and a request without it is answered 404 before it reaches any route. A peer that is not loopback may hold at most 24 of the server's 32 connection slots (the rest stay free for the local player), and a connection that has not presented the token has 10 seconds from accept to deliver a whole request head. The token is also kept out of the diagnostic log. Beyond that listener nothing is exposed off the device, and there is no external analytics or session reporting. - **Memory safety.** Crashes or out-of-bounds behavior triggered by crafted input. Out of scope: vulnerabilities in a host app's own UI or networking (report those on the host app's tracker), and issues in upstream FFmpeg / dav1d themselves (report those upstream, though we are glad to know if a bundled build is affected). diff --git a/Scripts/fetch-fixtures.sh b/Scripts/fetch-fixtures.sh index 2c2e4514e..320facfb6 100755 --- a/Scripts/fetch-fixtures.sh +++ b/Scripts/fetch-fixtures.sh @@ -19,6 +19,7 @@ # a53-captions.mp4 - H.264 with in-picture A/53 CEA-608 SEI (#131, #259) # hev1-inband-xps.mp4 - HEVC with in-band VPS/SPS/PPS and an empty hvcC # cue-axis-bframes.mkv/.mp4 - one HEVC B-pyramid stream in both containers (AE#561) +# long-first-gop-{aac,pcm}.mkv - 11 s first GOP, then 2 s GOPs; passthrough and bridged audio # bridge-eac3-51.mkv - 5.1 PCM in MKV, drives the EAC3 audio bridge (AE#561 follow-up) # # Real-world DV / Atmos / multichannel sources have to come from your @@ -370,6 +371,21 @@ ffmpeg -hide_banner -loglevel error -y -copyts -i "$OFFSET_TS" -c copy \ -hls_segment_filename "$FIXTURES_DIR/user/hls-hevc-vod-offset/seg%03d.ts" \ "$FIXTURES_DIR/user/hls-hevc-vod-offset/media.m3u8" +# A sequential origin's playlist is cut on video keyframes. A first GOP far longer than the 4 s stride +# is the shape a remote MKV had (11.3 s). Served forward-only its Cues are never read, so the plan is +# the uniform stride and audio reaches every boundary long before a keyframe does. One clip per audio +# path: AAC is passed through, stereo PCM goes through the FLAC bridge (the reporter's Vorbis did). +for AUDIO in aac pcm; do + CODEC=aac; [ "$AUDIO" = pcm ] && CODEC=pcm_s16le + echo "→ long-first-gop-$AUDIO.mkv (keyframes at 0, 11, 13, 15 ... s, 30 s)" + ffmpeg -hide_banner -loglevel error -y \ + -f lavfi -i "testsrc2=size=320x180:rate=24" \ + -f lavfi -i "sine=frequency=440:sample_rate=48000" -t 30 \ + -c:v libx264 -preset veryfast -pix_fmt yuv420p -b:v 200k -bf 0 \ + -g 1000 -sc_threshold 0 -force_key_frames "0,11,13,15,17,19,21,23,25,27,29" \ + -c:a "$CODEC" -ac 2 -f matroska "$FIXTURES_DIR/long-first-gop-$AUDIO.mkv" +done + echo "" echo "Done. Try:" echo " swift run aetherctl probe $FIXTURES_DIR/sdr-h264.mp4" diff --git a/Scripts/hls-burst-origin.py b/Scripts/hls-burst-origin.py new file mode 100755 index 000000000..71d5ca245 --- /dev/null +++ b/Scripts/hls-burst-origin.py @@ -0,0 +1,167 @@ +#!/usr/bin/env python3 +r"""A live HLS origin whose DELIVERY RHYTHM is scripted (AE#684). + +`aetherctl hlsfixture` publishes on a metronome, and a metronome is the one thing no IPTV origin is: +the defect this was written for only exists when a delivery arrives later than the segment is long. +This serves a sliding playlist over pre-cut TS segments and publishes segment k at + + end of segment k - what was prefilled + delays[k % len(delays)] + +seconds after it starts, so the mean cadence stays the segment duration (the origin never falls +behind real time) and the jitter is the delay cycle. `--freeze-at K --freeze-seconds F` holds +everything from segment K on for F seconds: an outage, after which the backlog lands at once. +Every request is one stderr line stamped with seconds since start. + + # 720x576 25 fps, 2 s GOPs, 6 s segments, a white frame and a 40 ms beep on every whole second + ffmpeg -f lavfi -i "color=c=black:s=720x576:r=25:d=600,drawbox=x=0:y=0:w=720:h=576:color=white:\ +t=fill:enable='lt(mod(t,1),0.039)',format=yuv420p" \ + -f lavfi -i "aevalsrc='if(lt(mod(t,1),0.04),0.8*sin(2*PI*1000*t),0)':s=48000:d=600:c=stereo" \ + -c:v libx264 -preset ultrafast -profile:v main -g 50 -keyint_min 50 -sc_threshold 0 -bf 0 \ + -b:v 1500k -maxrate 1500k -bufsize 3000k -x264-params nal-hrd=cbr \ + -c:a aac -b:a 128k -ar 48000 -f hls -hls_time 6 -hls_list_size 0 -hls_segment_type mpegts \ + -hls_flags independent_segments -hls_segment_filename 'seg6/seg%d.ts' seg6/out.m3u8 + + Scripts/hls-burst-origin.py --dir seg6 --delays 0,0.3,2.6,0.2,0.6,2.4,0.1,0.5 & + aetherctl play --live --live-ingest --fast-zap --seconds 180 http://127.0.0.1:8684/live.m3u8 + +Threaded and HTTP/1.0 on purpose: the ingest fetches up to four segments at once, and a +keep-alive connection would hide which request paid for what. +""" +import argparse, os, sys, threading, time +from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer + +ap = argparse.ArgumentParser() +ap.add_argument("--dir", required=True) +ap.add_argument("--port", type=int, default=8684) +ap.add_argument("--dur", type=float, default=6.0) +ap.add_argument("--durs", default="", help="comma list of per-segment durations, cycled (overrides --dur)") +ap.add_argument("--target-duration", type=int, default=6) +ap.add_argument("--window", type=int, default=6) +ap.add_argument("--prefill", type=int, default=6, help="segments already published at start") +ap.add_argument("--delays", default="0", help="comma list of per-segment publish delays, cycled") +ap.add_argument("--freeze-at", type=int, default=-1) +ap.add_argument("--freeze-seconds", type=float, default=0) +ap.add_argument("--latency-ms", type=float, default=0, help="delay before every response header") +ap.add_argument("--rate-kbps", type=float, default=0, help="link rate shared by all in-flight responses") +args = ap.parse_args() + +delays = [float(x) for x in args.delays.split(",")] +count = len([f for f in os.listdir(args.dir) if f.startswith("seg") and f.endswith(".ts")]) +# Both index the segment list below, so a short folder would otherwise end in an IndexError. +if not 0 <= args.prefill <= count: + ap.error("--prefill must be 0 to %d, the segments in %s (got %d)" % (count, args.dir, args.prefill)) +if args.freeze_at >= count: + ap.error("--freeze-at must be below %d, the segments in %s (got %d)" % (count, args.dir, args.freeze_at)) +durs = [float(x) for x in args.durs.split(",")] if args.durs else [args.dur] +seg_dur = [durs[k % len(durs)] for k in range(count)] +ends = [] +acc = 0.0 +for k in range(count): + acc += seg_dur[k] + ends.append(acc) +prefill_end = ends[args.prefill - 1] if args.prefill > 0 else 0.0 +publish = [] +for k in range(count): + p = 0.0 if k < args.prefill else ends[k] - prefill_end + delays[k % len(delays)] + publish.append(p) +if args.freeze_at >= 0: + thaw = publish[args.freeze_at] + args.freeze_seconds + for k in range(args.freeze_at, count): + publish[k] = max(publish[k], thaw) +for k in range(1, count): + publish[k] = max(publish[k], publish[k - 1]) + +t0 = time.monotonic() +lock = threading.Lock() + + +def log(msg): + with lock: + sys.stderr.write("%8.3f %s\n" % (time.monotonic() - t0, msg)) + sys.stderr.flush() + + +rate_lock = threading.Lock() +rate_clock = [0.0] + + +def send_paced(wfile, body): + """One virtual clock for every response: a second connection gets no bandwidth of its own.""" + if args.rate_kbps <= 0: + wfile.write(body) + return + chunk = 16 * 1024 + for i in range(0, len(body), chunk): + part = body[i:i + chunk] + with rate_lock: + now = time.monotonic() + start = max(now, rate_clock[0]) + rate_clock[0] = start + len(part) * 8 / (args.rate_kbps * 1000.0) + due = rate_clock[0] + wait = due - time.monotonic() + if wait > 0: + time.sleep(wait) + wfile.write(part) + + +def playlist(): + now = time.monotonic() - t0 + last = -1 + for k in range(count): + if publish[k] <= now: + last = k + else: + break + first = max(0, last - args.window + 1) + lines = ["#EXTM3U", "#EXT-X-VERSION:3", "#EXT-X-TARGETDURATION:%d" % args.target_duration, + "#EXT-X-MEDIA-SEQUENCE:%d" % first] + for k in range(first, last + 1): + lines.append("#EXTINF:%.3f," % seg_dur[k]) + lines.append("seg%d.ts" % k) + return ("\n".join(lines) + "\n").encode(), last + + +class H(BaseHTTPRequestHandler): + protocol_version = "HTTP/1.0" + + def log_message(self, *a): + pass + + def do_GET(self): + if args.latency_ms > 0: + time.sleep(args.latency_ms / 1000.0) + path = self.path.split("?")[0] + if path.endswith(".m3u8"): + body, last = playlist() + log("REQ playlist last=seg%d" % last) + self.send_response(200) + self.send_header("Content-Type", "application/vnd.apple.mpegurl") + elif path.endswith(".ts"): + name = os.path.basename(path) + try: + with open(os.path.join(args.dir, name), "rb") as f: + body = f.read() + except OSError: + log("REQ %s 404" % name) + self.send_response(404) + self.end_headers() + return + log("REQ %s" % name) + self.send_response(200) + self.send_header("Content-Type", "video/mp2t") + else: + self.send_response(404) + self.end_headers() + return + self.send_header("Content-Length", str(len(body))) + self.end_headers() + try: + send_paced(self.wfile, body) + except OSError: + pass + + +gaps = [publish[k] - publish[k - 1] for k in range(args.prefill, min(count, args.prefill + 40))] +log("origin up: %d segments, publish gaps (first 40 after prefill): %s" % ( + count, " ".join("%.1f" % g for g in gaps))) +ThreadingHTTPServer(("127.0.0.1", args.port), H).serve_forever() diff --git a/Scripts/make-hdr10plus-fixture.py b/Scripts/make-hdr10plus-fixture.py index 9dd4f33da..015ceaaa5 100755 --- a/Scripts/make-hdr10plus-fixture.py +++ b/Scripts/make-hdr10plus-fixture.py @@ -1,6 +1,9 @@ #!/usr/bin/env python3 """Build a tiny HEVC/PQ MP4 that carries a real ST 2094-40 (HDR10+) T.35 SEI. +With `--vivid` it builds an HEVC/HLG MP4 carrying a CUVA HDR Vivid T.35 SEI instead, laid out as +libavcodec/dynamic_hdr_vivid.c parses it, and verified by ffprobe reporting "CUVA" (#699). + Layout of the payload follows libavutil/hdr_dynamic_metadata.c's parser exactly, so `ffprobe -show_frames` reporting "HDR Dynamic Metadata SMPTE2094-40" is the proof the fixture is genuine rather than a byte pattern that merely looks like one. @@ -57,6 +60,31 @@ def hdr10plus_payload(): return header + body +def hdr_vivid_payload(): + w = BitWriter() + w.u(8, 1) # system_start_code + for value in (0, 3046, 1535, 4095): + w.u(12, value) # minimum / average / variance / maximum maxrgb + w.u(1, 1) # tone_mapping_mode_flag + w.u(1, 0) # tone_mapping_param_num - 1 + w.u(12, 2770) # targeted_system_display_maximum_luminance + w.u(1, 1) # base_enable_flag + for bits, value in ((14, 5734), (6, 24), (10, 510), (10, 0), (6, 10), (2, 1), (2, 1), (4, 1), (3, 0), (7, 0)): + w.u(bits, value) + w.u(1, 1) # 3Spline_enable_flag + w.u(1, 0) # 3Spline_num - 1 + w.u(2, 0) # 3Spline_TH_mode + w.u(8, 224) # 3Spline_TH_enable_MB + for bits, value in ((12, 0), (10, 511), (10, 511), (8, 127)): + w.u(bits, value) + w.u(1, 1) # color_saturation_mapping_flag + w.u(3, 2) # color_saturation_num + w.u(8, 38) + w.u(8, 25) + header = bytes([0x26, 0x00, 0x04, 0x00, 0x05]) + return header + w.bytes() + + def emulation_prevent(rbsp): out = bytearray() zeros = 0 @@ -111,18 +139,20 @@ def nal_type(nal): def main(): - out_dir = sys.argv[1] + vivid = "--vivid" in sys.argv[1:] + out_dir = [a for a in sys.argv[1:] if a != "--vivid"][0] + transfer = ("18", "arib-std-b67") if vivid else ("16", "smpte2084") plain = os.path.join(out_dir, "plain.mp4") annexb = os.path.join(out_dir, "plain.hevc") injected = os.path.join(out_dir, "injected.hevc") - fixture = os.path.join(out_dir, "hdr10plus-hevc.mp4") + fixture = os.path.join(out_dir, "hdr-vivid-hevc.mp4" if vivid else "hdr10plus-hevc.mp4") subprocess.run([ "ffmpeg", "-y", "-v", "error", "-f", "lavfi", "-i", "color=c=black:s=64x64:r=10:d=0.2", "-c:v", "libx265", "-pix_fmt", "yuv420p10le", - "-x265-params", "log-level=none:info=0:keyint=1:min-keyint=1:colorprim=9:transfer=16:colormatrix=9", - "-color_primaries", "bt2020", "-color_trc", "smpte2084", "-colorspace", "bt2020nc", + "-x265-params", "log-level=none:info=0:keyint=1:min-keyint=1:colorprim=9:transfer=" + transfer[0] + ":colormatrix=9", + "-color_primaries", "bt2020", "-color_trc", transfer[1], "-colorspace", "bt2020nc", "-frames:v", "2", plain, ], check=True) @@ -132,7 +162,7 @@ def main(): ], check=True) data = open(annexb, "rb").read() - payload = hdr10plus_payload() + payload = hdr_vivid_payload() if vivid else hdr10plus_payload() sei = sei_nal(payload) out = bytearray() for nal in split_annexb(data): @@ -150,10 +180,13 @@ def main(): "ffprobe", "-v", "error", "-select_streams", "v:0", "-show_frames", "-show_entries", "frame=side_data_list", fixture, ], capture_output=True, text=True) - ok = "SMPTE2094-40" in probe.stdout or "HDR10+" in probe.stdout + if vivid: + ok = "CUVA" in probe.stdout + else: + ok = "SMPTE2094-40" in probe.stdout or "HDR10+" in probe.stdout print(probe.stdout[:600]) print("SIZE:", os.path.getsize(fixture), "bytes") - print("HDR10+ PARSED BY FFMPEG:", ok) + print("HDR VIVID" if vivid else "HDR10+", "PARSED BY FFMPEG:", ok) if ok: print("BASE64_START") print(base64.b64encode(open(fixture, "rb").read()).decode()) diff --git a/Scripts/tests/SoftwareStoredPacketStandalone.swift b/Scripts/tests/SoftwareStoredPacketStandalone.swift index 8dba719a0..5a9b93c53 100644 --- a/Scripts/tests/SoftwareStoredPacketStandalone.swift +++ b/Scripts/tests/SoftwareStoredPacketStandalone.swift @@ -85,7 +85,9 @@ struct SoftwareStoredPacketTests { ) precondition(stored == expected) let encoded = try stored.encoded() - precondition(encoded.starts(with: Data("bplist00".utf8))) + precondition(encoded.first == SoftwareStoredPacket.version) + precondition(encoded.count == SoftwareStoredPacket.headerBytes + bytes.count + + sideData.reduce(0) { $0 + SoftwareStoredPacket.sideDataHeaderBytes + $1.1.count }) let decoded = try SoftwareStoredPacket.decode(encoded) precondition(decoded == expected) let restored = try decoded.makeAVPacket() diff --git a/Sources/AetherEngine/AetherEngine+AudioDelay.swift b/Sources/AetherEngine/AetherEngine+AudioDelay.swift index d843520ca..6c6d993e5 100644 --- a/Sources/AetherEngine/AetherEngine+AudioDelay.swift +++ b/Sources/AetherEngine/AetherEngine+AudioDelay.swift @@ -180,7 +180,12 @@ extension AetherEngine { /// offset from there when they rebuild, so a re-anchor that has not reached its rebuild yet /// already carries the newest value. Only a press that lands after the rebuild has read it needs /// anything more, and that is one catch-up pass: the loop ends as soon as what was delivered is - /// what is in force, so a stepper held down converges instead of queueing a rebuild per press. + /// what is in force. Round 5 (cmcpherson274, measured): the pass is reached only by a press that + /// lands INSIDE a re-anchor, a band as wide as the re-anchor itself (54-65 ms on the simulator, + /// 87-88 ms on an Apple TV). Presses at a human cadence arrive after it and are each their own + /// complete re-anchor; a burst in one runloop turn arrives before the Task reads the value and + /// costs one. The pass starts the moment the first re-anchor returns, which on `.loopback` is + /// before the rebuilt host has published a position; `rebuildPosition` covers that window. private func reanchorForAudioDelay(_ reanchor: @escaping (_ position: Double, _ delay: Double) async -> Void) { guard Self.audioDelayRecutIsPossible(state: state, isLive: isLive, liveWindow: liveWindow) else { EngineLog.emit( diff --git a/Sources/AetherEngine/AetherEngine+AudioTap.swift b/Sources/AetherEngine/AetherEngine+AudioTap.swift index 5c18a20dd..b337ec374 100644 --- a/Sources/AetherEngine/AetherEngine+AudioTap.swift +++ b/Sources/AetherEngine/AetherEngine+AudioTap.swift @@ -130,7 +130,8 @@ extension AetherEngine { guard let masterURL = loadedURL, let yield = controller.makeYield() else { return nil } // Same headers the player's AVURLAsset sends (#119); header-enforcing IPTV origins 403 // the tap's own playlist / segment fetches without them. - let fetcher = AudioTapHLSFetcher(httpHeaders: loadedOptions.httpHeaders) + let fetcher = AudioTapHLSFetcher( + httpHeaders: loadedOptions.httpHeaders, credentialOrigin: masterURL) let decoder = AudioTapSegmentDecoder() let base = AudioTapBaseBox(masterURL) // renderedPositionMirror is fed by loadRemoteHLS's $currentTime sink (shift 0 here, so it diff --git a/Sources/AetherEngine/AetherEngine+Diagnostics.swift b/Sources/AetherEngine/AetherEngine+Diagnostics.swift index ca97196f2..875dde213 100644 --- a/Sources/AetherEngine/AetherEngine+Diagnostics.swift +++ b/Sources/AetherEngine/AetherEngine+Diagnostics.swift @@ -506,6 +506,12 @@ extension AetherEngine { native: nativeVideoSession?.demuxerBytesFetched) } + /// AE#514: the played-media ledger of whichever path owns the pump, same precedence as the byte + /// counter above. nil before a pipeline exists and on the paths with no pump to feed one. + var playedMediaLedger: PlayedMediaLedger? { + softwareHost?.playedMediaLedger ?? nativeVideoSession?.playedMediaLedger + } + /// #306: the precedence itself, as a function, so the ordering is assertable without a live /// session on either path. Software first: only one of the two exists per session, and a /// software session's counter is the one that used to be dropped. diff --git a/Sources/AetherEngine/AetherEngine+Live.swift b/Sources/AetherEngine/AetherEngine+Live.swift index 7f8101b60..903be119e 100644 --- a/Sources/AetherEngine/AetherEngine+Live.swift +++ b/Sources/AetherEngine/AetherEngine+Live.swift @@ -38,7 +38,8 @@ extension AetherEngine { scrubThumbnailExtractors.append(hit) extractor = hit.extractor } else { - extractor = FrameExtractor(reader: DataIOReader(data: source.data), formatHint: "mp4") + guard let reader = source.makeReader() else { return nil } + extractor = FrameExtractor(reader: reader, formatHint: "mp4") scrubThumbnailExtractors.append((source.segmentIndex, extractor)) while scrubThumbnailExtractors.count > 2 { let evicted = scrubThumbnailExtractors.removeFirst() @@ -395,6 +396,45 @@ extension AetherEngine { Swift.max(0, producerFloorSession - (itemSeekableStart + shift)) } + /// AE#684: what a native live item starts ON, once per item. + /// + /// Reported from the field: after an item rebuild placed mid-segment (`segment 26 + 1.91s`) sound + /// and picture were slightly apart, and together again after the next rebuild, which landed on a + /// boundary (`segment 129 + 0.03s`). The served media measures aligned on the harness (see + /// `docs/cli.md`), and nothing on macOS can observe what AVPlayer's audio renderer does with it, + /// so the line states the two facts a device capture needs beside the viewer's verdict: where in + /// which segment the item started, and where that segment's sound begins against its picture. + /// The AE#440 line beside it says whether the start was forced with `playImmediately`, which in + /// that capture was true for the one item reported out of sync and for none of the other four. + @MainActor + func noteLiveItemStart() { + guard isLive, let host = nativeHost, let session = nativeVideoSession else { return } + let itemSeconds = host.currentTime + let outputSeconds = itemSeconds + liveItemAxisOffsetSeconds + EngineLog.emit( + Self.liveItemStartAccount( + item: host.itemGeneration, itemSeconds: itemSeconds, + heads: session.liveSegmentHeads(atOutputSeconds: outputSeconds)), + category: .engine) + } + + nonisolated static func liveItemStartAccount( + item: Int, itemSeconds: Double, + heads: (index: Int, secondsIntoSegment: Double, pictureStart: Double, sound: (first: Double, last: Double)?)? + ) -> String { + let head = "[AetherEngine] #684 item #\(item) starts at its own " + + "\(String(format: "%.3f", itemSeconds))s" + guard let heads else { return head + ", in no segment the producer lists yet" } + let place = ": segment \(heads.index) + \(String(format: "%.3f", heads.secondsIntoSegment))s, " + + "whose picture opens at \(String(format: "%.3f", heads.pictureStart))s" + guard let sound = heads.sound else { return head + place + " and whose sound was not recorded" } + let lead = (heads.pictureStart - sound.first) * 1000 + return head + place + " and whose sound runs " + + "\(String(format: "%.3f", sound.first))..\(String(format: "%.3f", sound.last))s " + + "(it opens \(String(format: "%.0f", abs(lead))) ms " + + (lead >= 0 ? "before" : "after") + " the picture)" + } + /// AE#446 round 4: the three readings a rejoin's placement is argued from, on one line. /// /// They were only ever available separately, which is why an item's clock and an item's seekable @@ -452,7 +492,7 @@ extension AetherEngine { guard var w = liveWindow else { return } w.noteEdge(edgeSessionTime) w.notePlayhead(currentTime) - w.noteResidentFloor(residentLiveFloorSessionSeconds()) + w.noteResidentFloor(residentLiveFloorSessionSeconds() ?? softwareHost?.dvrResidentFloorSessionSeconds) // Sodalite#104 round 4: the cadence the playlist declares, which outranks how the source // happened to deliver. nil on the paths that serve no playlist of ours. w.noteTargetDuration(liveTargetDurationSeconds) @@ -731,7 +771,16 @@ extension AetherEngine { "[AetherEngine] live-only edge snap: clockTarget=\(String(format: "%.1f", clockTarget))", category: .engine ) + let loadGen = loadGeneration + let seekGen = currentSeekGeneration await host.seek(to: clockTarget) + // Audit CORE-7: the native host survives a native-to-native zap, so this seek can finish + // against the next channel's item, and a scrub started meanwhile owns the clock too. + guard loadGeneration == loadGen, currentSeekGeneration == seekGen else { + EngineLog.emit("[AetherEngine] live-only edge snap superseded; clock left to the successor", + category: .engine) + return + } nativeClockSeconds = clockTarget clock.currentTime = clockTarget + playlistShiftSeconds clock.sourceTime = currentTime diff --git a/Sources/AetherEngine/AetherEngine+LiveAudioDrop.swift b/Sources/AetherEngine/AetherEngine+LiveAudioDrop.swift new file mode 100644 index 000000000..45a78f200 --- /dev/null +++ b/Sources/AetherEngine/AetherEngine+LiveAudioDrop.swift @@ -0,0 +1,56 @@ +import Foundation + +/// AE#641: a live channel whose audio the bridge cannot decode plays video-only instead of stalling. +/// +/// The load-time cascade already ends in video-only when no pipeline can be BUILT (AE#462). A bridge +/// that is built and then decodes nothing reaches no such tail: on VOD the first segment cut fails +/// and the session surfaces `audioBridgeProducedNoOutput` (AE#396), but a live FLAC bridge builds its +/// sample entry from the encoder's extradata, so segments keep being cut with an audio track that +/// never carries a sample. AVPlayer presents the first picture and waits on the audio for as long as +/// the session lasts, with nothing surfaced. The reporter's channel was DTS-HD labelled as MPEG audio. +/// +/// Nothing inside the session reaches it: a producer rebuild hands the same decoder the same bytes. +/// So the session is rebuilt with that stream marked undecodable, the cascade takes its video-only +/// tail, and `audioDelivery` reads `.droppedNoPipeline`, the fact a host ladder demotes on. +extension AetherEngine { + + /// What a load keeps of the undecodable streams: the session's own rebuilds keep them, which is + /// what keeps them video-only, and a host load is a new source and starts clean. + nonisolated static func undecodableAudioStreamIndicesAcrossLoad( + _ current: Set, sessionPreservingReload: Bool + ) -> Set { + sessionPreservingReload ? current : [] + } + + @MainActor + func dropUndecodableLiveAudio(streamIndex: Int32, bridgeSummary: String) async { + guard undecodableLiveAudioStreamIndices.insert(streamIndex).inserted else { + EngineLog.emit( + "[AetherEngine] AE#641 stream \(streamIndex) is already marked undecodable " + + "(\(bridgeSummary)); the session is already video-only for it", + category: .engine + ) + return + } + EngineLog.emit( + "[AetherEngine] AE#641 the live audio bridge decoded nothing from stream \(streamIndex) " + + "(\(bridgeSummary)); rebuilding the session video-only instead of serving an audio " + + "track that will never be filled", + category: .engine + ) + do { + try await startTakeoverRebuild { _ in }.value + EngineLog.emit( + "[AetherEngine] AE#641 rebuilt video-only, audio delivery = \(audioDelivery.rawValue)", + category: .engine) + } catch is CancellationError { + EngineLog.emit( + "[AetherEngine] AE#641 video-only rebuild superseded; nothing to do", category: .engine) + } catch { + EngineLog.emit( + "[AetherEngine] AE#641 video-only rebuild refused (\(error)); the session keeps its " + + "silent audio track", + category: .engine) + } + } +} diff --git a/Sources/AetherEngine/AetherEngine+LiveSubtitleRenditions.swift b/Sources/AetherEngine/AetherEngine+LiveSubtitleRenditions.swift index bbc86702d..71916cdd2 100644 --- a/Sources/AetherEngine/AetherEngine+LiveSubtitleRenditions.swift +++ b/Sources/AetherEngine/AetherEngine+LiveSubtitleRenditions.swift @@ -22,6 +22,15 @@ extension AetherEngine { id >= liveSubtitleRenditionTrackIDBase && id < liveSubtitleRenditionTrackIDBase + 1_000 } + /// Audit Vcred-101: the rendition playlist and every segment it lists are URIs the master named, + /// on any host or scheme, so the host's credentials go only to the ingest's own origin, as on the + /// ingest's other fetches. No live ingest reader, no anchor, no credentials. + nonisolated static func liveSubtitleRenditionCredentials(headers: [String: String], source: IOReader?) + -> CredentialScope + { + CredentialScope(headers: headers, anchor: (source as? LiveIngestSourceInfo)?.credentialOrigin) + } + /// Publish the renditions the live ingest resolved. Called once per load, before playback settles; /// the master's declaration is proof enough that the track exists, so unlike the caption tap there /// is nothing to wait for. @@ -95,7 +104,8 @@ extension AetherEngine { // The state line is worth having but not every two seconds: LogTap is a ring buffer, and a // line that repeats 30 times a minute pushes out everything a reader came for. var pollsSinceStateLine = 0 - let headers = loadedOptions.httpHeaders + let credentials = Self.liveSubtitleRenditionCredentials( + headers: loadedOptions.httpHeaders, source: customReader) while !Task.isCancelled { guard activeSubtitleTrackIndex == trackID else { return } // The source axis can be re-anchored under a running session (a producer seam republishes @@ -111,9 +121,12 @@ extension AetherEngine { } var pollInterval = 2.0 do { - let text = try await Self.fetchText(rendition.playlistURL, headers: headers) - guard case .media(let media) = try HLSPlaylistParser.parse(text) else { return } - pollInterval = max(1, media.targetDuration) + // Audit FEA-104: the fetch and both parses run off the main actor; only the merge and + // the publish below are back on it. + guard let media = try await Self.fetchLiveSubtitleRenditionPlaylist( + rendition.playlistURL, headers: credentials.headers(for: rendition.playlistURL)) + else { return } + pollInterval = Self.liveSubtitlePollInterval(targetDuration: media.targetDuration) // Anchor the work at the playhead, not at the start of the playlist. A rendition // playlist is not a handful of segments: MDR publishes its whole two hour DVR window, // 3600 entries, and walking it from the front means thousands of fetches for content @@ -134,8 +147,8 @@ extension AetherEngine { let url = HLSPlaylistParser.resolve(uri: segment.uri, against: rendition.playlistURL) else { continue } seen.insert(segment.uri) - guard let body = try? await Self.fetchText(url, headers: headers), - let parsed = WebVTTSegmentParser.parse(body) else { continue } + guard let parsed = await Self.fetchLiveSubtitleRenditionSegment( + url, headers: credentials.headers(for: url)) else { continue } guard activeSubtitleTrackIndex == trackID else { return } let fresh = WebVTTSegmentParser.cues(from: parsed, segmentWallStart: wallStart, segmentDuration: segment.duration, @@ -172,28 +185,76 @@ extension AetherEngine { } } + /// Audit FEA-102: a week-long TARGETDURATION would put the loop to sleep for a week and end the + /// subtitles; past about 2e10 s it trapped `UInt64(_:)` outright. + nonisolated static func liveSubtitlePollInterval(targetDuration: Double) -> Double { + guard !targetDuration.isNaN else { return 1 } + return min(max(1, targetDuration), 30) + } + /// Keep the published array bounded: a channel left running for hours would otherwise carry every /// line it ever showed, and #271 is the standing reminder that this array is paid for per publish. private func pruned(_ cues: [SubtitleCue]) -> [SubtitleCue] { let horizon = clock.currentTime - (loadedOptions.dvrWindowSeconds ?? 600) - 60 - guard horizon > 0 else { return cues } - return cues.filter { $0.endTime >= horizon } + guard horizon > 0 else { return Self.capLiveSubtitleCues(cues) } + return Self.capLiveSubtitleCues(cues.filter { $0.endTime >= horizon }) + } + + /// Audit SUB-106: the time horizon above is the normal bound; this one is for an origin whose cues + /// never age out of it. The newest cues are the ones that were appended last. + nonisolated static let maxLiveSubtitleCues = 2000 + + nonisolated static func capLiveSubtitleCues(_ cues: [SubtitleCue]) -> [SubtitleCue] { + cues.count > maxLiveSubtitleCues ? Array(cues.suffix(maxLiveSubtitleCues)) : cues + } + + /// Audit FEA-107: this session was `.default`, a 7-day resource timeout and a disk cache, so one + /// endless body grew until jetsam. Ephemeral and finite, like the ingest readers' own. + nonisolated static func liveSubtitleRenditionSessionConfiguration() -> URLSessionConfiguration { + let configuration = URLSessionConfiguration.ephemeral + configuration.timeoutIntervalForRequest = 10 + configuration.timeoutIntervalForResource = 30 + return configuration } /// `URLSession.shared` cannot carry a delegate, so this was the one engine fetch a host's trust /// decision could never reach. Owned and process-wide for the same reason the carriage probe's /// is: an uninvalidated session outlives its caller. - private static let renditionSession = URLSession( - configuration: .default, delegate: EngineTLS.sessionDelegate, delegateQueue: nil) + nonisolated static let renditionSession = URLSession( + configuration: liveSubtitleRenditionSessionConfiguration(), + delegate: EngineTLS.sessionDelegate, delegateQueue: nil) - private static func fetchText(_ url: URL, headers: [String: String]) async throws -> String { + /// One poll of a rendition playlist, fetched under a size cap with its status checked and parsed + /// off the main actor (audit FEA-104, FEA-107). nil when the URL answers with a master playlist. + nonisolated static func fetchLiveSubtitleRenditionPlaylist( + _ url: URL, headers: [String: String], session: URLSession = renditionSession + ) async throws -> HLSMediaPlaylist? { var request = URLRequest(url: url) for (key, value) in headers { request.setValue(value, forHTTPHeaderField: key) } - let (data, _) = try await renditionSession.data(for: request) + let (data, response) = try await BoundedPlaylistFetch.data( + for: request, session: session, limit: HLSLiveIngestReader.maximumPlaylistBytes) + let status = (response as? HTTPURLResponse)?.statusCode ?? -1 + guard (200..<300).contains(status) else { throw HLSIngestError.playlistUnreachable(status: status) } guard let text = String(data: data, encoding: .utf8) else { throw HLSIngestError.playlistInvalid(reason: "rendition payload is not UTF-8") } - return text + guard case .media(let media) = try HLSPlaylistParser.parse(text) else { return nil } + return media + } + + /// One WebVTT segment: nil for anything that is not a parsable body within the cap (audit SUB-106, + /// FEA-107). A segment that fails is not retried, as before. + nonisolated static func fetchLiveSubtitleRenditionSegment( + _ url: URL, headers: [String: String], session: URLSession = renditionSession + ) async -> WebVTTSegment? { + var request = URLRequest(url: url) + for (key, value) in headers { request.setValue(value, forHTTPHeaderField: key) } + guard let (data, response) = try? await BoundedFetch.data( + for: request, session: session, limit: BoundedFetch.webVTTSegmentLimit), + let status = (response as? HTTPURLResponse)?.statusCode, (200..<300).contains(status), + let text = String(data: data, encoding: .utf8) + else { return nil } + return WebVTTSegmentParser.parse(text) } /// Drop the renditions and stop any fetch. Called from the session teardown paths that already diff --git a/Sources/AetherEngine/AetherEngine+Loading.swift b/Sources/AetherEngine/AetherEngine+Loading.swift index f79aae82e..eb49233ed 100644 --- a/Sources/AetherEngine/AetherEngine+Loading.swift +++ b/Sources/AetherEngine/AetherEngine+Loading.swift @@ -28,7 +28,9 @@ extension AetherEngine { // just left, and the window's edge is a running maximum a single wrong sample latches. if liveItemPlacementPending { return } // Newest seam at or before the raw clock wins: activates seams on forward play, re-applies pre-seam shift on backward DVR seeks. - if let active = presentationAxis.shiftSeconds(atItemSeconds: value) { + // Audit PERF-106: published on the engine, so an unchanged write would fire its + // objectWillChange on every 10 Hz tick (the reason `clock` exists). + if let active = presentationAxis.shiftSeconds(atItemSeconds: value), active != playlistShiftSeconds { playlistShiftSeconds = active } if pendingRecoverySeekClockTarget == nil { @@ -90,7 +92,8 @@ extension AetherEngine { /// hold-vs-settle decision is unit-testable without driving a live AVPlayer. func applySeekFinalizeSourceTime(target: Double, bufferingTowardTarget: Bool) { if Self.seekLandingSettlesToTarget(bufferingTowardTarget: bufferingTowardTarget) { - clock.sourceTime = target + // AE#616: zero except on a remote-HLS bypass whose item time leads its frames. + clock.sourceTime = max(0, target - remoteHLSItemOffset) } } @@ -217,6 +220,7 @@ extension AetherEngine { // it can play, nothing it reports is a reading of where the session is; see // `liveItemPlacementPending`. if ready { + self.noteLiveItemStart() // The placement is spent here, on every path rather than only on the one that // replays the stashed seek: a pre-ready seek can be superseded by a host scrub // (latest-wins), and an arm left standing would be inherited by whatever item the @@ -317,7 +321,8 @@ extension AetherEngine { /// which is the point, since re-creating it is what breaks AVKit's MediaRemote registration. func makeNativeHost() -> NativeAVPlayerHost { #if os(tvOS) || os(iOS) - return NativeAVPlayerHost(ownsNowPlayingSession: ownsVideoNowPlayingSession) + return NativeAVPlayerHost(ownsNowPlayingSession: Self.ownsNowPlaying( + hostOptIn: ownsVideoNowPlayingSession, role: loadedOptions.sharedOutputRole)) #else return NativeAVPlayerHost() #endif @@ -337,10 +342,15 @@ extension AetherEngine { #endif } - func loadRemoteHLS(url: URL, options: LoadOptions, startPosition: Double? = nil) async throws { + func loadRemoteHLS( + url: URL, options: LoadOptions, startPosition: Double? = nil, generation: UInt64? = nil + ) async throws { + // Audit CORE-6: the generation of the load() that routed here, not a fresh read of it, so a + // stale caller cannot adopt its successor's generation. + if let generation { try checkLoadCurrent(generation) } playbackBackend = .native // #168 follow-up: detect a superseding load()/stop() between the carriage verdict and the reroute. - let bypassGeneration = loadGeneration + let bypassGeneration = generation ?? loadGeneration let host: NativeAVPlayerHost if let existing = nativeHost { @@ -362,6 +372,7 @@ extension AetherEngine { applyDesiredRate(to: host) // No loopback producer; playhead is the raw AVPlayer clock. Shift stays 0. self.playlistShiftSeconds = 0 + detachRemoteHLSCueClock() self.setPresentationAxis(PresentationAxisMap()) if currentAVPlayer !== host.avPlayer { self.currentAVPlayer = host.avPlayer @@ -381,10 +392,12 @@ extension AetherEngine { .store(in: &nativeCancellables) host.$renderedTime .sink { [weak self] value in - self?.clock.sourceTime = value + guard let self else { return } + // AE#616: item time, less what the injected renditions measured it leads the picture by. + self.clock.sourceTime = max(0, value - self.remoteHLSItemOffset) // Feed the playhead mirror the remote-HLS audio tap (#95) reads off its ingest task; // shift 0 on this path, so the rendered position is the source-PTS playhead. - self?.renderedPositionMirror.set(value) + self.renderedPositionMirror.set(value) } .store(in: &nativeCancellables) // #168: mirror the item's parsed dynamic range into the published format AND program the panel. @@ -406,6 +419,29 @@ extension AetherEngine { self.applyRemoteHLSDisplayCriteria(format: fmt, options: options) } .store(in: &nativeCancellables) + // What was DELIVERED, which on a capped transcode is not what the host's library holds: without it + // a stats panel fell back to the original file's 3840x2160 for a 1280x720 stream. Its own sink, + // because a later read can refine the description without changing the dynamic range. + // dropFirst on both: a reused host replays the outgoing item's reading on subscribe, before load resets it. + host.$detectedVideoDescription + .dropFirst() + .compactMap { $0 } + .sink { [weak self] video in + guard let self else { return } + self.publishRemoteHLSVideoDescription(video) + } + .store(in: &nativeCancellables) + // No probe lists this route's audio, so AVPlayer's own tracks are the list. Informational: AVPlayer + // owns the audio selection here, and `selectAudioTrack` refuses the route rather than reload it. + host.$detectedAudioTracks + .dropFirst() + .sink { [weak self] readings in + guard let self else { return } + let (tracks, active) = RemoteHLSStreamDescription.audioTracks(readings) + self.audioTracks = tracks + self.activeAudioTrackIndex = active + } + .store(in: &nativeCancellables) // #168 follow-up: an advertised video rendition that never builds an item track means HEVC carried // in MPEG-TS segments, which AVFoundation's HLS demuxer does not support (audio-only, black). The // loopback ingest remuxes TS to fMP4 and plays the same stream, so reroute there transparently. @@ -539,7 +575,11 @@ extension AetherEngine { // #334: the ceiling on silence this path never had. AVPlayer's "gave up" covers an // origin that stops answering; it does not cover one that answers everything while // AVFoundation builds no track, where nothing terminal is ever published. - readinessDeadline: RemoteHLSReadinessDeadline.defaultBudgetSeconds)) + readinessDeadline: RemoteHLSReadinessDeadline.defaultBudgetSeconds, + // No probe lists this route's audio, so the item's own tracks are the list. + readsBackAudioTracks: true)) + + attachRemoteHLSCueClock(host: host, expectedGeneration: bypassGeneration) // AE#154: surface the item's legible AVMediaSelectionGroup as `subtitleTracks` so hosts with // their own picker see the external WebVTT renditions AVPlayer renders on this bypass. @@ -553,7 +593,42 @@ extension AetherEngine { host.play() } startMemoryProbe() - // No startLiveTelemetrySampler: all sampler counters read the loopback pipeline (demuxer / producer / cache / server), none of which exists on this bypass. + // The sampler reads AVPlayer's access log on this route: both bitrates, network rate and transfer, + // dropped frames and forward buffer. The loopback counters (producer, muxer, server) read zero. + startLiveTelemetrySampler() + } + + /// AE#616: on this bypass `sourceTime` would otherwise be item time, which an origin that restarts + /// its transcode at the keyframe before a slot puts ahead of the picture. The engine wrote the + /// injected renditions, so it can match what AVPlayer presents back to the cue and read the offset. + /// Without injected renditions nothing is measurable and `sourceTime` stays item time. + private func attachRemoteHLSCueClock(host: NativeAVPlayerHost, expectedGeneration: UInt64) { + detachRemoteHLSCueClock() + // Item time until a line says otherwise, and for the whole session without injected renditions. + clock.sourceTimeFollowsPicture = false + guard let provider = remoteHLSSubtitleProxy?.provider, + let item = host.currentPlayerItem else { return } + remoteHLSCueClock = RemoteHLSCueClockObserver( + item: item, provider: provider, + onOffset: { [weak self] offset in + guard let self, self.loadGeneration == expectedGeneration else { return } + self.remoteHLSItemOffset = offset + if let rendered = self.nativeHost?.renderedTime { + self.clock.sourceTime = max(0, rendered - offset) + } + self.clock.sourceTimeFollowsPicture = true + }, + onTimeJump: { [weak self] in + guard let self, self.loadGeneration == expectedGeneration else { return } + self.clock.sourceTimeFollowsPicture = false + }) + } + + func detachRemoteHLSCueClock() { + remoteHLSCueClock?.detach() + remoteHLSCueClock = nil + remoteHLSItemOffset = 0 + clock.sourceTimeFollowsPicture = true } /// Stand a loopback origin in front of the remote master and return the URL AVPlayer should open. @@ -618,10 +693,12 @@ extension AetherEngine { return nil } remoteHLSSubtitleProxy = prepared + // Audit NAT-2: the NAMEs the served master carries, which the selection and the legible-list + // filter match against, not the names the tracks asked for (the rewriter disambiguates and + // escapes them). injectedSubtitleRenditionNames = prepared.servesSubtitleRenditions ? Dictionary( - uniqueKeysWithValues: zip(tracks.map(\.externalID), - RemoteHLSSubtitleProvider.renditions(for: tracks).map(\.name))) + uniqueKeysWithValues: zip(tracks.map(\.externalID), prepared.renditionNames)) : [:] #if os(iOS) // #86 / #227: a receiver cannot reach 127.0.0.1. Mounting while already AirPlaying has to hand out @@ -729,7 +806,11 @@ extension AetherEngine { // served), the advert rides along for the seal log only. Both weak, same reason as above. let liveClosedCadenceObservation: (@Sendable () -> Double?)? let liveUpstreamSegmentDurationObservation: (@Sendable () -> Double?)? + let liveJoinBacklogObservation: (@Sendable () -> Double?)? + let liveJoinSpentObservation: (@Sendable () -> Bool?)? if let liveIngest { + liveJoinBacklogObservation = { [weak liveIngest] in liveIngest?.joinBacklogSeconds } + liveJoinSpentObservation = { [weak liveIngest] in liveIngest?.joinIsSpent } liveClosedCadenceObservation = { [weak liveIngest] in liveIngest?.closedLiveCadenceSeconds } liveUpstreamSegmentDurationObservation = { [weak liveIngest] in liveIngest?.upstreamSegmentDurationSeconds @@ -737,6 +818,8 @@ extension AetherEngine { } else { liveClosedCadenceObservation = nil liveUpstreamSegmentDurationObservation = nil + liveJoinBacklogObservation = nil + liveJoinSpentObservation = nil } let upstreamSelfReportedTargetDuration = liveIngest?.upstreamTargetDuration // #199: in-engine reopen transport for live ingest sessions. Only HLSLiveIngestReader main @@ -769,6 +852,7 @@ extension AetherEngine { matchContentEnabled: matchContentEnabled, panelIsInHDRMode: panelIsInHDRMode, audioSourceStreamIndexOverride: audioSourceStreamIndex, + undecodableAudioStreamIndices: undecodableLiveAudioStreamIndices, audioBridgeMode: audioBridgeMode, objectAudioRendering: loadedOptions.objectAudioRendering, isLiveSession: isLive, @@ -780,6 +864,8 @@ extension AetherEngine { liveCadenceObservation: liveCadenceObservation, liveClosedCadenceObservation: liveClosedCadenceObservation, liveUpstreamSegmentDurationObservation: liveUpstreamSegmentDurationObservation, + liveJoinBacklogObservation: liveJoinBacklogObservation, + liveJoinSpentObservation: liveJoinSpentObservation, upstreamSelfReportedTargetDuration: upstreamSelfReportedTargetDuration, preopenedDemuxer: preopenedDemuxer, sourceReopenableByURL: !isCustomSource, @@ -800,13 +886,23 @@ extension AetherEngine { // #240: the pump claims the source link through this gate while it is fetching, so the // subtitle side readers can stay out of its way. Set before start(). session.sideReaderLinkGate = sideReaderLinkGate + // AE#682: set before start(), where the session decides whether it can serve the rendition. + // A custom-IO source is not reopenable by URL, so the side reader gets a clone of its own; + // a reader that cannot clone leaves the rendition absent. + if loadedOptions.serveIFramePlaylist { + session.requestIFramePlaylist() + if isCustomSource, let clone = customReader?.makeIndependentReader() { + session.customIFrameReader = (reader: clone, formatHint: customFormatHint) + } + } // #260: an observer installed before load has to reach this session's producers too. session.setNativeVideoFrameTimeObserver(nativeVideoFrameTimeObserver) + // Audit Vcore-101: every hop below is dropped once this session has ended (`hop(for:)`). session.onFirstHDR10PlusDetected = { [weak self] in - Task { @MainActor in self?.handleHDR10PlusDetected() } + self?.hop(for: generation) { [weak self] in self?.handleHDR10PlusDetected() } } session.onPlaylistShiftChanged = { [weak self] seconds, seamItemSeconds in - Task { @MainActor in + self?.hop(for: generation) { [weak self] in guard let self = self else { return } let prevShift = self.playlistShiftSeconds let delta = seconds - prevShift @@ -836,7 +932,7 @@ extension AetherEngine { // Fold with the shift in effect AT the raw clock, not with the newest one: while old-epoch buffer // is still on screen those differ, and the picture is what the clock has to describe. let activeShift = self.presentationAxis.shiftSeconds(atItemSeconds: self.nativeClockSeconds) ?? seconds - self.playlistShiftSeconds = activeShift + if activeShift != self.playlistShiftSeconds { self.playlistShiftSeconds = activeShift } // The cache did not move, but the fold onto the display axis did. Re-publish the band // from the raw spans so it does not carry the retired epoch's offset until the next // segment lands (AE#468 follow-up). @@ -870,7 +966,7 @@ extension AetherEngine { } } session.onSeekStateChanged = { [weak self] inFlight, playlistTime in - Task { @MainActor in + self?.hop(for: generation) { [weak self] in guard let self = self else { return } // Fold playlist-axis segment time onto the published display axis (#38); the origin keeps a disc // scrub target 0-based like currentTime (0 off disc). nil clears without disturbing the last value. @@ -886,7 +982,7 @@ extension AetherEngine { } } session.onNetworkPhaseChanged = { [weak self] phase in - Task { @MainActor in self?.setReaderNetworkPhase(phase) } + self?.hop(for: generation) { [weak self] in self?.setReaderNetworkPhase(phase) } } // #65: let the producer read AVPlayer's real position off-main when it re-anchors on a backpressure wedge. session.currentPlaybackPositionProvider = { [renderedPositionMirror] in renderedPositionMirror.get() } @@ -909,7 +1005,7 @@ extension AetherEngine { // pipeline (the effect a manual back-out had). Opens the spurious-pause window too, since // the nudge can bounce the transport state. session.onConsumerReengageNeeded = { [weak self] position in - Task { @MainActor [weak self] in + self?.hop(for: generation) { [weak self] in guard let self else { return } self.reengageStalledConsumer(position: position, trigger: "wedge re-anchor") // #93 startup: a loader that died BEFORE the first frame never posts @@ -936,7 +1032,7 @@ extension AetherEngine { } } session.onPlaylistShiftRebased = { [weak self] seconds, seamOutputSeconds in - Task { @MainActor in + self?.hop(for: generation) { [weak self] in guard let self = self else { return } // Program boundary: producer rebased but AVPlayer is still rendering old program (buffer + holdback). Record the seam so $currentTime resolves the active shift from history, keeping currentTime/sourceTime behind what is on screen. Backward DVR seeks re-apply the pre-seam shift. Seams append in output-timeline order (continuation dts is monotonic). var map = self.presentationAxis @@ -976,6 +1072,45 @@ extension AetherEngine { self.liveSourceReset.send() } } + // AE#627: the first join found no entry point the native route can open. Reopening joins the + // same bitstream, so the session goes to the software path, or straight to the host when + // that rung is not on offer, instead of spending three 15 s reopen cycles first. + session.onLiveJoinWithoutEntryPoint = { [weak self, weak session] in + Task { @MainActor in + guard let self, let session, self.nativeVideoSession === session else { return } + let request = SoftwarePathEscalation.Request( + domain: SoftwarePathEscalation.liveJoinErrorDomain, + code: 0, + message: "live join found no entry point the native route can open", + positionSeconds: 0 + ) + let offered = SoftwarePathEscalation.shouldEscalate( + errorDomain: request.domain, + availability: SoftwarePathEscalation.Availability( + alreadyEscalated: self.softwarePathEscalationBudget.isSpent, + preferredDecodePath: self.loadedOptions.preferredDecodePath, + nativeRemoteHLS: self.loadedOptions.nativeRemoteHLS, + hostAllowsEscalation: self.loadedOptions.escalatesToSoftwarePath)) + guard offered else { + EngineLog.emit( + "[AetherEngine] AE#627 software path not on offer for this session; " + + "handing the join failure to the host", + category: .session + ) + session.giveUpLiveJoinWithoutEntryPoint() + return + } + await self.escalateToSoftwarePath(request, expectedGeneration: generation) + } + } + // AE#641: the live bridge decoded nothing, so the served media carries an audio track that + // will never be filled and AVPlayer waits on it forever. The session is rebuilt video-only. + session.onLiveAudioDecodesNothing = { [weak self, weak session] streamIndex, summary in + Task { @MainActor in + guard let self, let session, self.nativeVideoSession === session else { return } + await self.dropUndecodableLiveAudio(streamIndex: streamIndex, bridgeSummary: summary) + } + } // #126: zero-progress VOD pump death (readError before any packet/segment), and #169: // mid-session readError after the revive cap. Without this the host sees // isPlayable=true / a stalled item and waits until its own timeout. @@ -1556,7 +1691,6 @@ extension AetherEngine { .sink { [weak self, weak host] count in guard let self, let host else { return } let clockAtFailure = host.renderedTime - let diedUnderPause = host.endFailureFollowedPause self.itemDeathConfirmTask?.cancel() self.itemDeathConfirmTask = Task { @MainActor [weak self, weak host] in try? await Task.sleep( @@ -1576,30 +1710,37 @@ extension AetherEngine { // AE#561: a frozen position across three reloads is the reload answering the // same bytes three times. Offer the source to the engine's own decoder before // the session is left dead. - await self.escalateToSoftwarePath( - SoftwarePathEscalation.Request( - domain: SoftwarePathEscalation.mediaErrorDomain, - code: 0, - message: "item death at a frozen position, revive budget exhausted", - positionSeconds: position.isFinite ? max(0, position) : 0 - ) + // + // Its own task (audit CORE-1): the rebuild's load() cancels THIS task in its + // prologue, and a rebuild left running in a cancelled task turns every + // `try? await Task.sleep` poll on its way (the panel-switch wait) into a hot + // spin on the main actor. Supersession is answered by the load generation. + let request = SoftwarePathEscalation.Request( + domain: SoftwarePathEscalation.mediaErrorDomain, + code: 0, + message: "item death at a frozen position, revive budget exhausted", + positionSeconds: position.isFinite ? max(0, position) : 0 ) + // AE#629: a host that declined the rung asked for exactly this failure. Left + // unsaid, the session sits dead, which is what 7.8.1 did here too. + guard self.loadedOptions.escalatesToSoftwarePath else { + EngineLog.emit( + "[AetherEngine] #629 the host declined the software-path rung; " + + "surfacing the item death", category: .engine) + self.publishError(Self.absorbedFailure(request)) + return + } + self.hop(for: generation) { [weak self] in + await self?.escalateToSoftwarePath(request, expectedGeneration: generation) + } return } - // Decided now rather than when the failure was counted: the viewer may have - // pressed Play or Pause while the death was being confirmed. - let resumesPlaying = NativeAVPlayerHost.itemDeathReloadResumesPlaying( - diedUnderPause: diedUnderPause, - commandSinceFailure: host.transportCommandSinceEndFailure, - transportRolling: host.rate != 0) EngineLog.emit( "[AetherEngine] #93 item death (failedToPlayToEndTime) at " + "\(String(format: "%.2f", position))s; reloading item through stage-2 " - + "recovery (attempt \(self.itemDeathReviveGate.attempts), pause guard bypassed" - + (resumesPlaying ? ")" : ", keeping the viewer's pause)"), + + "recovery (attempt \(self.itemDeathReviveGate.attempts), pause guard bypassed)", category: .engine) - self.reloadStalledConsumerItem( - position: position, allowPausedConsumer: true, resumesPlaying: resumesPlaying) + self.reloadStalledConsumerItem(position: position, allowPausedConsumer: true) } } .store(in: &nativeCancellables) @@ -1609,7 +1750,9 @@ extension AetherEngine { host.$pendingDisplayRejection .compactMap { $0 } .sink { [weak self] rejection in - Task { @MainActor [weak self] in self?.fallBackToMediaPlaylist(rejection) } + self?.hop(for: generation) { [weak self] in + self?.fallBackToMediaPlaylist(rejection, expectedGeneration: generation) + } } .store(in: &nativeCancellables) @@ -1621,17 +1764,21 @@ extension AetherEngine { let escalationBudget = softwarePathEscalationBudget let escalationPreferred = loadedOptions.preferredDecodePath let escalationRemoteHLS = loadedOptions.nativeRemoteHLS + let escalationAllowed = loadedOptions.escalatesToSoftwarePath host.softwarePathAvailability = { SoftwarePathEscalation.Availability( alreadyEscalated: escalationBudget.isSpent, preferredDecodePath: escalationPreferred, - nativeRemoteHLS: escalationRemoteHLS + nativeRemoteHLS: escalationRemoteHLS, + hostAllowsEscalation: escalationAllowed ) } host.$pendingSoftwarePathEscalation .compactMap { $0 } .sink { [weak self] request in - Task { @MainActor [weak self] in await self?.escalateToSoftwarePath(request) } + self?.hop(for: generation) { [weak self] in + await self?.escalateToSoftwarePath(request, expectedGeneration: generation) + } } .store(in: &nativeCancellables) @@ -1673,6 +1820,14 @@ extension AetherEngine { // report the same two channels. audioIsAtmosStreamCopy: nativeVideoSession?.audioIsAtmosStreamCopy == true)) forceNativeLegibleDeselectedUntilHostSelects() + // Sodalite#175: from the session's own pick, since a track-switch reload has no active index yet. + let sessionAudioPick = nativeVideoSession.map(\.activeAudioSourceStreamIndex).flatMap { $0 >= 0 ? Int($0) : nil } + let audioPick = sessionAudioPick ?? audioSourceStreamIndex.flatMap { Int(exactly: $0) } ?? activeAudioTrackIndex + let pickTracks = nativeVideoSession.map(\.companionAudioTracks).flatMap { $0.isEmpty ? nil : $0 } ?? audioTracks + SharedOutputCoordinator.shared.noteSourceChannels( + audioPick.flatMap { index in pickTracks.first { $0.id == index }?.channels } + .flatMap { $0 > 0 ? $0 : nil }, + for: ObjectIdentifier(self)) // AE#458: what AVFoundation makes of the audio rendition this load just served, which is the // half of the exchange no log has ever carried. logAudibleReadback(host: host) @@ -1700,8 +1855,10 @@ extension AetherEngine { } else { nil } + SharedOutputCoordinator.shared.noteSourceChannels(sourceChannels, for: ObjectIdentifier(self)) + let preferred = SharedOutputCoordinator.shared.preferredSourceChannels ?? sourceChannels await enqueueAudioSessionTransition { - AetherEngine.applyRendererAudioSession(sourceChannels: sourceChannels) + AetherEngine.applyRendererAudioSession(sourceChannels: preferred) }.value #endif } @@ -1770,6 +1927,12 @@ extension AetherEngine { host.onFirstHDR10PlusDetected = { [weak self] in Task { @MainActor in self?.handleHDR10PlusDetected() } } + host.onDecodedVideoFormat = { [weak self, weak host] format in + Task { @MainActor in + guard let self, let host, self.softwareHost === host else { return } + self.decodedVideoFormat = format + } + } // SW host provides session-relative edge on each tick; publishLiveWindow is a no-op when liveWindow is nil. host.onLiveEdge = { [weak self] edge in self?.publishLiveWindow(edgeSessionTime: edge) @@ -1791,7 +1954,7 @@ extension AetherEngine { guard let self, let host else { return } host.updateSubtitleCompositor( cues: self.softwarePiPSubtitleCues(primary: primary, secondary: secondary), - enabled: self.pictureInPictureActive) + enabled: self.pictureInPictureActive, delaySeconds: self.softwareSubtitleDelaySeconds) } .store(in: &softwareCancellables) // #131: no demuxable CC track on the SW path either: arm an A53 tap fed by decoded-frame @@ -1871,7 +2034,7 @@ extension AetherEngine { let declaredDuration = loadedOptions.declaredDurationSeconds // Built on the main actor, captured into the detach: surfaces source stall/reconnect to playbackPhase (#85). let networkPhaseSink: @Sendable (ReaderNetworkPhase) -> Void = { [weak self] phase in - Task { @MainActor in self?.setReaderNetworkPhase(phase) } + self?.hop(for: generation) { [weak self] in self?.setReaderNetworkPhase(phase) } // audit Vcore-101 } if loadGeneration == generation { recordStartupCheckpoint(.sessionConstructed) } // #361 let forwardBufferSegments = loadedOptions.forwardBufferSegments @@ -1949,7 +2112,7 @@ extension AetherEngine { let declaredDuration = loadedOptions.declaredDurationSeconds // Built on the main actor, captured into the detach: surfaces source stall/reconnect to playbackPhase (#85). let networkPhaseSink: @Sendable (ReaderNetworkPhase) -> Void = { [weak self] phase in - Task { @MainActor in self?.setReaderNetworkPhase(phase) } + self?.hop(for: generation) { [weak self] in self?.setReaderNetworkPhase(phase) } // audit Vcore-101 } if loadGeneration == generation { recordStartupCheckpoint(.sessionConstructed) } // #361 try await Task.detached(priority: .userInitiated) { @@ -1985,7 +2148,9 @@ extension AetherEngine { // Reuse the persistent host (MPNowPlayingSession survives across tracks). host.load() swaps the item via replaceCurrentItem. await activateRendererAudioSession() try checkLoadCurrent(generation) - let host = audioAVPlayerHost ?? AudioAVPlayerHost() + let ownsNowPlaying = Self.ownsNowPlaying(hostOptIn: true, role: loadedOptions.sharedOutputRole) + let host = audioAVPlayerHost ?? AudioAVPlayerHost(ownsNowPlaying: ownsNowPlaying) + host.ownsNowPlaying = ownsNowPlaying self.audioAVPlayerHost = host applyDesiredVolume(to: host) self.audioAVPlayerActive = true @@ -2070,13 +2235,17 @@ extension AetherEngine { /// This path publishes its failure and used to return silently either way, so a caller could /// not tell a rebuilt session from a dead one; `reloadAtCurrentPosition(applying:)` is built on /// exactly that distinction. Callers that only drive UI keep discarding it. + /// + /// `resumePlaying` is the transport the rebuild comes back in; nil reads the session's own + /// (`sessionRebuildResumesPlaying`, AE#464 round 2), which is what an audio or title pick wants. @discardableResult func reloadWithAudioOverride( url: URL, audioStreamIndex: Int32?, expectedGeneration: UInt64, discTitleIDOverride: Int? = nil, - resumeOverride: Double? = nil + resumeOverride: Double? = nil, + resumePlaying: Bool? = nil ) async -> Error? { // Liveness guard: a stop()/load() between scheduling and here would resurrect a dismissed session or kill the successor. Generation captured at schedule time; both stop() and load() invalidate it. guard loadGeneration == expectedGeneration, loadedURL != nil else { @@ -2149,13 +2318,16 @@ extension AetherEngine { category: .engine ) + // Audit LIF-102: read before `.loading`, which is the state this rebuild is about to hide the + // session's transport behind. The audio pick never wrote it into `loadedOptions.autoplay` + // (that is the mount flag there), so the rebuild used to end in an unconditional `play()`. + let resumesPlaying = resumePlaying ?? sessionRebuildResumesPlaying state = .loading // AE#464 round 2: this branch reaches `loadSoftware` / `loadNative` rather than `load`, so it // parks its own rebuild position for anything that stacks behind it. Round 3 parks the - // transport beside it; this branch reads `loadedOptions` field by field, and the caller has - // already written the session's own transport into it. + // transport beside it. positionUnderReconstruction = resumeAt - transportIntentUnderReconstruction = loadedOptions.autoplay + transportIntentUnderReconstruction = resumesPlaying let previousAudioIndex = activeAudioTrackIndex // Snapshot before stopInternal wipes state. Must reload on the same backend: loadNative on a SW-routed AV1 source throws unsupportedCodec (HLSVideoEngine only accepts HEVC / H.264 / VP9 / probed-AV1). let wasOnSoftwarePath = (playbackBackend == .software) @@ -2189,9 +2361,25 @@ extension AetherEngine { // It follows the TARGET route, not the previous one: a rebuild that flips to software renders // into its own layer, and a preserved host would leave AVKit bound to a stale player with // audio still flowing into the next load (the release `load()` does by hand on that branch). - stopInternal(resetDisplayCriteria: false, keepNativeHost: !targetSoftwarePath, keepCustomReader: true) + // Audit LIF-104: the AE#597 reset outranks the #15 reuse here exactly as it does in `load`, + // or the first rebuild after a reset mounts its item on the invalidated AVPlayer. + let mediaServicesWereReset = consumeMediaServicesReset() + claimSoftwarePathTakeover() // AE#629 + // Keep the current native item through an ordinary audio switch, but never reuse a host + // invalidated by a media-services reset (audit LIF-104). + let keepAudioSwitchItem = audioStreamIndex != nil && discTitleIDOverride == nil + && !targetSoftwarePath && !mediaServicesWereReset && nativeHost?.avPlayer.currentItem != nil + if keepAudioSwitchItem { nativeHost?.pause() } + stopInternal(resetDisplayCriteria: false, + keepNativeHost: !targetSoftwarePath && !mediaServicesWereReset, + keepCustomReader: true, keepCurrentItem: keepAudioSwitchItem) + pendingInPlaceItemHandover = keepAudioSwitchItem + if mediaServicesWereReset { dropAudioPlayerHostAfterMediaServicesReset() } EngineLog.emit("[AetherEngine] reload: stopInternal done (\(elapsedMs(since: reloadStart))ms)", category: .engine) let gen = loadGeneration + // Audit LIF-101: the same settle point as `load`, for the same readiness waypoint. + beginLoadInFlight(gen) + defer { endLoadInFlight(gen) } loadedURL = url lastDetectedVideoCodec = preservedVideoCodec @@ -2215,6 +2403,11 @@ extension AetherEngine { return d }.value } catch { + // A superseded or caller-cancelled reopen is not a reload failure. + if loadGeneration != gen || Task.isCancelled { + EngineLog.emit("[AetherEngine] reload superseded during custom reader reopen; unwinding", category: .engine) + return nil + } EngineLog.emit("[AetherEngine] reload: custom reader reopen failed: \(error)", category: .engine) activeAudioTrackIndex = previousAudioIndex publishError(.reloadFailed, "Reload failed: \(error.localizedDescription)", underlying: error) @@ -2240,6 +2433,11 @@ extension AetherEngine { return d }.value } catch { + // A superseded or caller-cancelled reopen is not a reload failure. + if loadGeneration != gen || Task.isCancelled { + EngineLog.emit("[AetherEngine] reload superseded during disc URL reopen; unwinding", category: .engine) + return nil + } EngineLog.emit("[AetherEngine] reload: disc URL reopen failed: \(error)", category: .engine) activeAudioTrackIndex = previousAudioIndex publishError(.reloadFailed, "Reload failed: \(error.localizedDescription)", underlying: error) @@ -2305,7 +2503,9 @@ extension AetherEngine { audioTracks: audioTracks, activeIndex: softwareHost?.audioStreamIndex ?? -1 ) presentCurrentLayer() - softwareHost?.play() + // Keep the latest transport intent if play/pause changed while the rebuild awaited I/O. + if transportIntentUnderReconstruction ?? resumesPlaying { softwareHost?.play() } + else { softwareHost?.pause() } } else { EngineLog.emit("[AetherEngine] reload: loadNative enter audio=\(audioStreamIndex.map(String.init) ?? "nil") resumeAt=\(String(format: "%.2f", resumeAt))s", category: .engine) // #339: the only write this reload can still produce is a sole-writer host's re-write on the @@ -2389,14 +2589,17 @@ extension AetherEngine { criteriaUnchanged: false, engineIsCriteriaWriter: !loadedOptions.suppressDisplayCriteria, formatKnown: true, - effectiveFormat: videoFormat + effectiveFormat: videoFormat, + noWriterExpected: loadedOptions.sharedOutputRole == .secondary ), - settleCap: loadedOptions.isLive ? .standard : .awaitObservedEnd) + settleCap: loadedOptions.isLive ? .standard : .awaitObservedEnd, + isCurrent: { self.loadGeneration == gen }) try checkLoadCurrent(gen) - nativeHost?.play() + if transportIntentUnderReconstruction ?? resumesPlaying { nativeHost?.play() } + else { nativeHost?.pause() } } try checkLoadCurrent(gen) - state = .playing + state = (transportIntentUnderReconstruction ?? resumesPlaying) ? .playing : .paused // Re-arm samplers: stopInternal nilled them, and the reload path bypasses public load() that normally restarts them. Without this, liveTelemetry stays nil and the stats overlay shows "-" after every audio switch. startMemoryProbe() startLiveTelemetrySampler() @@ -2408,11 +2611,12 @@ extension AetherEngine { if loadedOptions.isLive, !targetSoftwarePath { armLiveReloadWatchdog(generation: gen) } - EngineLog.emit("[AetherEngine] reload: state=.playing total=\(elapsedMs(since: reloadStart))ms", category: .engine) + EngineLog.emit("[AetherEngine] reload: state=\(state == .playing ? ".playing" : ".paused") total=\(elapsedMs(since: reloadStart))ms", category: .engine) } catch is CancellationError { // Superseded by a newer load/stop: it owns the engine state. return nil } catch { + guard loadGeneration == gen, !Task.isCancelled else { return nil } EngineLog.emit( "[AetherEngine] selectAudioTrack reload failed: \(error), playback stopped", category: .engine diff --git a/Sources/AetherEngine/AetherEngine+Probe.swift b/Sources/AetherEngine/AetherEngine+Probe.swift index d9bd51768..49d4c9aec 100644 --- a/Sources/AetherEngine/AetherEngine+Probe.swift +++ b/Sources/AetherEngine/AetherEngine+Probe.swift @@ -109,7 +109,9 @@ extension AetherEngine { /// - **Dolby Atmos**, which for E-AC-3 means the JOC flag in the dependent substream and only exists /// post-decode (`.atmos`, see `AtmosDetectionOptions`), and /// - **HDR10+**, whose ST 2094-40 metadata rides an in-band ITU-T T.35 SEI that no demuxer parses - /// (`.hdr10Plus`, see `HDR10PlusDetectionOptions`). + /// (`.hdr10Plus`, see `HDR10PlusDetectionOptions`), and + /// - **HDR Vivid**, whose CUVA metadata rides the same kind of SEI in HEVC (`.hdrVivid`, scanned in + /// the same pass and under the same `HDR10PlusDetectionOptions` budget). /// /// Both cost reads past `avformat_find_stream_info`, which is why `probe(url:)` does neither. /// Neither is for the playback-start critical path. Asking for both runs both @@ -117,7 +119,7 @@ extension AetherEngine { /// queued, then the queue-flushing seek the Atmos decode pass needs. /// /// Both passes are additive and one-directional. They can only ever SET `isAtmos` / - /// `carriesHDR10PlusMetadata`, never clear what the container already declared, and a pass that hits a cap + /// `carriesHDR10PlusMetadata` / `carriesHDRVividMetadata`, never clear what the container already declared, and a pass that hits a cap /// leaves the base probe's answer exactly where it was. /// /// - Parameters: @@ -125,7 +127,7 @@ extension AetherEngine { /// - options: Forwarded verbatim to `probe(url:)` (`httpHeaders` only). /// - detecting: Which extra passes to run. Empty is the header probe with the same controls. /// - atmosDetection: Bounds + optional track override for the Atmos decode pass. Ignored without `.atmos`. - /// - hdr10PlusDetection: Bounds for the HDR10+ scan. Ignored without `.hdr10Plus`. + /// - hdr10PlusDetection: Bounds for the HDR10+ / HDR Vivid scan. Ignored without either. /// - limits: Whole-probe limits, shared across open, stream analysis, seeks and both passes. /// - cancellation: Cancellation reaches HTTP I/O or the custom reader's `cancel()`. /// - Throws: Open errors, `ProbeError` for whole-probe stops, or `CancellationError`. Ordinary @@ -216,13 +218,18 @@ extension AetherEngine { // HDR10+ first, and before any seek: `avformat_find_stream_info` leaves its packets queued and // `av_read_frame` hands those back first, so at the head of a container the scan gets video packets // that have already been paid for. Running it after the Atmos pass would mean re-reading them. - if detecting.contains(.hdr10Plus) { + let dynamicTargets = detecting.intersection([.hdr10Plus, .hdrVivid]) + if !dynamicTargets.isEmpty { let outcome = Self.detectHDR10Plus( - demuxer: demuxer, videoIndex: demuxer.videoStreamIndex, options: hdr10PlusDetection) + demuxer: demuxer, videoIndex: demuxer.videoStreamIndex, options: hdr10PlusDetection, + targets: dynamicTargets) try control?.check() if outcome.carriesHDR10Plus { probe = Self.enrichHDR10Plus(base: probe) } + if outcome.carriesHDRVivid { + probe.carriesHDRVividMetadata = true + } } let targetIndex = Self.atmosDecodeTargetIndex( @@ -317,6 +324,7 @@ extension AetherEngine { var width: Int32 = 0 var height: Int32 = 0 var dvProfileNum: Int? = nil + var streamFormat: VideoStreamFormat? = nil let videoIdx = demuxer.videoStreamIndex if videoIdx >= 0, let stream = demuxer.stream(at: videoIdx) { detectedFormat = Self.detectVideoFormat(stream: stream) @@ -325,6 +333,7 @@ extension AetherEngine { width = stream.pointee.codecpar.pointee.width height = stream.pointee.codecpar.pointee.height dvProfileNum = Self.dvProfile(stream: stream) + streamFormat = VideoStreamFormat(codecpar: stream.pointee.codecpar) } let codecName: String? = { guard detectedCodecID != AV_CODEC_ID_NONE, @@ -352,7 +361,8 @@ extension AetherEngine { audioTracks: demuxer.audioTrackInfos(), subtitleTracks: demuxer.subtitleTrackInfos(), metadata: demuxer.mediaMetadata(), - isLive: isLive + isLive: isLive, + videoStreamFormat: streamFormat ) } @@ -419,6 +429,7 @@ extension AetherEngine { var firstFramePixelFormat: String? var firstFrameWidth: Int = 0 var firstFrameHeight: Int = 0 + var firstFrameColor: String? /// #407: in decoder output order, i.e. presentation order. Capped so a long run stays a /// diagnostic rather than an allocation. var frameTimesSeconds: [Double] = [] @@ -444,6 +455,12 @@ extension AetherEngine { accum.firstFramePixelFormat = "\(fourCC) (0x\(String(pfType, radix: 16)))" accum.firstFrameWidth = CVPixelBufferGetWidth(pixelBuffer) accum.firstFrameHeight = CVPixelBufferGetHeight(pixelBuffer) + accum.firstFrameColor = [ + kCVImageBufferColorPrimariesKey, + kCVImageBufferTransferFunctionKey, + kCVImageBufferYCbCrMatrixKey, + ].map { (CVBufferCopyAttachment(pixelBuffer, $0, nil) as? String) ?? "-" } + .joined(separator: " / ") } } } catch { @@ -504,7 +521,8 @@ extension AetherEngine { firstFrameWidth: accum.firstFrameWidth, firstFrameHeight: accum.firstFrameHeight, firstError: firstError, - frameTimesSeconds: accum.frameTimesSeconds + frameTimesSeconds: accum.frameTimesSeconds, + firstFrameColor: accum.firstFrameColor ) } @@ -555,19 +573,36 @@ extension AetherEngine { /// /// `formatKnown` is false when the open-time probe failed: the real range is then unknown and a DV write /// may still be inbound, so a suppressed host keeps the full budget. + /// + /// `noWriterExpected` is a `.secondary` load (Sodalite#175): nobody writes criteria for it, so there is + /// no inbound switch to wait for. nonisolated static func playGateGrace( criteriaUnchanged: Bool, engineIsCriteriaWriter: Bool, formatKnown: Bool, - effectiveFormat: VideoFormat + effectiveFormat: VideoFormat, + noWriterExpected: Bool = false ) -> DisplayCriteriaController.StartGrace { // #133: the criteria were already active, nothing was written, nothing can settle. - if criteriaUnchanged { return .skip } + if criteriaUnchanged || noWriterExpected { return .skip } if engineIsCriteriaWriter { return .brief } guard formatKnown else { return .full } return effectiveFormat == .sdr ? .brief : .full } + /// Sodalite#175: a secondary never writes criteria, whatever the host passed. + nonisolated static func applyingSharedOutputRole(_ options: LoadOptions) -> LoadOptions { + guard options.sharedOutputRole == .secondary else { return options } + var adjusted = options + adjusted.suppressDisplayCriteria = true + return adjusted + } + + /// Sodalite#175: Now Playing belongs to the primary; a secondary never takes it. + nonisolated static func ownsNowPlaying(hostOptIn: Bool, role: SharedOutputRole) -> Bool { + hostOptIn && role == .primary + } + /// Whitelist (not blacklist) of AVPlayer-native audio codecs: AAC, MP3, MP2, ALAC, AC-3/E-AC-3, LPCM, FLAC (native since iOS/tvOS 11). Anything else falls back to `AudioPlaybackHost` (FFmpeg). nonisolated static func avPlayerCanDecodeAudio(_ codecID: AVCodecID) -> Bool { switch codecID { @@ -910,6 +945,27 @@ extension AetherEngine { return attemptWhenUnproven && displayEligibleForHDR && !panelRefusedHDRMaster } + /// AE#667: whether an unproven master has to wait for the running switch before it is served. + /// + /// The pre-flight releases the load at its 2 s cap while an HDR switch is still in flight, on purpose: + /// the overlap is what #348 kept, and a proven or media route has nothing to lose by it. The unproven + /// master is the one route that does, because its whole point is to let AVPlayer's acceptance stand in + /// for the readout, and AVPlayer answers for the mode the panel is in NOW. Measured on an Apple TV 4K + /// (tvOS 27.0, HDR10 panel): master served 140 ms after the cap, `-11868` 80 ms later, the switch + /// ending 660 ms after that, and the refusal latched, so every later HDR title went media-direct. + /// + /// Only a switch seen to START counts. A DV switch that never reports would make this an unbounded wait + /// for an end that cannot arrive, and a proven panel is excluded because it already answered. What the + /// wait costs is the prep that used to overlap the switch's tail; the play gate held the first frame + /// until that end anyway. + nonisolated static func unprovenMasterAwaitsSwitchEnd( + routesAsHDR: Bool, + panelPresentsHDR: Bool, + switchRunning: Bool + ) -> Bool { + routesAsHDR && !panelPresentsHDR && switchRunning + } + /// AE#541: the HDR route of an in-place rebuild, composed from the same two decisions the load makes. /// /// The readout and the display's eligibility are the load's, because the rebuild runs no handshake to diff --git a/Sources/AetherEngine/AetherEngine+Recording.swift b/Sources/AetherEngine/AetherEngine+Recording.swift index 968e9be0c..1becc3c68 100644 --- a/Sources/AetherEngine/AetherEngine+Recording.swift +++ b/Sources/AetherEngine/AetherEngine+Recording.swift @@ -22,6 +22,16 @@ public extension AetherEngine { /// or a path that cannot be created. Anything that can only be discovered while writing arrives /// through `$recordingState` as `.failed`, never through both. func startRecording(to url: URL) async throws { + // The previous file is still being finalized off the main actor; it may be this path. The wait + // suspends, so a stop or a new load can run inside it (audit FEA-106): either one retires this + // start, which would otherwise install a writer the host had already stopped, or one on the + // channel that replaced the one it was asked for. + let generation = loadGeneration + let stopSerial = recordingStopSerial + await recordingFinish?.value + guard loadGeneration == generation, recordingStopSerial == stopSerial else { + throw CancellationError() + } if case .recording(let progress) = recordingState { throw RecordingFailure.alreadyRecording(progress.url) } @@ -40,9 +50,7 @@ public extension AetherEngine { url: url, streams: descriptors, ceilingBytes: Self.recordingQueueCeilingBytes, - onFailure: { [weak self] failure in - Task { @MainActor in self?.recordingDidFail(failure) } - } + onFailure: makeRecordingFailureHandler() ) activeRecording = writer @@ -63,7 +71,9 @@ public extension AetherEngine { /// Ends the recording and closes the file. Idempotent, and a no-op when nothing is recording. func stopRecording() async { + recordingStopSerial &+= 1 endRecordingIfRunning(reason: .stoppedByHost) + await recordingFinish?.value } } @@ -97,19 +107,56 @@ extension AetherEngine { } /// Ends a running recording, if any, for a reason that is not a failure. Idempotent. + /// + /// Audit REC-1: the writer drains up to the queue ceiling to disk and writes the trailer, which + /// used to run right here on the main actor. It runs detached now, and `.ended` is published + /// once it is done, because `.ended` promises a closed, playable file. A recording started in + /// the meantime owns the state, so the late `.ended` of this one is dropped. + /// + /// Audit FEA-105: when the writer was already tearing itself down (the queue ceiling, a write + /// error), the writer's own failure report is ignored here because this call released the + /// recording first, so the failure it returns is what gets published instead of `.ended`. func endRecordingIfRunning(reason: RecordingEndReason) { guard let writer = activeRecording else { return } (activeRecordingHost as? LiveRecordingHost)?.setRecordingSink(nil) activeRecordingHost = nil activeRecording = nil stopRecordingProgressTimer() - writer.finish(reason: reason) - recordingState = .ended(reason) + let generation = recordingGeneration + let previous = recordingFinish + recordingFinish = Task { [weak self] in + await previous?.value + let failure = await Task.detached(priority: .utility) { writer.finish(reason: reason) }.value + guard let self, self.recordingGeneration == generation else { return } + if let failure { + self.recordingState = .failed(failure) + EngineLog.emit("[Recording] failed: \(failure)", category: .session) + } else { + self.recordingState = .ended(reason) + } + } + } + + /// The writer's failure callback, bound to the recording it is built for. The writer reports + /// from its teardown queue after draining and writing the trailer, so the report of a recording + /// that has already ended can land after the next one started (a zap with auto-record). Without + /// the binding it ended that next recording, unfinalized, with the previous one's reason + /// (audit CORE-5). + func makeRecordingFailureHandler() -> @Sendable (RecordingFailure) -> Void { + recordingGeneration &+= 1 + let generation = recordingGeneration + return { [weak self] failure in + Task { @MainActor in self?.recordingDidFail(failure, generation: generation) } + } } /// Called from the writer's failure callback, already hopped to the main actor. - func recordingDidFail(_ failure: RecordingFailure) { - guard activeRecording != nil else { return } + func recordingDidFail(_ failure: RecordingFailure, generation: UInt64) { + guard activeRecording != nil, generation == recordingGeneration else { + EngineLog.emit("[Recording] failure of an earlier recording ignored: \(failure)", + category: .session) + return + } (activeRecordingHost as? LiveRecordingHost)?.setRecordingSink(nil) activeRecordingHost = nil activeRecording = nil @@ -188,15 +235,14 @@ extension AetherEngine { /// Test-only: runs the real `startRecording` path against a stub route, so a lifecycle test /// exercises production code rather than a parallel implementation. - func _testStartRecordingWithStubHost(to url: URL, host: TestRecordingHost) throws { + func _testStartRecordingWithStubHost(to url: URL, host: TestRecordingHost, + ceilingBytes: Int = AetherEngine.recordingQueueCeilingBytes) throws { _testSetLiveRoute(isLive: true, route: .loopback) let writer = try LiveRecordingWriter( url: url, streams: host.recordingStreamDescriptors(), - ceilingBytes: Self.recordingQueueCeilingBytes, - onFailure: { [weak self] failure in - Task { @MainActor in self?.recordingDidFail(failure) } - } + ceilingBytes: ceilingBytes, + onFailure: makeRecordingFailureHandler() ) activeRecording = writer activeRecordingHost = host diff --git a/Sources/AetherEngine/AetherEngine+ReloadWithOptions.swift b/Sources/AetherEngine/AetherEngine+ReloadWithOptions.swift index f7530466a..9b8e1958e 100644 --- a/Sources/AetherEngine/AetherEngine+ReloadWithOptions.swift +++ b/Sources/AetherEngine/AetherEngine+ReloadWithOptions.swift @@ -275,6 +275,7 @@ enum SessionOptionCorrection { "nativeRemoteHLS", "sequentialOrigin", "heldSourceConnection", + "sharedOutputRole", ] /// The fields a running SESSION owns, which a correction may name and the rebuild then decides @@ -314,6 +315,8 @@ enum SessionOptionCorrection { if proposed.heldSourceConnection != current.heldSourceConnection { refused.append("heldSourceConnection") } + // The role decides which engine drives the panel and owns Now Playing, joined once at load. + if proposed.sharedOutputRole != current.sharedOutputRole { refused.append("sharedOutputRole") } return refused } @@ -406,11 +409,11 @@ enum SessionOptionCorrection { "dvrWindowSeconds", "liveBlockingReload", "liveJoinProfile", "liveJoinStartsImmediately", "clampsLiveResumeToWindow", "nativeRemoteHLS", "nativeRemoteHLSIngestFallback", - "preserveASSMarkup", "prepareNativeSubtitles", "eagerNativeSubtitleReaders", "confirmAtmos", + "preserveASSMarkup", "prepareNativeSubtitles", "eagerNativeSubtitleReaders", "serveIFramePlaylist", "confirmAtmos", "nativeSubtitlePreferredLanguages", "sequentialOrigin", "maxConcurrentSourceRequests", "heldSourceConnection", "declaredDurationSeconds", "probesize", "maxAnalyzeDuration", "preferredAudioLanguages", "preferredSubtitleLanguages", "externalSubtitles", "forwardBufferSegments", "autoplay", "audioDelaySeconds", "teletextPage", "deinterlaceMode", "deinterlaceFieldRate", "preferredDecodePath", - "isLiveRejoin", "subtitleSessionCarryover", + "escalatesToSoftwarePath", "sharedOutputRole", "isLiveRejoin", "subtitleSessionCarryover", ] } diff --git a/Sources/AetherEngine/AetherEngine+ScrubThumbnail.swift b/Sources/AetherEngine/AetherEngine+ScrubThumbnail.swift index ddc1f2741..ebff1fc35 100644 --- a/Sources/AetherEngine/AetherEngine+ScrubThumbnail.swift +++ b/Sources/AetherEngine/AetherEngine+ScrubThumbnail.swift @@ -5,7 +5,8 @@ extension AetherEngine { /// The 2026-09-02 Whole Title trace opened 30 software decoders in 20 seconds while /// scrubbing three resident 100 MB segments and back. Six keeps that small working set - /// warm; the segment bytes are already disk-backed and the bound remains fixed. + /// warm; each extractor maps its segment file (audit SEG-2), so the entries hold clean + /// file-backed pages rather than heap copies, and the bound remains fixed. nonisolated static let scrubThumbnailExtractorLimit = 6 /// Cache-backed scrub still for the active native session (live or VOD). Decodes from @@ -30,8 +31,17 @@ extension AetherEngine { /// to 0: thumbnail mode returns the first frame after the seek, so 0 lands on that segment's /// first keyframe whether its fMP4 tfdt is absolute or zero-based (post-restart). This makes /// the decode axis-independent and correct by construction. Per-segment granularity. + /// + /// A software session (AE#605) has no segments; it decodes the still out of the packet cache + /// its seeks already land in, keyframe to target, on the same session axis `seek(to:)` takes. public func vodScrubThumbnail(atSeconds seconds: Double, maxWidth: Int = 320) async -> CGImage? { - guard !isLive, let session = nativeVideoSession else { return nil } + guard !isLive else { return nil } + guard let session = nativeVideoSession else { + guard let host = softwareHost else { return nil } + let gen = loadGeneration + let image = await host.vodScrubStill(atSessionSeconds: seconds, maxWidth: maxWidth) + return loadGeneration == gen ? image : nil + } let gen = loadGeneration let source = await Task.detached(priority: .userInitiated) { [session] in session.scrubThumbnailSource(atSeconds: seconds) @@ -45,7 +55,8 @@ extension AetherEngine { scrubThumbnailExtractors.append(hit) extractor = hit.extractor } else { - extractor = FrameExtractor(reader: DataIOReader(data: source.data), formatHint: "mp4") + guard let reader = source.makeReader() else { return nil } + extractor = FrameExtractor(reader: reader, formatHint: "mp4") scrubThumbnailExtractors.append((source.segmentIndex, extractor)) trimScrubThumbnailExtractors() } @@ -61,12 +72,18 @@ extension AetherEngine { } } - /// True when a native session (live or VOD) is active, so `scrubThumbnail` can serve - /// cache-backed stills with no second connection. False on the software path (no - /// SegmentCache) and before load. Hosts on a single-connection source should gate the + /// True when the active session can serve cache-backed stills from `scrubThumbnail` with no + /// second connection: any native session (live or VOD, SegmentCache), a software live session + /// (DVR packet ring, #544) and a software VOD session reading a remote source (packet cache, + /// AE#605). False before load and on a software session that plays a local file, which keeps no + /// cache because re-reading the file is free; `makeFrameExtractor()` serves that case. Hosts on a single-connection source should gate the /// scrub-preview affordance on this: true means use `scrubThumbnail`; false means hide /// the preview rather than show blank frames from a refused second-connection /// FrameExtractor (#106). It reports capability, not per-frame availability: transient /// nils from `scrubThumbnail` while a segment is still being produced are expected. - public var supportsCacheBackedStills: Bool { nativeVideoSession != nil } + public var supportsCacheBackedStills: Bool { + if nativeVideoSession != nil { return true } + guard let softwareHost else { return false } + return isLive || softwareHost.servesPacketCacheStills + } } diff --git a/Sources/AetherEngine/AetherEngine+SoftwarePathEscalation.swift b/Sources/AetherEngine/AetherEngine+SoftwarePathEscalation.swift index 1f73cba04..784152394 100644 --- a/Sources/AetherEngine/AetherEngine+SoftwarePathEscalation.swift +++ b/Sources/AetherEngine/AetherEngine+SoftwarePathEscalation.swift @@ -17,32 +17,78 @@ import Foundation extension AetherEngine { /// Rebuild this session on the software path, once, because the native one refused the media. + /// `expectedGeneration` is the session the failure was raised under (audit LIF-103). @MainActor - func escalateToSoftwarePath(_ request: SoftwarePathEscalation.Request) async { + func escalateToSoftwarePath(_ request: SoftwarePathEscalation.Request, expectedGeneration: UInt64) async { + // Audit LIF-103: a stop() or a new load() between the failure and this job ended that session; + // its options and budget now belong to whatever replaced it. + guard loadGeneration == expectedGeneration else { + EngineLog.emit( + "[AetherEngine] #561 escalation dropped: the session it was raised for has ended", + category: .engine) + return + } guard SoftwarePathEscalation.shouldEscalate( errorDomain: request.domain, availability: SoftwarePathEscalation.Availability( alreadyEscalated: softwarePathEscalationBudget.isSpent, preferredDecodePath: loadedOptions.preferredDecodePath, - nativeRemoteHLS: loadedOptions.nativeRemoteHLS + nativeRemoteHLS: loadedOptions.nativeRemoteHLS, + hostAllowsEscalation: loadedOptions.escalatesToSoftwarePath ) ) else { return } // The host's probe read mount-time options and the #93 rung does not consult it at all, so // the decision is made again here, against what the session is actually running on. guard softwarePathEscalationBudget.take() else { return } + let absorbed = Self.absorbedFailure(request) + let duringStartup = waitingLoadGenerations.contains(loadGeneration) + // AE#629 round 3: the position the rebuild resumes at, not the host's reading at the refusal. + // Under a mount a load() of the host's own raised, AVPlayer's clock reads the start of the + // segment it decodes up from until the mount seek lands (12.00 s under 15.90 s on the + // reporter's box), so the event and the rebuild disagreed by that gap. + let resumesAt = positionForSessionRebuild + + let verdict = request.domain == SoftwarePathEscalation.liveJoinErrorDomain + ? "the native route cannot open this live bitstream" + : "AVPlayer refused the media (\(request.domain)/\(request.code))" EngineLog.emit( - "[AetherEngine] #561 AVPlayer refused the media (\(request.domain)/\(request.code)) at " - + "\(String(format: "%.2f", request.positionSeconds))s; rebuilding this session on the " - + "software path, which decodes it with libavcodec instead: \(request.message)", + "[AetherEngine] #561 \(verdict) at " + + "\(String(format: "%.2f", resumesAt))s; rebuilding this session on the " + + "software path, which decodes it with libavcodec instead" + + (duringStartup ? " (the waiting load follows it)" : "") + ": \(request.message)", category: .engine ) + softwarePathEscalations.send(SoftwarePathEscalationEvent( + absorbedFailure: absorbed, + positionSeconds: resumesAt, + duringStartup: duringStartup)) + + let rebuild = startTakeoverRebuild { $0.preferredDecodePath = .software } do { - try await reloadAtCurrentPosition { $0.preferredDecodePath = .software } + try await rebuild.value EngineLog.emit( "[AetherEngine] #561 rebuilt on the software path", category: .engine) + } catch is CancellationError { + // Audit CORE-1: a stop() or a new load() superseded the rebuild. The session this + // failure belonged to is gone, and `.error` would land on whatever replaced it. + EngineLog.emit( + "[AetherEngine] #561 software rebuild superseded; nothing to surface", + category: .engine) } catch { + // AE#629: a rebuild that failed after its teardown has already surfaced its own failure, + // and a load() that followed it threw that same error. Publishing the absorbed one on top + // would hand the host two `.error`s for one failure, the second contradicting the throw. + // The absorbed failure is not lost: `softwarePathEscalations` carried it. Audit LIF-103: + // asked of the generation, not of `state`, which does not say whose failure it holds. + guard loadGeneration == expectedGeneration else { + EngineLog.emit( + "[AetherEngine] #561 the software rebuild failed after its teardown (\(error)); " + + "its own failure is the one surfaced", + category: .engine) + return + } // The rung is gone and the failure was never surfaced, so it has to be surfaced here or // the session would sit on a picture that stopped with nothing said. EngineLog.emit( @@ -50,14 +96,58 @@ extension AetherEngine { + "surfacing the original failure", category: .engine ) - publishError( - PlaybackErrorInfo( - kind: .nativeItemFailed, - message: request.message, - underlyingDomain: request.domain.isEmpty ? nil : request.domain, - underlyingCode: request.code == 0 ? nil : request.code - ) - ) + publishError(absorbed) + } + } + + /// Start a rebuild of this session that the engine decided on itself (AE#561 software path, AE#641 + /// video-only), in a shape a `load()` still waiting on the session's startup can follow. + /// + /// AE#629: that load is about to be superseded by the rebuild's own. It follows the rebuild instead + /// of unwinding, and the rebuild continues its #361 startup sequence rather than dropping the + /// host's bar back to zero. + @MainActor + func startTakeoverRebuild(applying change: @escaping @Sendable (inout LoadOptions) -> Void) -> Task { + let supersededGeneration = loadGeneration + let duringStartup = waitingLoadGenerations.contains(supersededGeneration) + let rebuild = Task { @MainActor in + // Both are armed where nothing can run before the rebuild's teardown consumes them, and + // withdrawn after, for the paths that never reach one (a correction refused up front). + // The takeover is only CLAIMED by that teardown, and only if the session it ends is still + // the one that failed: a rebuild refused before it tore anything down, or a host stop() + // that got in first, must leave the waiting load to unwind as the host's own. + // The failure's session is already gone: rebuilding now would tear down its successor. + guard self.loadGeneration == supersededGeneration else { throw CancellationError() } + if duringStartup { self.continueStartupAcrossReroute() } + self.softwarePathTakeoverArm = supersededGeneration + defer { + self.abandonStartupContinuation() + self.softwarePathTakeoverArm = nil + } + _ = try await self.reloadAtCurrentPosition(applying: change) } + softwarePathRebuild = rebuild + return rebuild + } + + /// Called by a teardown about to end `loadGeneration`: when that teardown is the engine's own + /// rebuild, a `load()` still waiting on the ended generation follows the rebuild (AE#629). + @MainActor + func claimSoftwarePathTakeover() { + guard let armed = softwarePathTakeoverArm else { return } + softwarePathTakeoverArm = nil + guard armed == loadGeneration, let rebuild = softwarePathRebuild else { return } + softwarePathTakeover = SoftwarePathEscalation.Takeover( + supersededGeneration: armed, rebuild: rebuild) + } + + /// The failure an escalation takes instead of surfacing, in the shape it would have surfaced in. + nonisolated static func absorbedFailure(_ request: SoftwarePathEscalation.Request) -> PlaybackErrorInfo { + PlaybackErrorInfo( + kind: .nativeItemFailed, + message: request.message, + underlyingDomain: request.domain.isEmpty ? nil : request.domain, + underlyingCode: request.code == 0 ? nil : request.code + ) } } diff --git a/Sources/AetherEngine/AetherEngine+SubtitleOCR.swift b/Sources/AetherEngine/AetherEngine+SubtitleOCR.swift index 86d127b02..0f64bbcaa 100644 --- a/Sources/AetherEngine/AetherEngine+SubtitleOCR.swift +++ b/Sources/AetherEngine/AetherEngine+SubtitleOCR.swift @@ -2,8 +2,8 @@ import Foundation // Phase D: selection-armed worker feeding a bitmap track's native WebVTT rendition with // OCR-recognized text cues, so PGS/DVB/DVD subtitles survive PiP / AirPlay / external display -// on the native path. Packet source is the session SubtitlePacketStore (#112 harvest); decode -// runs on the MainActor tick (overlay-drainer cost class), Vision runs on a dedicated thread. +// on the native path. Packet source is the session SubtitlePacketStore (#112 harvest); the tick +// plans and resolves on the MainActor, decodes off it (AE#628), and Vision runs on a dedicated thread. extension AetherEngine { /// Arm for the selected embedded bitmap track. The per-ordinal cursor survives re-arming. @@ -17,9 +17,16 @@ extension AetherEngine { subtitleOCRWorkerTask = Task.detached(priority: .utility) { [weak self] in while !Task.isCancelled { guard let self else { return } + // AE#628: plan on the MainActor, decode here, off it, then resolve ends back on it. + // The decode is the bitmap blit, the heaviest frame of the report's profile. + let planned = await MainActor.run { [weak self] () -> SubtitleOCRTickPlan? in + guard !Task.isCancelled, let self else { return nil } + return self.subtitleOCRPlanTick(ordinal: ordinal, streamIndex: streamIndex) + } + let decoded = planned?.decodeHandoff.decode().first ?? [] let batch = await MainActor.run { [weak self] in - guard !Task.isCancelled, let self else { return [SubtitleCue]() } - let batch = self.subtitleOCRCollectTick(ordinal: ordinal, streamIndex: streamIndex) + guard !Task.isCancelled, let self, let planned else { return [SubtitleCue]() } + let batch = self.subtitleOCRFinishTick(planned, events: decoded) self.subtitleOCRBatchInFlight = !batch.isEmpty return batch } @@ -57,17 +64,12 @@ extension AetherEngine { subtitleOCRPendingStates.removeAll() } - /// MainActor tick: plan the window (drainer pacing, larger lead), decode the stored packets - /// (bounded per tick), resolve composition ends, return CLOSED cues for off-main OCR. - private func subtitleOCRCollectTick(ordinal: Int, streamIndex: Int32) -> [SubtitleCue] { - guard let packetStore = activeSubtitlePacketStore else { return [] } + /// MainActor half before the decode: plan the window (drainer pacing, larger lead) and pick the + /// stored packets (bounded per tick). The decode runs off the MainActor (AE#628) and + /// `subtitleOCRFinishTick` resolves composition ends into CLOSED cues for off-main OCR. + fileprivate func subtitleOCRPlanTick(ordinal: Int, streamIndex: Int32) -> SubtitleOCRTickPlan? { + guard let packetStore = activeSubtitlePacketStore else { return nil } let playhead = sourceTime - var pending = subtitleOCRPendingStates[ordinal] ?? SubtitleOCRPendingState() - var closed: [SubtitleCue] = [] - defer { - closed.append(contentsOf: pending.expired(asOf: playhead)) - subtitleOCRPendingStates[ordinal] = pending - } // #271: same rule as the overlay drainer, a tick that ran long is not a seek. let tickUptime = Double(DispatchTime.now().uptimeNanoseconds) / 1_000_000_000 let elapsed = subtitleOCRLastTickUptime.map { tickUptime - $0 } ?? 0 @@ -82,18 +84,21 @@ extension AetherEngine { switch plan { case .idle: subtitleOCRCursors[ordinal]?.lastPlayhead = playhead - return closed + return SubtitleOCRTickPlan(ordinal: ordinal, playhead: playhead, plan: plan, + window: (playhead, playhead), decoder: nil, batch: []) case .decode(let from, let through): window = (from, through) case .resetAndDecode(let from, let through): subtitleOCRDecoder = nil - pending = SubtitleOCRPendingState() window = (from, through) } if subtitleOCRDecoder == nil { subtitleOCRDecoder = makeSubtitleDrainDecoder(streamIndex: streamIndex, channel: .primary) } - guard let decoder = subtitleOCRDecoder else { return closed } + guard let decoder = subtitleOCRDecoder else { + return SubtitleOCRTickPlan(ordinal: ordinal, playhead: playhead, plan: .idle, + window: window, decoder: nil, batch: []) + } let entries = packetStore.entries(streamIndex: streamIndex, from: window.from, through: window.through) // #271: the cap has to fall on a PTS boundary. The cursor is a bare PTS advanced by @@ -104,20 +109,38 @@ extension AetherEngine { let batchEnd = SubtitleOverlayDrainer.batchEnd( count: entries.count, cap: Self.subtitleOCRMaxPacketsPerTick, ptsAt: { entries[$0].ptsSeconds }) - let batch = entries[.. [SubtitleCue] { + let ordinal = tick.ordinal + var pending = subtitleOCRPendingStates[ordinal] ?? SubtitleOCRPendingState() + var closed: [SubtitleCue] = [] + defer { + closed.append(contentsOf: pending.expired(asOf: tick.playhead)) + subtitleOCRPendingStates[ordinal] = pending + } + guard let decoder = tick.decoder else { return closed } + guard subtitleOCRArmedOrdinal == ordinal, subtitleOCRDecoder === decoder else { return closed } + if case .resetAndDecode = tick.plan { pending = SubtitleOCRPendingState() } var lastDecoded = subtitleOCRCursors[ordinal]?.lastDecodedPts - for entry in batch { - if let event = Self.decodeStoredSubtitlePacket(entry, with: decoder) { + for (entry, decoded) in zip(tick.batch, events) { + if let event = decoded { closed.append(contentsOf: pending.consume( eventPts: entry.ptsSeconds, cues: event.cues, trimAt: event.pgsTrimAt)) } lastDecoded = entry.ptsSeconds } - if case .resetAndDecode = plan, batch.isEmpty { - lastDecoded = window.from + if case .resetAndDecode = tick.plan, tick.batch.isEmpty { + lastDecoded = tick.window.from } subtitleOCRCursors[ordinal] = SubtitleDrainCursor( - lastDecodedPts: lastDecoded ?? window.from, lastPlayhead: playhead) + lastDecodedPts: lastDecoded ?? tick.window.from, lastPlayhead: tick.playhead) return closed } @@ -139,3 +162,18 @@ extension AetherEngine { } } } + +/// AE#628: one OCR worker tick between its MainActor halves. `decoder` is nil for a tick that +/// decodes nothing (idle, or no decoder could be built), which still expires pending cues. +fileprivate struct SubtitleOCRTickPlan: @unchecked Sendable { + let ordinal: Int + let playhead: Double + let plan: SubtitleDrainPlan + let window: (from: Double, through: Double) + let decoder: EmbeddedSubtitleDecoder? + let batch: ArraySlice + + var decodeHandoff: SubtitleDrainDecodeHandoff { + SubtitleDrainDecodeHandoff(jobs: decoder.map { [($0, batch)] } ?? []) + } +} diff --git a/Sources/AetherEngine/AetherEngine+Subtitles.swift b/Sources/AetherEngine/AetherEngine+Subtitles.swift index 896f79350..9c2b22841 100644 --- a/Sources/AetherEngine/AetherEngine+Subtitles.swift +++ b/Sources/AetherEngine/AetherEngine+Subtitles.swift @@ -119,7 +119,7 @@ extension AetherEngine { // #112 rework: every embedded track (text and bitmap, VOD and live) is served by the // playhead-paced drainer from the session's packet store. The producer keeps all subtitle // streams from init, so selection needs no side demuxer, no positioning, and no recovery: - // the immediate drainer tick backfills the window around the playhead synchronously. + // the immediate drainer tick backfills the window around the playhead (off the MainActor, AE#628). cancelSidecarTask() isSubtitleActive = true subtitleCues = [] @@ -140,7 +140,7 @@ extension AetherEngine { isLoadingSubtitles = false EngineLog.emit( "[AetherEngine] overlay fed by packet-store drainer for stream=\(index) " - + "(backfilled \(subtitleCues.count) cues)", + + "(backfill requested)", category: .engine ) } @@ -199,7 +199,7 @@ extension AetherEngine { cancelSidecarTask(channel: .secondary) // #112 rework: secondary embedded tracks ride the same packet-store drainer on their - // own channel; the immediate tick backfills synchronously. + // own channel; the immediate tick backfills it. isSecondarySubtitleActive = true secondarySubtitleCues = [] pgsStaleArrivalGates[.secondary]?.reset() // #100 @@ -246,21 +246,54 @@ extension AetherEngine { /// Build a fresh overlay decoder for the stream on whichever host owns the session demuxer. func makeSubtitleDrainDecoder(streamIndex: Int32, channel: SubtitleChannel) -> EmbeddedSubtitleDecoder? { + #if DEBUG + if let factory = subtitleDrainDecoderFactoryForTesting { return factory(streamIndex) } + #endif let preserveASS = channel == .primary && loadedOptions.preserveASSMarkup return nativeVideoSession?.makeOverlayDecoder(streamIndex: streamIndex, preserveASSMarkup: preserveASS) ?? softwareHost?.makeOverlayDecoder(streamIndex: streamIndex, preserveASSMarkup: preserveASS) } - /// Start (or keep) the 500ms drain loop. Performs an immediate tick so a fresh selection - /// backfills from the packet store synchronously, matching the #32 tap-overlay UX. + /// Start (or keep) the 500ms drain loop. Requests an immediate tick so a fresh selection + /// backfills from the packet store without waiting out the first interval (#32 tap-overlay UX). func startSubtitleDrainer() { - subtitleDrainTick() + requestSubtitleDrainTick() guard subtitleDrainerTask == nil else { return } subtitleDrainerTask = Task { [weak self] in while !Task.isCancelled { try? await Task.sleep(nanoseconds: AetherEngine.subtitleDrainTickNanoseconds) guard let self, !Task.isCancelled else { return } - self.subtitleDrainTick() + self.requestSubtitleDrainTick() + } + } + } + + /// AE#628: the drain loop's tick. Plans on the MainActor, decodes the batch off it, and applies + /// the result back on it. A bitmap decode expands a full-canvas indexed plane per display set, + /// and a selection or seek backfills a whole window of them at once; on the MainActor that was + /// the top frame of the report's profile. A tick asked for while one is in flight is queued, + /// not run beside it: the decoders are not reentrant. + func requestSubtitleDrainTick() { + guard subtitleDrainTickInFlight == nil else { + subtitleDrainTickRequested = true + return + } + guard let work = prepareSubtitleDrainTick() else { return } + let handoff = work.decodeHandoff + guard !handoff.isEmpty else { + finishSubtitleDrainTick(work, events: handoff.decode()) + return + } + subtitleDrainTickSerial &+= 1 + let serial = subtitleDrainTickSerial + subtitleDrainTickInFlight = Task { [weak self] in + let events = await Task.detached(priority: .userInitiated) { handoff.decode() }.value + guard let self, self.subtitleDrainTickSerial == serial else { return } + self.subtitleDrainTickInFlight = nil + self.finishSubtitleDrainTick(work, events: events) + if self.subtitleDrainTickRequested { + self.subtitleDrainTickRequested = false + self.requestSubtitleDrainTick() } } } @@ -268,6 +301,9 @@ extension AetherEngine { func stopSubtitleDrainer(reason: SubtitleDrainStopReason) { subtitleDrainerTask?.cancel() subtitleDrainerTask = nil + subtitleDrainTickInFlight = nil // AE#628: its batch lands on a stale serial and is dropped + subtitleDrainTickRequested = false + subtitleDrainTickSerial &+= 1 subtitleDrainDecoders.removeAll() subtitleDrainCursors.removeAll() subtitleDrainLastTickUptime = nil // #271 @@ -296,8 +332,22 @@ extension AetherEngine { activeSubtitlePacketStore?.setProtectedStreams(Set(subtitleDrainTargets.values)) } + /// The synchronous tick: plan, decode and apply in one go on the MainActor. Kept for the + /// teletext page change and for tests; the drain loop goes through `requestSubtitleDrainTick`, + /// which decodes off the MainActor (AE#628). Queued rather than run while that one is in flight. func subtitleDrainTick() { - guard !subtitleDrainTargets.isEmpty, let store = activeSubtitlePacketStore else { return } + guard subtitleDrainTickInFlight == nil else { + subtitleDrainTickRequested = true + return + } + guard let work = prepareSubtitleDrainTick() else { return } + finishSubtitleDrainTick(work, events: work.decodeHandoff.decode()) + } + + /// AE#628: the MainActor half before the decode. Plans every channel, states the ticks that + /// decode nothing (idle, no decoder), and hands back what the decode needs. + func prepareSubtitleDrainTick() -> SubtitleDrainTickWork? { + guard !subtitleDrainTargets.isEmpty, let store = activeSubtitlePacketStore else { return nil } store.setProtectedStreams(Set(subtitleDrainTargets.values)) // #166: re-assert protection let playhead = sourceTime // #416: the pump is rendering the frame at the playhead, so it has necessarily read from @@ -311,7 +361,7 @@ extension AetherEngine { let tickUptime = Double(DispatchTime.now().uptimeNanoseconds) / 1_000_000_000 let elapsed = subtitleDrainLastTickUptime.map { tickUptime - $0 } ?? 0 subtitleDrainLastTickUptime = tickUptime - var prefetchNeedsReanchor = false + var work = SubtitleDrainTickWork(store: store, playhead: playhead) for (channel, streamIndex) in subtitleDrainTargets { let hadCursor = subtitleDrainCursors[channel] != nil let plan = SubtitleOverlayDrainer.drainPlan( @@ -322,7 +372,7 @@ extension AetherEngine { jumpThreshold: Self.subtitleDrainJumpThresholdSeconds, elapsedSinceLastPlan: elapsed) if Self.subtitleForwardPrefetchNeedsReanchor(plan: plan, hadCursor: hadCursor) { - prefetchNeedsReanchor = true + work.prefetchNeedsReanchor = true } let window: (from: Double, through: Double) // #250: a reset starts a fresh contiguous decoded run; a steady tick extends the one @@ -426,6 +476,40 @@ extension AetherEngine { gapHoldTicksLeft = held?.harvestGapAt == cut.at ? (held?.harvestGapTicksLeft ?? 0) - 1 : Self.subtitleDrainHarvestGapTicks } + work.channels.append(SubtitleDrainChannelWork( + channel: channel, streamIndex: streamIndex, plan: plan, isReset: isReset, + window: window, coverageStart: coverageStart, retained: retained, + decoder: decoder, entries: entries, batchEnd: batchEnd, decodeEnd: decodeEnd, + gapHoldAt: gapHoldAt, gapHoldSequence: gapHoldSequence, + gapHoldTicksLeft: gapHoldTicksLeft)) + } + return work + } + + /// AE#628: the MainActor half after the decode. `events` holds one decoded result per packet + /// the prepare half handed out, channel by channel. A channel re-selected, cleared or rebuilt + /// while its batch decoded is skipped: its cursor did not move, so the next tick redoes it. + func finishSubtitleDrainTick(_ work: SubtitleDrainTickWork, + events decodedByChannel: [[EmbeddedSubtitleDecoder.SubtitleEvent?]]) { + let store = work.store + let playhead = work.playhead + guard activeSubtitlePacketStore === store else { return } + for (job, events) in zip(work.channels, decodedByChannel) { + let channel = job.channel + let streamIndex = job.streamIndex + guard subtitleDrainTargets[channel] == streamIndex, + subtitleDrainDecoders[channel] === job.decoder else { continue } + let plan = job.plan + let isReset = job.isReset + let window = job.window + let coverageStart = job.coverageStart + let retained = job.retained + let entries = job.entries + let batchEnd = job.batchEnd + let decodeEnd = job.decodeEnd + let gapHoldAt = job.gapHoldAt + let gapHoldSequence = job.gapHoldSequence + let gapHoldTicksLeft = job.gapHoldTicksLeft // #271: bind the channel's cue array ONCE for the whole batch. `subtitleCues` is // `@Published`, whose wrapper exposes get/set and no `_modify`, so passing it inout per // event both copy-on-writes the array and publishes it: every consumer then walks a @@ -451,8 +535,8 @@ extension AetherEngine { // #362: the cursor's own harvest sequence rides with it, so the next tick can tell // whether its first entry was read by the same run or is the far side of a hole. var lastDecodedSequence = subtitleDrainCursors[channel]?.lastDecodedSequence ?? 0 - for entry in entries[.. 0 ? sourceVideoWidth : 1920 let h = sourceVideoHeight > 0 ? sourceVideoHeight : 1080 let startAt = startAtSeconds ?? sourceTime + // Sodalite#156: the anchor next to the playhead it is supposed to mean. `sourceTime` is + // written on the native path only by the render sink and by seek landings, never by the clock + // tick (#49), and while an external screen holds the picture nothing renders locally. A + // reader anchored on a stale or zero source time refills from the head of the file and then + // serves empty .vtt for the window the receiver is actually asking for, which is a caption + // box with nothing in it. These two numbers disagreeing is that defect; them agreeing means + // the reader simply has not caught up yet, which is a different problem with a different fix. + EngineLog.emit("[AetherEngine] #156 reader anchor: startAt=\(String(format: "%.2f", startAt))s " + + "sourceTime=\(String(format: "%.2f", sourceTime))s " + + "clock=\(String(format: "%.2f", currentTime))s " + + "explicit=\(startAtSeconds.map { String(format: "%.2f", $0) } ?? "nil")", + category: .engine) let reader = customClone // #76: same bounded-probe + active-title open as the inline reader. let probesize = loadedOptions.probesize @@ -2141,6 +2239,7 @@ extension AetherEngine { EngineLog.emit("[AetherEngine] native subtitle select: item=\(itemID) opt=\(option.displayName) groupOpts=\(group.options.count) criteriaAuto=\(currentAVPlayer?.appliesMediaSelectionCriteriaAutomatically ?? true) itemIsCurrent=\(currentAVPlayer?.currentItem === item)", category: .engine) item.select(nil, in: group) try? await Task.sleep(nanoseconds: 100_000_000) + releaseInjectedRenditionSuppression() item.select(option, in: group) let after = item.currentMediaSelection.selectedMediaOption(in: group)?.displayName ?? "nil" EngineLog.emit("[AetherEngine] native subtitle select done: selected=\(after) itemIsCurrent=\(currentAVPlayer?.currentItem === item)", category: .engine) @@ -2592,8 +2691,9 @@ extension AetherEngine { /// the host registered it under, but the selection is an `AVMediaSelection` one, so AVPlayer renders /// it and it survives leaving the view hierarchy. /// - /// Matched by NAME: the rewriter guarantees uniqueness within the group (it disambiguates against the - /// origin's own names), and `AVMediaSelectionOption.displayName` is the rendition's NAME attribute. + /// Matched by NAME: `name` is the one the served master declares (`RemoteHLSMasterRewrite` + /// disambiguates it against the origin's names and every other sidecar, and escapes it), and + /// `AVMediaSelectionOption.displayName` is the rendition's NAME attribute. /// A miss leaves the previous selection alone and says so rather than silently reporting success. func selectInjectedSubtitleRendition(id: Int, name: String) { guard currentAVPlayer?.currentItem != nil else { return } @@ -2625,6 +2725,7 @@ extension AetherEngine { guard let id = activeSubtitleTrackIndex, injectedSubtitleRenditionNames[id] != nil else { return } injectedSubtitleSelectionTask?.cancel() injectedSubtitleSelectionTask = nil + deselectInjectedRenditionNow() forceNativeLegibleDeselectedUntilHostSelects() } @@ -2639,43 +2740,111 @@ extension AetherEngine { || externalPlaybackHoldsThePicture) } + /// An overlay-drawn ASS track keeps its plain rendition selected behind a suppressing legible + /// output instead of deselecting it. AE#616 measures the bypass clock off the selected injected + /// rendition, and a host drawing ASS from `sourceTime` needs that measurement most. A native + /// surface (PiP, AirPlay, an external display) drops the suppression and shows the rendition. private func updateInjectedSubtitleMediaSelection(id: Int, name: String, nativeRendering: Bool) { injectedSubtitleSelectionTask?.cancel() injectedSubtitleSelectionTask = nil - if !nativeRendering { - forceNativeLegibleDeselectedUntilHostSelects() - return - } cancelNativeLegibleDeselectPin() guard let item = currentAVPlayer?.currentItem else { return } injectedSubtitleSelectionTask = Task { @MainActor [weak self] in guard let self else { return } + // Every suspension below can outlive the item or the pick it was made for. + @MainActor func stillWanted() -> Bool { + guard !Task.isCancelled, self.currentAVPlayer?.currentItem === item, + self.activeSubtitleTrackIndex == id else { return false } + return true + } self.currentAVPlayer?.appliesMediaSelectionCriteriaAutomatically = false guard let group = try? await item.asset.loadMediaSelectionGroup(for: .legible), - !Task.isCancelled, self.currentAVPlayer?.currentItem === item, - self.activeSubtitleTrackIndex == id else { return } - var match: AVMediaSelectionOption? - var seen: [String] = [] + stillWanted() else { return } + var snapshots: [RemoteHLSMediaSelection.LegibleOption] = [] for option in group.options { let playlistName = await RemoteHLSMediaSelection.playlistName(of: option) - guard !Task.isCancelled, self.currentAVPlayer?.currentItem === item, - self.activeSubtitleTrackIndex == id else { return } - seen.append(playlistName ?? option.displayName) - if match == nil, playlistName == name || option.displayName == name { match = option } + guard stillWanted() else { return } + snapshots.append(RemoteHLSMediaSelection.LegibleOption( + displayName: option.displayName, extendedLanguageTag: nil, + isDefault: false, isForced: false, isSDH: false, + playlistName: playlistName)) } - guard let option = match else { + let seen = snapshots.map(RemoteHLSMediaSelection.injectionKey) + guard let index = RemoteHLSMediaSelection.injectedRenditionIndex(named: name, in: snapshots) else { EngineLog.emit( "[AetherEngine] #316: injected rendition \"\(name)\" is not in the item's legible " + "group (\(seen.joined(separator: ", ")))", category: .engine) + if !nativeRendering { + // Nothing of ours to hide behind, so keep whatever AVPlayer would pick off the screen. + self.forceNativeLegibleDeselectedUntilHostSelects() + } return } - item.select(option, in: group) - EngineLog.emit("[AetherEngine] #316: selected injected rendition \"\(name)\" for external id=\(id)", + if nativeRendering { + self.releaseInjectedRenditionSuppression() + } else if self.injectedRenditionSuppression?.item !== item { + // A line on screen when an output starts suppressing stays there for good (AE#616). + // The reporter's order: take the rendition down, let a frame or two pass, then hide + // it and select it again. + item.select(nil, in: group) + // AVPlayer fetches and buffers a selected rendition, so a hidden one selected before + // playback runs joins the startup buffer for a subtitle nobody sees. It is there only + // for AE#616, which measures presented lines and reads nothing before the fill is done. + while !self.hiddenRenditionMayJoin() { + try? await Task.sleep(nanoseconds: 200_000_000) + guard stillWanted() else { return } + } + try? await Task.sleep(nanoseconds: 100_000_000) + guard stillWanted() else { return } + self.suppressInjectedRenditionDrawing(on: item) + } + item.select(group.options[index], in: group) + self.injectedRenditionSelection = InjectedRenditionSelection(item: item, group: group) + EngineLog.emit("[AetherEngine] #316: selected injected rendition \"\(name)\" for external id=\(id)" + + (nativeRendering ? "" : ", drawn by the host overlay (rendition suppressed)"), category: .engine) } } + /// Hangs a suppressing legible output on `item`, replacing one left on an earlier item. + private func suppressInjectedRenditionDrawing(on item: AVPlayerItem) { + releaseInjectedRenditionSuppression() + let output = AVPlayerItemLegibleOutput() + output.suppressesPlayerRendering = true + item.add(output) + injectedRenditionSuppression = InjectedRenditionSuppression(output: output, item: item) + } + + /// When a hidden ASS rendition may be selected: its whole-program .vtt is extracted, and playback + /// is running, so AE#616 has presented lines to measure. + private func hiddenRenditionMayJoin() -> Bool { + if let provider = remoteHLSSubtitleProxy?.provider, !provider.isFillFinished { return false } + #if DEBUG + if hiddenRenditionIgnoresPlaybackForTesting { return true } + #endif + return currentAVPlayer?.timeControlStatus == .playing + } + + /// Takes the injected rendition the engine selected down on the spot. The pinned deselect that + /// follows has to load the legible group first, and a rendition still selected when playback + /// starts is one AVPlayer goes on fetching. + private func deselectInjectedRenditionNow() { + guard let selection = injectedRenditionSelection else { return } + injectedRenditionSelection = nil + guard let item = selection.item, item === currentAVPlayer?.currentItem else { return } + item.select(nil, in: selection.group) + } + + /// Lets AVPlayer draw legible media again. Called right before a selection AVPlayer is meant to + /// draw, and at teardown. A deselect keeps the output: nothing selected draws nothing, and a switch + /// between styled tracks must not uncover the old rendition while the next selection resolves. + func releaseInjectedRenditionSuppression() { + guard let suppression = injectedRenditionSuppression else { return } + injectedRenditionSuppression = nil + suppression.item?.remove(suppression.output) + } + /// Select a remote-HLS legible option by synthetic track id (AE#154). AVPlayer renders the cues /// itself on this bypass; there is no overlay pipeline, so activation only drives /// AVMediaSelection (criteria pinned manual so the explicit choice sticks, #15). @@ -2684,14 +2853,23 @@ extension AetherEngine { let item = currentAVPlayer?.currentItem else { return } cancelNativeLegibleDeselectPin() cancelSidecarTask() + injectedSubtitleSelectionTask?.cancel() + injectedSubtitleSelectionTask = nil subtitleCues = [] isSubtitleActive = true activeSubtitleTrackIndex = id isLoadingSubtitles = false Task { @MainActor in self.currentAVPlayer?.appliesMediaSelectionCriteriaAutomatically = false + // A pick made while the group loaded owns the selection, and may be a hidden ASS + // rendition whose suppression this must not take down. guard let group = try? await item.asset.loadMediaSelectionGroup(for: .legible), - ordinal < group.options.count else { return } + ordinal < group.options.count, + self.currentAVPlayer?.currentItem === item, + self.activeSubtitleTrackIndex == id else { return } + // The origin's own rendition is AVPlayer's to draw. + self.releaseInjectedRenditionSuppression() + self.injectedRenditionSelection = nil item.select(group.options[ordinal], in: group) EngineLog.emit( "[AetherEngine] AE#154: remote-HLS legible select ordinal=\(ordinal) (\(group.options[ordinal].displayName))", @@ -2699,3 +2877,61 @@ extension AetherEngine { } } } + +/// A suppressing legible output and the item it hangs on. Weak, so a replaced item is not kept alive. +struct InjectedRenditionSuppression { + let output: AVPlayerItemLegibleOutput + weak var item: AVPlayerItem? +} + +/// The injected rendition the engine selected and the group it belongs to. +struct InjectedRenditionSelection { + weak var item: AVPlayerItem? + let group: AVMediaSelectionGroup +} + +/// AE#628: one channel's share of a drain tick, everything the decode needs and everything the +/// apply half reads back. Built and consumed on the MainActor; only `decodeHandoff` leaves it. +struct SubtitleDrainChannelWork { + let channel: SubtitleChannel + let streamIndex: Int32 + let plan: SubtitleDrainPlan + let isReset: Bool + let window: (from: Double, through: Double) + let coverageStart: Double? + let retained: SubtitleResolutionStatement.Retention? + let decoder: EmbeddedSubtitleDecoder + let entries: [StoredSubtitlePacket] + let batchEnd: Int + let decodeEnd: Int + let gapHoldAt: Double? + let gapHoldSequence: UInt64 + let gapHoldTicksLeft: Int +} + +struct SubtitleDrainTickWork { + let store: SubtitlePacketStore + let playhead: Double + var channels: [SubtitleDrainChannelWork] = [] + var prefetchNeedsReanchor = false + + var decodeHandoff: SubtitleDrainDecodeHandoff { + SubtitleDrainDecodeHandoff(jobs: channels.map { ($0.decoder, $0.entries[..<$0.decodeEnd]) }) + } +} + +/// AE#628: the decode half of a drain tick, the only part that runs off the MainActor. Unchecked +/// because a decoder wraps an AVCodecContext; sound because a decoder is only ever driven by the +/// one tick in flight (`subtitleDrainTickInFlight`), and a decoder the channel dropped meanwhile is +/// never read again once its batch lands. +struct SubtitleDrainDecodeHandoff: @unchecked Sendable { + let jobs: [(decoder: EmbeddedSubtitleDecoder, packets: ArraySlice)] + + var isEmpty: Bool { jobs.allSatisfy { $0.packets.isEmpty } } + + func decode() -> [[EmbeddedSubtitleDecoder.SubtitleEvent?]] { + jobs.map { job in + job.packets.map { AetherEngine.decodeStoredSubtitlePacket($0, with: job.decoder) } + } + } +} diff --git a/Sources/AetherEngine/AetherEngine+Version.swift b/Sources/AetherEngine/AetherEngine+Version.swift index 843682801..fdf3e4f9f 100644 --- a/Sources/AetherEngine/AetherEngine+Version.swift +++ b/Sources/AetherEngine/AetherEngine+Version.swift @@ -11,5 +11,5 @@ extension AetherEngine { /// /// Between releases, and in a consumer that pins an unreleased commit for a device test, this /// names the last PUBLISHED version the checkout descends from, not every patch in it. - public nonisolated static let version = "7.13.0" + public nonisolated static let version = "7.28.3" } diff --git a/Sources/AetherEngine/AetherEngine.swift b/Sources/AetherEngine/AetherEngine.swift index 03aecd5f4..2ed1bb446 100644 --- a/Sources/AetherEngine/AetherEngine.swift +++ b/Sources/AetherEngine/AetherEngine.swift @@ -61,6 +61,8 @@ public final class AetherEngine: ObservableObject { if case .error = state {} else { errorInfo = nil } recomputePlaybackPhase() resolveLoadingStashedSeek(from: oldValue) + settleOwedBackgroundAction() + if let logTag { EngineLog.emit("[AetherEngine:\(logTag)] state=\(state)", category: .engine) } } } @@ -497,6 +499,10 @@ public final class AetherEngine: ObservableObject { /// software or audio-only load is still activating releases the session after that activation (AE#538). public var deactivatesAudioSessionOnStop: Bool = false + /// Sodalite#175: a short name for this instance ("tile2") carried by the shared-output lines and the + /// state transitions, so a log with several engines in it can be read. nil for a lone engine. + public var logTag: String? + @Published public internal(set) var duration: Double = 0 /// Forwarder; see `clock.progress`. @@ -593,6 +599,22 @@ public final class AetherEngine: ObservableObject { /// different question (a codec has more than one decoder, and the answer changes with hardware). @Published public internal(set) var sourceVideoCodecName: String? = nil + /// AE#658: the source video stream's pixel format, bit depth, colour description and profile as the + /// container and the probe's decoder declared them, nil before load and on sources without video. + /// Companion to `decodedVideoFormat`, which is what the engine's own decoder actually produced. + /// + /// On the probe-free native HLS bypass it is read back from AVPlayer's item video track once that + /// resolves: colour from the format description's extensions, profile, bit depth and pixel format from + /// its avcC / hvcC record where it carries one (nil otherwise, AV1 and VP9 included). That is the + /// DELIVERED stream, which under a server-side transcode is not the file the host's library holds. + @Published public internal(set) var sourceVideoStreamFormat: VideoStreamFormat? = nil + + /// AE#658: the format the engine's software decoder produced and the display buffer it went into, + /// updated when either changes mid-stream. nil whenever the engine does not decode the picture + /// itself, which on the native path is always: AVPlayer decodes there, and its frames never pass + /// through the engine, so `sourceVideoStreamFormat` is the only description a host has. + @Published public internal(set) var decodedVideoFormat: DecodedVideoFormat? = nil + /// Container libavformat opened ("matroska,webm", "mpegts", "mov,mp4,m4a,3gp,3g2,mj2"), nil before load /// and on the native HLS bypass (AVFoundation opens that one, there is no libav context to ask). This is /// the container that ARRIVED: on a remux or transcode session it differs from the one the host's library @@ -628,7 +650,36 @@ public final class AetherEngine: ObservableObject { /// Exposed for diagnostic overlays; hosts should not branch on it. Branch on `videoRoute` instead, /// which also separates the two native pipelines (#321). @Published public internal(set) var playbackBackend: PlaybackBackend = .none { - didSet { recomputeVideoRoute(); recomputeAudioDelivery() } + didSet { recomputeVideoRoute(); recomputeAudioDelivery(); recomputeAirPlayPictureStaysLocal() } + } + + /// True while a wireless AirPlay receiver holds the audio route and the session's picture cannot + /// follow it, so the receiver plays the sound and the picture stays on this device. + /// + /// That is the software host (VP9, AV1 without a hardware decoder, deinterlaced MPEG-2 / VC-1 / + /// H.264, a host's `preferredDecodePath = .software`): it draws into an `AVSampleBufferDisplayLayer`, + /// and only its `AVSampleBufferAudioRenderer` follows the route. Nothing fails, so without this a host + /// has no way to tell the viewer why the TV shows no picture. iOS only; false everywhere else, and + /// false on the native paths, where AVPlayer hands the receiver the stream. + @Published public internal(set) var airPlayPictureStaysLocal = false + + nonisolated static func airPlayPictureStaysLocal(backend: PlaybackBackend, + wirelessAirPlayRoute: Bool) -> Bool { + wirelessAirPlayRoute && backend == .software + } + + /// Idempotent, and logged on each change, so a report shows when the receiver lost the picture. + func recomputeAirPlayPictureStaysLocal() { + let next = Self.airPlayPictureStaysLocal(backend: playbackBackend, + wirelessAirPlayRoute: Self.isWirelessAirPlayRoute()) + guard airPlayPictureStaysLocal != next else { return } + airPlayPictureStaysLocal = next + EngineLog.emit( + next + ? "[AirPlay] wireless receiver holds the audio route, but the software path draws the " + + "picture on this device: the receiver plays sound only" + : "[AirPlay] picture and audio route agree again", + category: .engine) } /// Pipeline actually serving this session (#321), including the reroutes the host never asked for. @@ -685,6 +736,23 @@ public final class AetherEngine: ObservableObject { } } + /// Timing adjustment for subtitles composited into software PiP frames, in media seconds. + /// Positive values delay subtitles; negative values advance them. Host overlays must apply + /// the same value against `clock.sourceTime`. Native AVPlayer renditions are not affected. + /// Persists across loads on this engine, like a host presentation preference. + public private(set) var softwareSubtitleDelaySeconds: Double = 0 + + /// Updates software subtitle composition without reopening media or shifting the A/V clock. + /// Non-finite values are ignored. Takes effect on the next composited video frame. + public func setSoftwareSubtitleDelay(_ seconds: Double) { + guard seconds.isFinite, seconds != softwareSubtitleDelaySeconds else { return } + softwareSubtitleDelaySeconds = seconds + softwareHost?.updateSubtitleCompositor( + cues: softwarePiPSubtitleCues(primary: subtitleCues, secondary: secondarySubtitleCues), + enabled: pictureInPictureActive, delaySeconds: seconds) + EngineLog.emit("[AetherEngine] software subtitle delay=\(seconds)s", category: .engine) + } + /// Master enable for background playback (iOS: PiP + background audio; tvOS: PiP keepalive). Default on. public var backgroundPlaybackEnabled = true /// Set by the host from its PiP delegate (iOS: AVKit; tvOS: host-built AVPictureInPictureController); @@ -695,15 +763,13 @@ public final class AetherEngine: ObservableObject { // window and never double-draw under the fullscreen host overlay. softwareHost?.updateSubtitleCompositor( cues: softwarePiPSubtitleCues(primary: subtitleCues, secondary: secondarySubtitleCues), - enabled: pictureInPictureActive) + enabled: pictureInPictureActive, delaySeconds: softwareSubtitleDelaySeconds) refreshInjectedASSSubtitleRendering() #if os(tvOS) - // PiP window closed while backgrounded: nothing keeps the app running anymore, so run the - // wedge-safe teardown now, before idle suspension (mirrors the iOS pause-while-backgrounded path). - if oldValue && !pictureInPictureActive && isBackgrounded, - !audioAVPlayerActive, audioHost == nil, softwareHost == nil, - state == .playing || state == .paused { - Task { @MainActor in await self.teardownVideoForBackground() } + // PiP window closed while backgrounded: nothing keeps the app running anymore, so the + // wedge-safe teardown is due before idle suspension (mirrors the iOS pause-while-backgrounded path). + if oldValue && !pictureInPictureActive { + settleBackgroundDecisionAfterPictureInPictureClosed() } #endif } @@ -872,11 +938,17 @@ public final class AetherEngine: ObservableObject { /// a fresh layer. A host driving PiP must still set `pictureInPictureActive`, which arms both the /// keepalive policy and that in-place handover. public var nativePlayerLayer: AVPlayerLayer? { nativeHost?.playerLayer } - #if os(iOS) || os(tvOS) /// True between didEnterBackground and didBecomeActive; gates the pause-while-backgrounded teardown - /// (iOS) and the PiP-closed-while-backgrounded teardown (tvOS). + /// (iOS) and the PiP-closed-while-backgrounded teardown (tvOS). Never set on macOS. private var isBackgrounded = false - #endif + /// Audit CORE-2: didEnterBackground (or a background PiP close) ran while a load or seek was in flight. + private var backgroundActionOwed = false + /// Audit LIF-101: the generation of the `load()` or `reloadWithAudioOverride` still running. A + /// load passes through `.paused` long before it returns (native readiness during the play gate), + /// so its state cannot say whether it settled; this can. + private var loadInFlightGeneration: UInt64? + /// Test seam for the owed background decision on a platform without an app lifecycle. + var owedBackgroundPolicyForTesting: (@MainActor (AetherEngine) async -> Void)? #if os(iOS) /// #127: pending grace-window teardown (sleep task + the background-task assertion holding it). private var backgroundGraceTask: Task? @@ -973,7 +1045,8 @@ public final class AetherEngine: ObservableObject { mediaServicesResetPending = true EngineLog.emit( "[AetherEngine] #597 media services were \(lost ? "lost" : "reset"): every audio and " - + "video object this process holds is invalid, so the next load builds a fresh host", + + "video object this process holds is invalid, so the next load builds fresh native and " + + "audio-only players", category: .engine) } @@ -983,6 +1056,21 @@ public final class AetherEngine: ObservableObject { return pending } + /// Audit CORE-4: the audio-only AVPlayer host is kept across every `stopInternal` on purpose (its + /// MPNowPlayingSession must persist between tracks), so after a reset nothing ever replaced it and + /// the next track was handed to a dead AVPlayer. Any load consumes the reset, so it is dropped + /// here whatever that load plays; the next audio load builds a fresh one with a fresh session. + func dropAudioPlayerHostAfterMediaServicesReset() { + guard let host = audioAVPlayerHost else { return } + host.stop() + audioAVPlayerHost = nil + audioAVPlayerActive = false + EngineLog.emit( + "[AetherEngine] #597 audio-only AVPlayer dropped after the media services reset; the next " + + "audio load builds a fresh one", + category: .engine) + } + /// Consume the one-shot host request at the load boundary, folded with PiP's mandatory handover. /// Consumed here rather than where it is read so one episode transition cannot change the /// teardown of a later, unrelated load. @@ -1037,6 +1125,13 @@ public final class AetherEngine: ObservableObject { return .teardownVideo } + /// Audit CORE-2: a state in which `backgroundAction` cannot judge the pipeline yet, although one + /// exists (a load has its loopback server and AVIO connection, a seek its whole session). The + /// decision is owed until the state settles. + nonisolated static func backgroundActionIsOwed(state: PlaybackState) -> Bool { + state == .loading || state == .seeking + } + /// #127: how to execute a BackgroundAction. A PAUSED teardown on platforms with quick app switches /// (iOS) is deferred by a grace window held under a background-task assertion, so a 10-30 s app /// switch does not pay a full pipeline rebuild. Wedge-safety holds: the assertion keeps the app @@ -1137,6 +1232,20 @@ public final class AetherEngine: ObservableObject { /// the embedded side-demuxer reader for every host, VOD and live, text and bitmap. var subtitleDrainTargets: [SubtitleChannel: Int32] = [:] var subtitleDrainerTask: Task? + /// AE#628: the drain tick whose decode is running off the MainActor, nil while none is. At most + /// one at a time, because a channel's decoder is one AVCodecContext and the tick in flight owns + /// it; a tick asked for meanwhile sets `subtitleDrainTickRequested` and runs when it lands. + var subtitleDrainTickInFlight: Task? + var subtitleDrainTickRequested = false + /// AE#628: bumped by `stopSubtitleDrainer`, so a batch still decoding for a stopped drainer + /// lands on a newer serial and is dropped instead of clearing a later tick's in-flight slot. + var subtitleDrainTickSerial: UInt64 = 0 + #if DEBUG + /// AE#628: lets a test drive the drain tick with a real decoder but no playback session. + var subtitleDrainDecoderFactoryForTesting: ((Int32) -> EmbeddedSubtitleDecoder?)? + /// Lets a test select the hidden ASS rendition on an item whose fixture media never plays. + var hiddenRenditionIgnoresPlaybackForTesting = false + #endif /// SW-host sessions have no HLSVideoEngine; their tap fills this store instead. var softwareSubtitlePacketStore: SubtitlePacketStore? var subtitleDrainDecoders: [SubtitleChannel: EmbeddedSubtitleDecoder] = [:] @@ -1284,6 +1393,14 @@ public final class AetherEngine: ObservableObject { /// True for a live session (`LoadOptions.isLive`). Cleared in stopInternal so it can't bleed into the next VOD load. @Published public internal(set) var isLive: Bool = false + /// True while a VOD session is served as a sequential origin: the source can only be read front to + /// back, either because `LoadOptions.sequentialOrigin` declared it or because the origin refused the + /// range form at open (twice, AE#693) and served a plain GET. Such a session plays from the start, + /// so a `startPosition` passed to `load()` is not honoured, and a seek lands only inside what the + /// session has already read. A host that needs the position reopens the source itself, or tells the + /// user. Set during `load()`, before `state` leaves `.loading`; cleared on stop. + @Published public internal(set) var isSequentialOrigin: Bool = false + /// What the session's live recording is doing (AE#560). `.idle` when nothing is recording. /// See `startRecording(to:)`. @Published public internal(set) var recordingState: RecordingState = .idle @@ -1295,6 +1412,16 @@ public final class AetherEngine: ObservableObject { /// producer or software host is owned by its session and may be torn down under us. weak var activeRecordingHost: AnyObject? + /// Which recording a writer failure belongs to (audit CORE-5). Bumped once per writer. + var recordingGeneration: UInt64 = 0 + + /// The ended recording's drain and trailer, run off the main actor (audit REC-1). + var recordingFinish: Task? + + /// Bumped by every `stopRecording()`, so a `startRecording` that was waiting on `recordingFinish` + /// can tell that a stop overtook it (audit FEA-106). + var recordingStopSerial: UInt64 = 0 + /// Republishes `recordingState` progress at 1 Hz while a recording runs. var recordingProgressTimer: Timer? @@ -1334,6 +1461,13 @@ public final class AetherEngine: ObservableObject { /// fired and which of its own tracks matches. No replay; subscribe per session. public let systemCaptionRequest = PassthroughSubject() + /// Fires when a native session AVPlayer refused on its merits is rebuilt on the software path + /// (AE#561), carrying the failure the rebuild absorbed. Without it a host sees only `videoRoute` + /// change, which a host re-planning failing titles itself cannot tell from a routing decision + /// (AE#629). `LoadOptions.escalatesToSoftwarePath = false` declines the rebuild instead. No + /// replay; subscribe per session. + public let softwarePathEscalations = PassthroughSubject() + // MARK: - Scrub thumbnails /// LRU (cap 2) of FrameExtractor contexts for cache-backed scrub thumbnails (live and @@ -1421,9 +1555,10 @@ public final class AetherEngine: ObservableObject { /// where the per-title repetition it was built to prevent would have happened. /// /// Deliberately not keyed to a fingerprint of the display configuration, which would be more - /// precise: the latch can only be set while `eligibleForHDRPlayback` is true, so the capability - /// table at refusal time may be identical to the table afterwards, and a fingerprint built on that - /// assumption would hold the stale latch silently. The return needs no such assumption. + /// precise: the latch is only set while `eligibleForHDRPlayback` reads true (enforced since AE#535, + /// see `MasterFallbackDecision.shouldLatchPanelRefusal`), so the capability table at refusal time + /// may be identical to the table afterwards, and a fingerprint built on that would hold the stale + /// latch silently. The return needs no such assumption. nonisolated static func clearPanelRefusalOnForegroundReturn() { guard panelRefusedHDRMaster else { return } panelRefusedHDRMaster = false @@ -1884,6 +2019,10 @@ public final class AetherEngine: ObservableObject { /// the time `load()` runs they look identical. private var startupContinuesAcrossReroute = false + /// AE#629: generations of the `load()` calls a caller is still awaiting. The software-path + /// rebuild reads it to tell a startup it is taking over from a session that already returned. + var waitingLoadGenerations: Set = [] + /// Open a startup sequence for the load now beginning, and return its generation for the call /// sites that have to record checkpoints from detached work. A reroute continues the sequence /// already in flight instead of opening a new one. @@ -1908,6 +2047,12 @@ public final class AetherEngine: ObservableObject { startupContinuesAcrossReroute = true } + /// Withdraw a continuation that no `load()` consumed, because the reroute was refused before it + /// reached one. Left set, it would fold the host's NEXT load into a startup that already ended. + func abandonStartupContinuation() { + startupContinuesAcrossReroute = false + } + /// Record a checkpoint against a startup generation. `generation` defaults to the running one, /// which is what every main-actor call site wants; detached work and Combine sinks that can /// outlive their load pass the generation they captured, so a straggler cannot write into a @@ -1940,6 +2085,17 @@ public final class AetherEngine: ObservableObject { } } + /// Audit LIF-103, Vcore-101: a session callback's hop onto the main actor, dropped when the session + /// that raised it has ended by the time it lands. Only `stopInternal` bumps `loadGeneration` and it + /// always ends the session, so the generation is the session's identity on every path, the software + /// one included, which has no session object to compare. + nonisolated func hop(for generation: UInt64, _ body: @escaping @Sendable @MainActor () async -> Void) { + Task { @MainActor [weak self] in + guard let self, self.loadGeneration == generation else { return } + await body() + } + } + /// Shift seam history on the item axis. The producer rebases immediately (live program boundary) or starts a /// fresh epoch (VOD restart); AVPlayer renders that content ~buffer+holdback later. The currentTime sink /// resolves the active shift by looking up the newest seam at or before the raw clock (a history, not a @@ -1983,6 +2139,10 @@ public final class AetherEngine: ObservableObject { /// offset by the session zero on live / mid-stream-joined sources (#107). Forwarder; subscribe to `clock.$sourceTime`. public var sourceTime: Double { clock.sourceTime } + /// Whether `sourceTime` is known to follow the displayed frame (AE#616). Forwarder; subscribe to + /// `clock.$sourceTimeFollowsPicture`. + public var sourceTimeFollowsPicture: Bool { clock.sourceTimeFollowsPicture } + /// Source-axis buffer frontier ahead of the playhead (AetherEngine#54). Forwarder; subscribe to `clock.$bufferedPosition`. public var bufferedPosition: Double { clock.bufferedPosition } @@ -2056,8 +2216,20 @@ public final class AetherEngine: ObservableObject { /// Source video dimensions from the probe. Used as a bitmap-subtitle canvas fallback before the first PCS /// is parsed. 0 before load or when source has no video (AetherEngine#28). Also available in SourceProbe. + /// On the probe-free native HLS bypass they are the delivered stream's, read back from AVPlayer's item + /// track once it resolves, so a capped transcode reports the resolution it is actually playing. @Published public private(set) var sourceVideoWidth: Int32 = 0 @Published public private(set) var sourceVideoHeight: Int32 = 0 + + /// The remote-HLS bypass's stand-in for the probe: what AVPlayer parsed of the delivered video. + func publishRemoteHLSVideoDescription(_ video: RemoteHLSStreamDescription.Video) { + if video.width > 0, video.height > 0 { + sourceVideoWidth = video.width + sourceVideoHeight = video.height + } + if let codec = video.codecName { sourceVideoCodecName = codec } + sourceVideoStreamFormat = video.format + } /// Display-width multiplier for non-square source pixels: `sourceVideoWidth * this` is the width /// the picture presents at. 1 before load, on square-pixel sources, and whenever the declared /// ratio is one the engine refuses to believe (#290), so it is never a number the picture @@ -2170,11 +2342,24 @@ public final class AetherEngine: ObservableObject { /// Cancelable while subtitle playlist preflight is suspended in a host authorizer. var remoteHLSPreparationTask: Task? + /// AE#616: measures how far AVPlayer's item time leads the presented frame on the bypass, off the + /// injected renditions. Nil whenever `remoteHLSSubtitleProxy` serves none. + var remoteHLSCueClock: RemoteHLSCueClockObserver? + + /// AE#616: the last offset `remoteHLSCueClock` measured, subtracted from item time to publish + /// `sourceTime`. Zero on every other path and until the first injected line is presented. + var remoteHLSItemOffset: Double = 0 + /// #316: external track id -> the NAME its injected rendition carries in the served master. Selecting /// one of these must drive AVMediaSelection, not the sidecar overlay, or the two draw on top of /// each other. Empty when no proxy is standing. var injectedSubtitleRenditionNames: [Int: String] = [:] var injectedSubtitleSelectionTask: Task? + /// The suppressing legible output behind which an overlay-drawn ASS track keeps its injected + /// rendition selected, so AE#616 goes on measuring. Nil whenever AVPlayer may draw legible media. + var injectedRenditionSuppression: InjectedRenditionSuppression? + /// What `deselectInjectedRenditionNow` takes down. Nil once anything else owns the selection. + var injectedRenditionSelection: InjectedRenditionSelection? /// Deferred lazy-reader start while a producer restart is in flight (#93 residual): the /// readers' side demuxer competed with the restart for the starved link. Cancelled by @@ -2279,6 +2464,19 @@ public final class AetherEngine: ObservableObject { /// the media. Replaced (not reset) on teardown, so a closure held by a dead host cannot spend /// the next session's. var softwarePathEscalationBudget = SoftwarePathEscalation.Budget() + + /// AE#629: the engine's own rebuild in flight or finished, kept so a `load()` whose startup it + /// superseded can wait for it instead of throwing. Matched by generation, so a stale one is inert. + var softwarePathTakeover: SoftwarePathEscalation.Takeover? + /// The engine's own rebuild most recently started, and the generation it is armed to take over + /// until its teardown claims it (see `claimSoftwarePathTakeover`). + var softwarePathRebuild: Task? + var softwarePathTakeoverArm: UInt64? + /// AE#641: the live audio streams whose bridge decoded nothing in this session. They outlive the + /// session's own rebuilds, which is what keeps them video-only, and a host load clears them. A set, + /// not a slot (audit FEA-103): a channel with two undecodable tracks would otherwise re-hang on the + /// second one, after the viewer's pick, with nothing surfaced. + var undecodableLiveAudioStreamIndices: Set = [] /// #65 final rung, storm shape: on a frozen live playlist each stage-2 reload replays the tail, /// re-stalls within seconds, and the fresh stall SUPERSEDES the ladder task before its /// post-reload rung can run, so the reload cycle alone would loop forever. This gate persists @@ -2444,7 +2642,9 @@ public final class AetherEngine: ObservableObject { /// React to AVPlayer rejecting the served master (#98, #130): if eligible, reload the media /// playlist in place (single-variant); otherwise surface the failure normally. @MainActor - func fallBackToMediaPlaylist(_ rejection: DisplayRejection) { + func fallBackToMediaPlaylist(_ rejection: DisplayRejection, expectedGeneration: UInt64) { + // Audit LIF-103: a rejection of an ended session's item is not the running session's to act on. + guard loadGeneration == expectedGeneration else { return } guard let host = nativeHost, let session = nativeVideoSession else { publishError(PlaybackErrorInfo(kind: .masterPlaylistRejected, message: rejection.message, underlyingDomain: rejection.domain, underlyingCode: rejection.code)) return @@ -2459,12 +2659,25 @@ public final class AetherEngine: ObservableObject { } masterFallbackUsed = true // AE#459: the display answered. Display-rejection codes only, never the -1002 parse failure. + // AE#535: and only while the display is eligible for HDR, or the answer is about the window. if MasterFallbackDecision.isDisplayRejectionCode(rejection.code), !Self.panelRefusedHDRMaster { - Self.panelRefusedHDRMaster = true - EngineLog.emit( - "[DisplayCriteria] panel refused an HDR master (code=\(rejection.code)); this process " - + "routes HDR sources media-direct until it restarts", - category: .engine) + let eligibleNow = AVPlayer.eligibleForHDRPlayback + let switching = displayCriteria.displayModeSwitchInProgress + if MasterFallbackDecision.shouldLatchPanelRefusal( + code: rejection.code, displayEligibleForHDRNow: eligibleNow, + displaySwitchInProgress: switching) { + Self.panelRefusedHDRMaster = true + EngineLog.emit( + "[DisplayCriteria] panel refused an HDR master (code=\(rejection.code)); this process " + + "routes HDR sources media-direct until it returns from the background (#588)", + category: .engine) + } else { + EngineLog.emit( + "[DisplayCriteria] HDR master refused (code=\(rejection.code)) while the display reads " + + "hdrEligible=\(eligibleNow ? "yes" : "no") switching=\(switching ? "yes" : "no") " + + "(#535, #667); this item falls back, the process does not latch the refusal", + category: .engine) + } } session.markServingMediaAfterFallback() nativeSubtitleRenditionsServed = false @@ -2479,26 +2692,17 @@ public final class AetherEngine: ObservableObject { // started: the #93/#65 stage-2 recovery swaps a fresh item in at the position it held, and a // rejection of THAT item has to come back there, not rewind to the first mount. let position = host.mountedStartPosition ?? 0 - // Read before the swap, which resets what it reads. - let resumesPlaying = host.mediaFallbackResumesPlaying() EngineLog.emit( "[AetherEngine] AVPlayer rejected the master (code=\(rejection.code)); falling back to " + "media playlist (no CC/subtitle renditions) at " - + (isLive ? "the live edge" : "\(String(format: "%.2f", position))s") - + (resumesPlaying ? "" : ", staying paused for the viewer"), + + (isLive ? "the live edge" : "\(String(format: "%.2f", position))s"), category: .session) host.swapItem(url: fallbackURL, startPosition: isLive ? nil : position, skipInitialSeek: LiveReloadPolicy.skipInitialSeek(isLive: isLive, isRejoin: true)) - // Resume only a viewer who was playing, read from both the engine's intent and AVPlayer's - // rate. A paused title refused behind the tvOS screensaver used to start itself and wake it. - if resumesPlaying { - host.play() - } else { - // Clears the intent latch a pause from AVKit, Control Center or PiP left set, so the fresh - // item's readyToPlay does not re-assert play() behind the viewer. - host.pause() - } + // Resume only a viewer who was playing; the host's intent (#122) survives the in-place swap. + // A paused title refused behind the tvOS screensaver used to start itself and wake it. + if host.transportIntentIsPlaying { host.play() } } /// #35 readiness-gate settle windows. Generous enough that a slow-but-healthy cold start reads as @@ -2567,12 +2771,21 @@ public final class AetherEngine: ObservableObject { /// survived cut `seg0+` on a title 15 s in. While a load is in flight the position that describes /// the session is the one THAT load was handed, which it received before the clock was cleared. /// - /// `.loading` is the whole of that window and nothing else: the only other writer of it holds it - /// through startup before the first roll (`host.$timeControlStatus`, which cannot reach it once - /// the session has played), so the parked value can never be read after the load it belongs to. + /// Round 5 (cmcpherson274, E8-F4): `.loading` was NOT the whole window. The autostart at the tail + /// of `load()` writes `.playing` before the new host has published a position, so for the next + /// ~50 ms the clock still reads the zero `load()` wrote, and a correction raised the moment a + /// rebuild returned rebuilt at the head. `setAudioDelay`'s own catch-up pass sits exactly there, + /// and so did a second stepper press 50-90 ms after the first on `.loopback` (measured on the CLI: + /// `#3 mount seek: item axis 0.00s`, `cutting seg0+`). A playable session whose clock reads exactly + /// the reset zero has not published yet, and the parked value is what it is mounted at. A seek + /// retires the parked value (`seek(to:origin:)`), so a genuine seek to 0 is never overridden. nonisolated static func rebuildPosition(state: PlaybackState, clock: Double, underReconstruction: Double?) -> Double { - guard state == .loading, let parked = underReconstruction else { return clock } - return parked + guard let parked = underReconstruction else { return clock } + switch state { + case .loading: return parked + case .playing, .paused: return clock == 0 ? parked : clock + case .idle, .seeking, .ended, .error: return clock + } } /// #227 round 2: whether a finishing session-preserving rebuild is the one that owns the held @@ -2839,12 +3052,7 @@ public final class AetherEngine: ObservableObject { /// `LiveReloadPolicy.recoveryRejoinPosition` cannot. A window closed with ENDLIST is a finite asset /// whose seekable end IS the playhead, so the distance-behind-live that policy reads is zero and it /// would aim at the edge, discarding the rewind the viewer kept through the whole outage. - /// - Parameter resumesPlaying: false when the viewer wants the item paused (see - /// `NativeAVPlayerHost.itemDeathReloadResumesPlaying`). The pause guard bypass admits the dead - /// item; it must not also overrule the viewer, so the fresh item mounts paused at the anchor - /// and the next Play resumes there. func reloadStalledConsumerItem(position: Double, allowPausedConsumer: Bool = false, - resumesPlaying: Bool = true, liveRejoinOverride: Double? = nil) { guard let host = nativeHost, let player = currentAVPlayer, let url = (player.currentItem?.asset as? AVURLAsset)?.url else { return } @@ -2887,7 +3095,7 @@ public final class AetherEngine: ObservableObject { + Self.recoveryAnchorLogSuffix( anchor: anchor, position: position, pendingSeekTarget: pendingRecoverySeekClockTarget) - + " (same URL, same host" + (resumesPlaying ? ")" : ", staying paused for the viewer)"), + + " (same URL, same host)", category: .engine ) // AE#454: the placement, expressed in the playlist the fresh item is about to load. A rejoin @@ -2927,13 +3135,11 @@ public final class AetherEngine: ObservableObject { // AE#454 round 2: the item that is about to load is the one the placement was armed for, and // the only one whose axis the playlist will state. if didArmPlacement { liveRejoinPlacementGeneration = host.itemGeneration } - if resumesPlaying { - host.play() - } else { - // Clears the intent latch a pause from AVKit, Control Center or PiP left set, so the fresh - // item's readyToPlay does not re-assert play() behind the viewer. - host.pause() - } + // Item death parks AVPlayer at .paused whatever the viewer wanted, which is why this reload may + // run for a paused consumer at all; resuming is still the viewer's call, read from the host's + // intent (#122), which the in-place swap keeps. Playing a paused title here woke the tvOS + // screensaver its item had died behind. + if host.transportIntentIsPlaying { host.play() } if let rejoinPosition { // Stashed rather than seeked: the pre-readiness seek IS the wedge LiveReloadPolicy exists // to avoid, and a live seek does not defer itself (`shouldDeferHostSeek` excludes live), so @@ -3260,15 +3466,26 @@ public final class AetherEngine: ObservableObject { /// AE#464 round 2: the position the load currently in flight was handed, parked across the window /// in which `load` has already zeroed the clock but the rebuilt session has not reached it yet. - /// Written at the two sites that raise `state = .loading` for a load; read only through - /// `positionForSessionRebuild`, which is what makes a reload stacked behind another one rebuild at - /// the playhead instead of at the head. + /// Written at the two sites that raise `state = .loading` for a load, retired by an accepted seek; + /// read only through `positionForSessionRebuild`, which is what makes a reload stacked behind + /// another one, or raised the moment one returned (round 5), rebuild at the playhead instead of at + /// the head. var positionUnderReconstruction: Double? /// AE#464 round 3: the transport intent the load in flight was handed, parked across the same /// window as `positionUnderReconstruction` and for the same reason. Read only through /// `sessionRebuildResumesPlaying`. See `rebuildResumesPlaying`. var transportIntentUnderReconstruction: Bool? + private(set) var audioSelectionTask: Task? + private var pendingAudioSelection: Int? + private var audioSelectionEpoch = UUID() + + private func cancelPendingAudioSelection() { + audioSelectionEpoch = UUID() + audioSelectionTask?.cancel() + audioSelectionTask = nil + pendingAudioSelection = nil + } /// AE#464 round 3: true while a re-anchor raised by `setAudioDelay` is running, so the presses /// that arrive during it are folded into it instead of stacking re-anchors of their own. The @@ -3284,6 +3501,15 @@ public final class AetherEngine: ObservableObject { /// The transport state a rebuild of this session has to come back in (#464 round 2). Asks the /// native host for its durable intent where there is one to ask, exactly as `togglePlayPause` /// does, and falls back to `state` on the routes that have no competing transport owner. + /// AE#711 follow-up: a play/pause that arrives while a session-preserving rebuild runs is the + /// transport that rebuild has to come back in. Recorded on the flag both rebuild routes raise + /// rather than on `.loading`, because `play()` itself moves `.loading` to `.playing` and a pause + /// after it would otherwise go unrecorded. Read back where each rebuild settles its transport. + func recordTransportDuringRebuild(playing: Bool) { + guard sessionPreservingReloadInFlight else { return } + transportIntentUnderReconstruction = playing + } + var sessionRebuildResumesPlaying: Bool { let nativeIntent = (nativeHost != nil && !audioAVPlayerActive && audioHost == nil && softwareHost == nil) ? nativeHost?.transportIntentIsPlaying @@ -3296,6 +3522,10 @@ public final class AetherEngine: ObservableObject { /// itself. Claimed by `consumeReloadSelection`, dropped by any other `load()` and by `stop()`. var backgroundTeardownSelection: BackgroundTeardownSelection? + /// The background policy released the video item while retaining the source, playhead, + /// selection and native host. A foreground host must rebuild before sending play(). + public var needsForegroundVideoRestore: Bool { backgroundTeardownSelection != nil } + /// Detached reader that decodes ALL embedded text subtitle streams in one side-demuxer pass into their /// ordinal's NativeSubtitleCueStore (#55, all-tracks). Parallel to the packet-store drainer (which drives /// subtitleCues for the active track with full styling). Cancelled on stop/clear/load. @@ -3403,9 +3633,6 @@ public final class AetherEngine: ObservableObject { private var audioSessionDeactivationTask: Task? #endif - /// The most recent off-main activation or release of the shared session. See `enqueueAudioSessionTransition`. - private var audioSessionTransition: Task? - /// Run a session activation or release off the main actor, after every one asked for before it. /// /// Since AE#538 both halves are detached tasks, and two detached tasks carry no order between them. @@ -3414,14 +3641,9 @@ public final class AetherEngine: ObservableObject { /// audio-only load's activation can send `setActive(false)` first and leave the session active after a /// final teardown. Each transition awaits its predecessor, including a cancelled one, which then drops /// on its own guard. Not platform-gated, so the order is testable where the session does not exist. + /// Since Sodalite#175 the queue is the process-wide one in `SharedOutputCoordinator`. func enqueueAudioSessionTransition(_ body: @escaping @Sendable () async -> Void) -> Task { - let previous = audioSessionTransition - let transition = Task.detached(priority: .userInitiated) { - await previous?.value - await body() - } - audioSessionTransition = transition - return transition + SharedOutputCoordinator.shared.enqueueTransition(body) } #if os(iOS) || os(tvOS) @@ -3607,9 +3829,126 @@ public final class AetherEngine: ObservableObject { options: LoadOptions = .init(), audioSourceStreamIndex: Int32? = nil, discTitleID: Int? = nil + ) async throws -> SourceProbe? { + // Sodalite#173: a task cancelled before the load began has nothing to end; the running session stays. + // A reroute's #361 continuation armed for this load is withdrawn with it. + if Task.isCancelled { + abandonStartupContinuation() + throw CancellationError() + } + let attempt = LoadAttempt() + defer { if let gen = attempt.generation { waitingLoadGenerations.remove(gen) } } + do { + let probe = try await withTaskCancellationHandler { + try await loadSession( + source: source, startPosition: startPosition, options: options, + audioSourceStreamIndex: audioSourceStreamIndex, discTitleID: discTitleID, + attempt: attempt) + } onCancel: { + Task { @MainActor [weak self] in self?.abandonCancelledLoad(attempt) } + } + if Task.isCancelled, abandonCancelledLoad(attempt) { throw CancellationError() } + return probe + } catch is CancellationError { + // AE#629: the engine took this startup over itself (the AE#561 / AE#641 rebuild), so the caller is + // still waiting for the same thing and gets it, the way a #361 reroute keeps its wait. A + // load the HOST superseded matches no takeover and unwinds as before. + guard let generation = attempt.generation, + let takeover = softwarePathTakeover, + takeover.supersededGeneration == generation else { + if Task.isCancelled { abandonCancelledLoad(attempt) } + throw CancellationError() + } + EngineLog.emit( + "[AetherEngine] #629 load (gen \(generation)) follows the engine's own rebuild " + + "instead of unwinding", category: .engine) + // Sodalite#173: a cancelled follower cancels the rebuild's Task, which ends a load() rebuild and + // any reroute nested in it through their own handlers, and ends the rebuild's generation, which + // the retained-reader branch observes only at its checkpoints. Both are generation-guarded. + do { + try await withTaskCancellationHandler { + try await takeover.rebuild.value + } onCancel: { + takeover.rebuild.cancel() + Task { @MainActor [weak self] in self?.abandonFollowedRebuild(takeover, follower: attempt) } + } + } catch { + if Task.isCancelled { throw CancellationError() } + throw error + } + if Task.isCancelled { + abandonFollowedRebuild(takeover, follower: attempt) + throw CancellationError() + } + return attempt.probe + } catch { + guard Task.isCancelled else { throw error } + abandonCancelledLoad(attempt) + throw CancellationError() + } + } + + /// What `loadSession` tells the public `load` about itself: the generation it ran under and the + /// probe it assembled, which a load followed across a takeover still returns (AE#629). + @MainActor + final class LoadAttempt { + var generation: UInt64? + var probe: SourceProbe? + } + + /// Sodalite#173: the caller cancelled the task awaiting this load. Ends it only while it is still the + /// load in flight: a newer load, or a session this one already returned, is not its to end. + @discardableResult + func abandonCancelledLoad(_ attempt: LoadAttempt) -> Bool { + guard let gen = attempt.generation, loadGeneration == gen, + waitingLoadGenerations.contains(gen) else { return false } + endCancelledGeneration(gen) + return true + } + + /// Sodalite#173: the same for a load following an AE#629 rebuild, whose generation is the one the + /// rebuild's teardown opened. Only while the follower still waits and nothing newer took over. + func abandonFollowedRebuild(_ takeover: SoftwarePathEscalation.Takeover, follower: LoadAttempt) { + guard let followed = follower.generation, waitingLoadGenerations.contains(followed), + loadGeneration == takeover.rebuildGeneration else { return } + endCancelledGeneration(takeover.rebuildGeneration) + } + + /// Leaves the engine the way the next `load()`'s teardown would: the generation moves on, every + /// blocking open is aborted and the custom reader closed, while the native host (#15) and an AE#158 + /// handover item are kept for that load and the panel keeps its mode. `state` reads `.idle`; a host + /// that is leaving playback calls `stop()` to release the rest. + private func endCancelledGeneration(_ gen: UInt64) { + EngineLog.emit( + "[AetherEngine] Sodalite#173: load (gen \(gen)) cancelled by its caller; ending it", + category: .engine) + let keepNativeHost = Self.shouldPreserveNativeHostAcrossLoad( + backend: playbackBackend, nativeHostSurvives: nativeHost != nil, + mediaServicesWereReset: mediaServicesResetPending) + stopInternal(resetDisplayCriteria: false, keepNativeHost: keepNativeHost, + keepCurrentItem: keepNativeHost && pendingInPlaceItemHandover) + pendingInPlaceItemHandover = false + state = .idle + startupProgress = nil + clock.currentTime = 0 + clock.bufferedPosition = 0 + clock.progress = 0 + loadedURL = nil + isCustomSource = false + } + + private func loadSession( + source: MediaSource, + startPosition: Double?, + options: LoadOptions, + audioSourceStreamIndex: Int32?, + discTitleID: Int?, + attempt: LoadAttempt ) async throws -> SourceProbe? { var source = source var options = options + options = Self.applyingSharedOutputRole(options) + SharedOutputCoordinator.shared.join(ObjectIdentifier(self), role: options.sharedOutputRole, tag: logTag, owner: self) // #436: a speed the host set belongs to the item it was set on. The rebuilds a session makes // on its own (reload at position, audio-track switch, AirPlay LAN swap, background return) // reopen the same source and keep it; a different item starts at 1.0, so a host whose speed @@ -3640,21 +3979,25 @@ public final class AetherEngine: ObservableObject { // registration survives the seam (issue #15). Captured before stopInternal resets playbackBackend; // the SW dispatch branch releases it if this source routes software. let priorBackendWasNative = (playbackBackend == .native) + let mediaServicesWereReset = consumeMediaServicesReset() let preserveNativeHost = Self.shouldPreserveNativeHostAcrossLoad( backend: playbackBackend, nativeHostSurvives: nativeHost != nil, - mediaServicesWereReset: consumeMediaServicesReset()) + mediaServicesWereReset: mediaServicesWereReset) // AE#158: while a PiP window is live, or when the host asked for it, the running item must // survive this load's teardown; the loopback host.load callsite finishes the handover // (inPlaceSwap). The host request is consumed here so it can affect only this load. The // RUNNING session is the right question here, unlike the host above: a background teardown // has already unloaded the item, so there is nothing left to hand over in place. let handOverInPlace = consumeInPlaceItemHandoverRequest(priorBackendWasNative: priorBackendWasNative) + cancelPendingAudioSelection() pendingInPlaceItemHandover = handOverInPlace // #128 follow-up: preserve the previous session's display criteria across the load seam. Nil-ing it // here bounces the panel through SDR before apply() re-negotiates the same mode on video->video // reloads. Sessions that never reach apply() clear a stale criteria via loadDisplayCriteriaAction // (audio-only fast path, suppressed hosts); a load() that throws before routing leaves it for stop(). + claimSoftwarePathTakeover() // AE#629 stopInternal(resetDisplayCriteria: false, keepNativeHost: preserveNativeHost, keepCurrentItem: handOverInPlace) + if mediaServicesWereReset { dropAudioPlayerHostAfterMediaServicesReset() } // #35/#93: a genuinely new item has not rendered yet; re-arm the cold-startup wedge suspension. // Scrub/seek/producer-restart never route through load(), so mid-stream #93 detection stays armed. hasRenderedFirstFrameMirror.set(false) @@ -3668,6 +4011,11 @@ public final class AetherEngine: ObservableObject { DiscReader.clearCache() // Capture generation; every suspension point re-checks for supersession. let gen = loadGeneration + attempt.generation = gen + waitingLoadGenerations.insert(gen) + // Audit LIF-101: an owed background decision is settled from here, not from a waypoint. + beginLoadInFlight(gen) + defer { endLoadInFlight(gen) } // #361: open the host-visible startup sequence. A reroute re-entering load() continues the // one already running rather than starting a second. let startupGen = beginStartupProgress() @@ -3692,12 +4040,17 @@ public final class AetherEngine: ObservableObject { // later reopen of the same source are bound by it too. if !isCustomSource { OriginRequestBudget.shared.setHostLimit(options.maxConcurrentSourceRequests, for: url) + } else if let ingest = customReader as? HLSLiveIngestReader { + // AE#678: the ingest fetches from the playlist's origin, not from `aether-custom://`. + OriginRequestBudget.shared.setHostLimit( + options.maxConcurrentSourceRequests, for: ingest.budgetOriginURL) } // #170: the carryover is consumed by THIS load only (registration site below, or never on // the branches that return before it); it must not persist into loadedOptions where a later // host-initiated reload would resurrect a stale session snapshot. loadedOptions.subtitleSessionCarryover = nil isLive = options.isLive + isSequentialOrigin = !options.isLive && options.sequentialOrigin // nativeRemoteHLS: DVR window is unbounded (AVPlayer clamps seeks to its real seekable range); // an over-wide published bound only affects range width, not seek landing. liveWindow = options.isLive @@ -3748,6 +4101,9 @@ public final class AetherEngine: ObservableObject { nativeSubtitleRenderingRequested = false injectedSubtitleSelectionTask?.cancel() injectedSubtitleSelectionTask = nil + releaseInjectedRenditionSuppression() + injectedRenditionSelection = nil + detachRemoteHLSCueClock() // AE#616 stallRecoveryWindowUntil = .distantPast stallRecoveryReasserts = 0 stallReengageTask?.cancel() @@ -3757,6 +4113,8 @@ public final class AetherEngine: ObservableObject { itemDeathReviveGate = ItemDeathReviveGate(maxAttempts: 3) stallReloadReviveGate = ItemDeathReviveGate(maxAttempts: 2) softwarePathEscalationBudget = SoftwarePathEscalation.Budget() + undecodableLiveAudioStreamIndices = Self.undecodableAudioStreamIndicesAcrossLoad( + undecodableLiveAudioStreamIndices, sessionPreservingReload: sessionPreservingReloadInFlight) masterFallbackUsed = false nativeSubtitleReanchorTask?.cancel() nativeSubtitleReanchorTask = nil @@ -3786,6 +4144,8 @@ public final class AetherEngine: ObservableObject { sessionObservedDisplayCaps = nil sourceVideoFrameRate = nil sourceVideoBitrate = 0 + sourceVideoStreamFormat = nil + decodedVideoFormat = nil sourceVideoCodecName = nil sourceContainerFormat = nil sourceVideoWidth = 0 @@ -3796,6 +4156,9 @@ public final class AetherEngine: ObservableObject { // below can activate the session: AVKit on the native/remote-HLS paths, activateRendererAudioSession() // on the SW and audio paths. The task is short and typically already complete, so this rarely suspends. await awaitAudioSessionCategoryConfigured() + // Audit CORE-6: the first suspension point. Two loads issued back to back right after init + // both wait on the same category task, and the older one must not resume as the current one. + try checkLoadCurrent(gen) // nativeRemoteHLS: skip probe + loopback; play HLS URL directly with AVPlayer (Jellyfin already serves HLS). // Routed before the probe because we never demux the m3u8. @@ -3823,10 +4186,13 @@ public final class AetherEngine: ObservableObject { // Live keeps the no-initial-seek contract every live caller relies on, so its anchor // stays nil even when the host passes one. try await loadRemoteHLS(url: url, options: options, - startPosition: options.isLive ? nil : startPosition) + startPosition: options.isLive ? nil : startPosition, + generation: gen) } catch is CancellationError { throw CancellationError() } catch { + // A failure that lands after a successor took over is not this session's to publish. + if loadGeneration != gen { throw CancellationError() } // Without this catch, a throwing loadRemoteHLS would strand state at .loading forever. publishError(.sourceOpenFailed, "Failed to load: \(error.localizedDescription)", underlying: error) throw error @@ -3843,6 +4209,7 @@ public final class AetherEngine: ObservableObject { var detectedDVBLCompatIDNum: Int? = nil var detectedRate: Double? = nil var detectedVideoBitrate: Int64 = 0 + var detectedVideoStreamFormat: VideoStreamFormat? = nil var detectedDVProfile: Bool = false // `dolbyVisionHandling = .baseLayerOnly` resolved against this source: the two halves the // format clamp, the criteria request and the software-path guard below all read. @@ -3857,6 +4224,10 @@ public final class AetherEngine: ObservableObject { var probedAudioTracks: [TrackInfo] = [] var probedSubtitleTracks: [TrackInfo] = [] var probedDefaultAudioIndex: Int32 = -1 + // Audit LIF-107: published only after the post-probe generation check below. The AE#532 audit + // is a second read `stopInternal` cannot abort, so a superseded load can return from it after + // its successor's prologue and would write this source's geometry over the reset. + var probedGeometry: (width: Int32, height: Int32, pixelAspect: Double?)? // AE#493: what this session takes the display to be. The observed table is what the platform // could answer; the session table adds what the host asserted, because Dolby Vision has no // public capability API on macOS and eligibility deliberately does not claim it. Composed once, @@ -3874,6 +4245,26 @@ public final class AetherEngine: ObservableObject { + (options.panelPresentsDolbyVision ? " (session dv asserted true)" : ""), category: .session ) + if options.isLive, !options.nativeRemoteHLS, case .url(let livePlaylistURL) = source, + RemoteHLSMediaSelection.isKnownLivePlaylist(livePlaylistURL) { + EngineLog.emit( + "[AetherEngine] AE#678: known HLS playlist URL on the live path; straight onto the " + + "live-ingest reader, no raw probe", + category: .engine + ) + continueStartupAcrossReroute() + return try await load( + source: .custom( + HLSLiveIngestReader(playlistURL: livePlaylistURL, + httpHeaders: loadedOptions.httpHeaders), + formatHint: "mpegts" + ), + startPosition: startPosition, + options: loadedOptions, + audioSourceStreamIndex: audioSourceStreamIndex, + discTitleID: discTitleID + ) + } let probe = Demuxer() // Register so stopInternal can markClosed(): avformat_open_input/find_stream_info can block for the // full AVIOReader reconnect budget (device repro: a 500-looping channel kept reconnecting across three @@ -3983,18 +4374,17 @@ public final class AetherEngine: ObservableObject { // The base-layer route is the one exception: it serves hvc1 and asks for hvc1. detectedDVProfile = (detectedFormat == .dolbyVision) && !presentsDolbyVisionBaseLayer detectedFieldOrder = stream.pointee.codecpar.pointee.field_order - sourceVideoWidth = stream.pointee.codecpar.pointee.width - sourceVideoHeight = stream.pointee.codecpar.pointee.height - if let sar = PixelAspectPolicy.declaredPixelAspect( + let width = stream.pointee.codecpar.pointee.width + let height = stream.pointee.codecpar.pointee.height + let sar = PixelAspectPolicy.declaredPixelAspect( bitstream: stream.pointee.codecpar.pointee.sample_aspect_ratio, container: stream.pointee.sample_aspect_ratio, - width: sourceVideoWidth, - height: sourceVideoHeight - ) { - sourceVideoPixelAspectRatio = Double(sar.num) / Double(sar.den) - } + width: width, + height: height + ) + probedGeometry = (width, height, sar.map { Double($0.num) / Double($0.den) }) detectedVideoBitrate = probe.declaredBitrate(stream: stream) - lastDetectedVideoCodec = detectedCodecID + detectedVideoStreamFormat = VideoStreamFormat(codecpar: stream.pointee.codecpar) } probedAudioTracks = probe.audioTrackInfos() probedSubtitleTracks = probe.subtitleTrackInfos() @@ -4014,6 +4404,12 @@ public final class AetherEngine: ObservableObject { } try checkLoadCurrent(gen) } + if let probedGeometry { + sourceVideoWidth = probedGeometry.width + sourceVideoHeight = probedGeometry.height + if let pixelAspect = probedGeometry.pixelAspect { sourceVideoPixelAspectRatio = pixelAspect } + lastDetectedVideoCodec = detectedCodecID + } // Custom sources have no URL to reopen from: a failed probe is fatal. if case .custom = source, !probeOpened { @@ -4037,6 +4433,7 @@ public final class AetherEngine: ObservableObject { + "live-ingest reader (headers ride every fetch)", category: .engine ) + RemoteHLSMediaSelection.noteLivePlaylist(livePlaylistURL) // #361: the host is still waiting for the load it asked for, so this is the same startup // taking a different route, not a second one. continueStartupAcrossReroute() @@ -4084,10 +4481,12 @@ public final class AetherEngine: ObservableObject { registerDeclaredExternalSubtitles(loadedOptions) recordStartupCheckpoint(.routed, generation: startupGen) // #361 do { - try await loadRemoteHLS(url: hlsURL, options: loadedOptions, startPosition: startPosition) + try await loadRemoteHLS(url: hlsURL, options: loadedOptions, startPosition: startPosition, + generation: gen) } catch is CancellationError { throw CancellationError() } catch { + if loadGeneration != gen { throw CancellationError() } publishError(.sourceOpenFailed, "Failed to load: \(error.localizedDescription)", underlying: error) throw error } @@ -4113,6 +4512,7 @@ public final class AetherEngine: ObservableObject { sourceDolbyVisionRPUProfile = detectedDVRPUProfile sourceVideoFrameRate = detectedRate sourceVideoBitrate = detectedVideoBitrate + sourceVideoStreamFormat = detectedVideoStreamFormat sourceVideoCodecName = detectedCodecID == AV_CODEC_ID_NONE ? nil : avcodec_get_name(detectedCodecID).map { String(cString: $0) } @@ -4146,6 +4546,7 @@ public final class AetherEngine: ObservableObject { let sourceProbe: SourceProbe? = probeOpened ? Self.makeSourceProbe(demuxer: probe, displayURL: url) : nil + attempt.probe = sourceProbe // Resolve the initial audio track: an explicit host override wins, else the ordered language // preference (#72) resolved from this single probe. selectedAudio is nil when neither applies, // so the session keeps its own default pick (empty preferences + no override is a behavioural @@ -4176,7 +4577,7 @@ public final class AetherEngine: ObservableObject { // apply(), so clear a criteria the previous video session left applied. The engine is a // process-wide singleton; without this, music playback keeps the panel in DV/HDR. if Self.loadDisplayCriteriaAction(suppressDisplayCriteria: options.suppressDisplayCriteria, audioOnlyPath: true) == .clearStale { - displayCriteria.reset() + resetDisplayCriteriaRespectingOthers() } // Read codec before closing the probe; custom sources always use FFmpeg (AVPlayer can't consume a custom demuxer). let audioCodecID: AVCodecID = (probeOpened && resolvedInitialAudio >= 0) @@ -4234,6 +4635,7 @@ public final class AetherEngine: ObservableObject { // Superseded: successor owns state. throw CancellationError() } catch { + if loadGeneration != gen { throw CancellationError() } publishLoadFailure(error) throw error } @@ -4273,7 +4675,8 @@ public final class AetherEngine: ObservableObject { didSwitchPanel = true // #339: consumesRecord: false, the play gate after loadNative is entitled to the same // start/end timestamps; spending them here made it pay Stage 1's grace for a settled switch. - await displayCriteria.waitForSwitch(consumesRecord: false) + await displayCriteria.waitForSwitch(consumesRecord: false, + isCurrent: { self.loadGeneration == gen }) // Superseded during panel handshake: close local probe and unwind. if loadGeneration != gen { probe.markClosed() @@ -4291,7 +4694,7 @@ public final class AetherEngine: ObservableObject { // Suppressed host: the load seam preserved the criteria (#128 follow-up), and AVKit writes its // own from the AVPlayerItem formatDescription later. Clear a leftover engine criteria now // (didApply-gated no-op for hosts that always suppress) so the two writers can't fight. - displayCriteria.reset() + resetDisplayCriteriaRespectingOthers() // #339: AVKit's write lands inside loadNative, so the observation has to be armed before the // load rather than when the play gate opens. After reset(), so a switch back to the default // mode is not recorded as this session's. Audio-only loads reach clearStale too and have no @@ -4315,20 +4718,47 @@ public final class AetherEngine: ObservableObject { // where criteria are suppressed. The readout answers only around a dynamic-range transition, // so a panel parked in HDR never proves itself and one tvOS 27 box stopped answering at all; // a host that knows better says so, and a wrong claim costs the -11848 fallback, not the item. - let criteriaPanelReadout: Bool? = + var criteriaPanelReadout: Bool? = options.suppressDisplayCriteria ? nil : displayCriteria.currentPanelIsHDR() - let panelHDRAfterHandshake = Self.sessionPanelPresentsHDR( + var panelHDRAfterHandshake = Self.sessionPanelPresentsHDR( hostAsserts: options.panelIsInHDRMode, criteriaReadout: criteriaPanelReadout) // AE#459: the ROUTE may assume more than the LABEL may claim. `panelHDRAfterHandshake` stays the // label's answer, conservative by design; the route additionally offers an unproven but // HDR-eligible display the master and lets AVFoundation's acceptance be the readout `UIScreen` // has been measured getting wrong on a panel that was demonstrably presenting HDR. Live is excluded: its fallback is a rejoin at // the edge rather than a restored position, and that cost has not been measured. - let routingPanelHDR = Self.sessionRoutesAsHDRPanel( + var routingPanelHDR = Self.sessionRoutesAsHDRPanel( panelPresentsHDR: panelHDRAfterHandshake, attemptWhenUnproven: options.attemptsHDRMasterOnUnprovenPanel && !options.isLive, displayEligibleForHDR: observedDisplayCaps.supportsHDR, panelRefusedHDRMaster: Self.panelRefusedHDRMaster) + // AE#667: that acceptance only answers once the panel has arrived. See `unprovenMasterAwaitsSwitchEnd`. + if Self.unprovenMasterAwaitsSwitchEnd( + routesAsHDR: routingPanelHDR, + panelPresentsHDR: panelHDRAfterHandshake, + switchRunning: displayCriteria.observedSwitchIsRunning()) { + EngineLog.emit( + "[DisplayCriteria] unproven HDR master waits for the running switch to end before it is " + + "served (#667)", + category: .session) + await displayCriteria.waitForSwitch(consumesRecord: false, settleCap: .awaitObservedEnd, + isCurrent: { self.loadGeneration == gen }) + if loadGeneration != gen { + probe.markClosed() + if probeOpened { + Task.detached { [probe] in probe.close() } + } + try checkLoadCurrent(gen) + } + criteriaPanelReadout = options.suppressDisplayCriteria ? nil : displayCriteria.currentPanelIsHDR() + panelHDRAfterHandshake = Self.sessionPanelPresentsHDR( + hostAsserts: options.panelIsInHDRMode, criteriaReadout: criteriaPanelReadout) + routingPanelHDR = Self.sessionRoutesAsHDRPanel( + panelPresentsHDR: panelHDRAfterHandshake, + attemptWhenUnproven: options.attemptsHDRMasterOnUnprovenPanel && !options.isLive, + displayEligibleForHDR: observedDisplayCaps.supportsHDR, + panelRefusedHDRMaster: Self.panelRefusedHDRMaster) + } sessionPanelHDRReadout = criteriaPanelReadout sessionObservedDisplayCaps = observedDisplayCaps if routingPanelHDR != panelHDRAfterHandshake { @@ -4480,11 +4910,13 @@ public final class AetherEngine: ObservableObject { // hardware-decode (H.264 High 4:2:2/4:4:4/High-10, HEVC Rext on Intel Macs / older Apple TV) reaches // readyToPlay then renders nothing on the native path. QuickTime plays it via its own software decoder; // there is no analogous fallback on the native route, so route it to the SoftwarePlaybackHost - // (libavcodec), which decodes these profiles. VOD only: forced-native live keeps its verified path, and - // broadcast H.264 / HEVC is HW-decodable. Apple Silicon HW-decodes these, so the probe keeps them native. + // (libavcodec), which decodes these profiles. VOD only for H.264 / HEVC: forced-native live keeps its + // verified path, and broadcast H.264 / HEVC is HW-decodable. AV1 is judged on live too (audit HLS-5). + // Apple Silicon HW-decodes these, so the probe keeps them native. // #176: DV Profile 5 is exempt inside the policy; the raw-hvcC probe misjudges the dvh1 route and the // software path decodes IPT-PQ-c2 with a green/purple cast, so P5 must stay native unconditionally. - if !useSoftwarePath, !options.isLive, probeOpened, + if !useSoftwarePath, probeOpened, + VideoRoutingPolicy.consultsUndecodableFormatGate(codecID: detectedCodecID, isLive: options.isLive), let vStream = probe.stream(at: probe.videoStreamIndex), let codecpar = vStream.pointee.codecpar { let dvProfile = Self.dvProfile(stream: vStream) @@ -4523,6 +4955,28 @@ public final class AetherEngine: ObservableObject { // (device trace: a 720p50 timeshift archive played clean on the native path and visibly // stuttered on the software one). Declared-interlaced archives still route software via // the field-order policy above - the #232 refute probe cannot run without a rewind. + if probeOpened, VideoRoutingPolicy.promotesForwardOnlySourceToSequential( + isSourceSeekable: probe.isSourceSeekable, + isLive: options.isLive, + declaredSequential: options.sequentialOrigin, + isCustomSource: isCustomSource, + routedSoftware: useSoftwarePath, + preferred: options.preferredDecodePath, + containerDurationSeconds: probe.duration + ) { + // Written into loadedOptions as well, so every rebuild of this session (AirPlay LAN swap, + // audio switch, background return) reopens it as the sequential origin it turned out to be. + options.sequentialOrigin = true + options.declaredDurationSeconds = probe.duration + loadedOptions.sequentialOrigin = true + loadedOptions.declaredDurationSeconds = probe.duration + isSequentialOrigin = true + EngineLog.emit( + "[AetherEngine] forward-only source with a container duration of " + + "\(String(format: "%.1f", probe.duration))s: serving it as a sequential origin", + category: .engine + ) + } if !probe.isSourceSeekable && !options.isLive { if options.sequentialOrigin { if !useSoftwarePath { @@ -4723,12 +5177,14 @@ public final class AetherEngine: ObservableObject { criteriaUnchanged: criteriaUnchanged, engineIsCriteriaWriter: !options.suppressDisplayCriteria, formatKnown: probeOpened, - effectiveFormat: effectiveFormat + effectiveFormat: effectiveFormat, + noWriterExpected: options.sharedOutputRole == .secondary ), // Sodalite#49: this gate runs after the item is ready, so waiting out an observed switch // blocks nothing else, and the panel is dark until it ends either way. Live keeps the // standard cap: a zap must not sit behind a panel handshake. - settleCap: options.isLive ? .standard : .awaitObservedEnd) + settleCap: options.isLive ? .standard : .awaitObservedEnd, + isCurrent: { self.loadGeneration == gen }) try checkLoadCurrent(gen) // automaticallyWaitsToMinimizeStalling=true (default) handles play-before-ready. // #35: on a real SDR->HDR switch while serving a VOD master, drive the bounded @@ -4767,6 +5223,9 @@ public final class AetherEngine: ObservableObject { // Superseded. throw CancellationError() } catch { + // A host or open that failed because stop()/load() tore it down mid-flight throws its own + // error before the post-load generation check; the successor owns the state (audit CORE-1). + if loadGeneration != gen { throw CancellationError() } // AE#246: the load-time probe failed for a reason that was not the HLS classification, so the // AE#154 check above saw an inconclusive failure and this load fell through to the loopback // path with no preopened demuxer. The session's own open is then the first one to read the @@ -4823,6 +5282,7 @@ public final class AetherEngine: ObservableObject { } public func play() { + recordTransportDuringRebuild(playing: true) // AetherEngine#164: a VOD parked at its final frame (scrubbed to the end, or paused there) // cannot advance; AVPlayer.play() would no-op and leave the button frozen. Rewind to the start // first, then resume. `.ended` is excluded (see shouldRewindBeforePlay): it stays terminal so a @@ -4844,6 +5304,7 @@ public final class AetherEngine: ObservableObject { } public func pause() { + recordTransportDuringRebuild(playing: false) resumeAfterInterruption = false activeTransportHost?.pause() isBuffering = false @@ -4927,11 +5388,16 @@ public final class AetherEngine: ObservableObject { ) throw AetherEngineError.sessionNotReloadable(refusal) } + // Sodalite#173: the rebuild's own teardown moves the generation by one; anything further is a + // stop() or load() that superseded it, which a returned nil does not say. + let reloadGeneration = loadGeneration &+ 1 + // Audit LIF-102: the value written above, or the mount flag after a background teardown. let failure = await reloadWithAudioOverride( url: placeholderURL, audioStreamIndex: selection.audioTrackIndex.map { Int32($0) }, expectedGeneration: loadGeneration, - discTitleIDOverride: selection.discTitleID + discTitleIDOverride: selection.discTitleID, + resumePlaying: loadedOptions.autoplay ) // AE#460 follow-up: a rebuild that died leaves the session in `.error`, and this branch // used to return as if it had come back, so `reloadAtCurrentPosition(applying:)` @@ -4940,6 +5406,7 @@ public final class AetherEngine: ObservableObject { // source whose reader read `cancel()` as terminal: the rebuild failed on stream info // and the call still returned success. if let failure { throw failure } + if loadGeneration != reloadGeneration { throw CancellationError() } // The reload restores from its own pre-stopInternal snapshot, which a torn-down session // no longer had anything in; replay the parked subtitle pick on top of it. if resumesTornDownSession { @@ -4983,6 +5450,11 @@ public final class AetherEngine: ObservableObject { options.subtitleSessionCarryover = carryover try await load(url: url, startPosition: resume, options: options, audioSourceStreamIndex: audioToRestore.map { Int32($0) }, discTitleID: titleID) + // AE#711 follow-up: `load` autostarts from the flag it was handed; a play/pause that arrived + // while it ran is the newer word. + if let intent = transportIntentUnderReconstruction, intent != options.autoplay { + if intent { play() } else { pause() } + } restoreSubtitleSelection(from: carryover, resumeAnchor: resume) // Arm the watchdog so a live reopen whose AVPlayer never becomes ready fails visibly instead of freezing. if options.isLive, !options.nativeRemoteHLS, playbackBackend == .native { @@ -4991,6 +5463,12 @@ public final class AetherEngine: ObservableObject { } public func seek(to seconds: Double) async { + // Audit DMX-113: the `.loading` branch below would otherwise publish a NaN or infinite + // position into the clock, and the hosts' seeks hand it to the Demuxer as ticks. + guard seconds.isFinite else { + EngineLog.emit("[AetherEngine] seek(to:\(seconds)) ignored: not a finite position", category: .engine) + return + } await seek(to: seconds, origin: .host) } @@ -5104,6 +5582,9 @@ public final class AetherEngine: ObservableObject { + (held ? ", the place it held" : ", which is as close to it as the cache still reaches"), category: .engine) } + // AE#464 round 5: from here the seek target describes the session, not the load before it, + // so a landing at exactly 0 is never read as a clock that has not published yet. + positionUnderReconstruction = nil state = .seeking // Span isSeeking across the real landing, not just the optimistic .playing flip (#38). // Generation guard at each finalize point prevents a superseded seek from clearing it. @@ -5759,6 +6240,7 @@ public final class AetherEngine: ObservableObject { } public func stop(resetDisplayCriteria: Bool = true, finalTeardown: Bool? = nil) { + cancelPendingAudioSelection() nextLoadRequestsInPlaceItemHandover = false stopInternal(resetDisplayCriteria: resetDisplayCriteria, finalTeardown: finalTeardown ?? resetDisplayCriteria) @@ -5800,6 +6282,9 @@ public final class AetherEngine: ObservableObject { nativeSubtitleRenderingRequested = false injectedSubtitleSelectionTask?.cancel() injectedSubtitleSelectionTask = nil + releaseInjectedRenditionSuppression() + injectedRenditionSelection = nil + detachRemoteHLSCueClock() // AE#616 // Font attachments are session-scoped but must survive stopInternal (audio-track-switch skips the probe; // clearing in stopInternal would leave the session with an empty font list after any audio switch). fontAttachments = [] @@ -5821,6 +6306,8 @@ public final class AetherEngine: ObservableObject { sessionObservedDisplayCaps = nil sourceVideoFrameRate = nil sourceVideoBitrate = 0 + sourceVideoStreamFormat = nil + decodedVideoFormat = nil sourceVideoCodecName = nil sourceContainerFormat = nil sourceVideoWidth = 0 @@ -6165,12 +6652,38 @@ public final class AetherEngine: ObservableObject { AirPlayPlaylistDecision.routeChangeNeedsReload( isRemoteHLSBypass: loadedOptions.nativeRemoteHLS, bypassServesLoopbackOrigin: remoteHLSSubtitleProxy != nil) else { return } + if swapsItemForAirPlayEdge() { return } EngineLog.emit("[AirPlay] external playback \(wantAirPlay ? "active (wireless) -> LAN reload" : "ended -> loopback reload")" + (loadedOptions.nativeRemoteHLS ? " (remote-HLS bypass on its #316 subtitle origin)" : ""), category: .engine) Task { try? await reloadAtCurrentPosition() } } + /// A sequential origin crosses an AirPlay edge by swapping the item, not by rebuilding the session. + /// A rebuild reopens the source, and such a source can only be read again from byte 0: the reload + /// resumed at 0 instead of the playhead and then sat seeking while the producer re-read the prefix. + /// Nothing about the session has to change for the hop anyway. The server already listens on every + /// interface and keeps every segment it cut, so only the address AVPlayer is handed differs (the LAN + /// IP and the receiver's playlist, or the loopback again when the receiver lets go). + @MainActor + private func swapsItemForAirPlayEdge() -> Bool { + guard loadedOptions.sequentialOrigin, !loadedOptions.isLive, !loadedOptions.nativeRemoteHLS, + let host = nativeHost, let session = nativeVideoSession, + let loopback = session.servingMasterPlaylist + ? session.masterPlaylistURL : session.mediaPlaylistURL else { return false } + let position = currentTime + let served = airPlayAdjustedPlayback(url: loopback, session: session) + nativeSubtitleRenditionsServed = served.subtitleRenditionsServed + EngineLog.emit( + "[AirPlay] external playback \(airPlayActive ? "active (wireless)" : "ended") on a sequential " + + "origin -> item swap to \(served.url.absoluteString) at \(String(format: "%.2f", position))s " + + "(the source cannot be reopened past byte 0)", + category: .engine) + host.swapItem(url: served.url, startPosition: position) + armAirPlayProgressWatchdog(gen: loadGeneration, position: position) + return true + } + /// Re-read external playback after a session-preserving reload and act on it if it really changed (#227). /// The player flag alone is not trustworthy at this instant: the rebuilt item may not have re-engaged the /// receiver yet, which would read as "AirPlay ended" and start the next reload of the same loop. The audio @@ -6415,11 +6928,25 @@ public final class AetherEngine: ObservableObject { // MARK: - Audio / subtitle track selection - /// Switch the active audio track mid-playback. Restarts the HLS pipeline with the new audio stream; - /// expects ~0.5-1 s black frame (AVPlayer.replaceCurrentItem tears the surface). Display-criteria handshake + /// Switch the active audio track mid-playback. Rebuilds the pipeline at the current position; + /// native audio handover retains the old item until replacement, but decoder readiness can still + /// interrupt presentation. Software/forward-only sources cannot promise a seamless switch. Display-criteria handshake /// is suppressed (video unchanged). `index` is the container stream index (TrackInfo.id). No-op if /// out-of-range, pointing at a non-audio stream, or already active. + /// + /// Not available on `VideoRoute.remoteBypass`: there `audioTracks` lists what AVPlayer built and is + /// informational, the selection belongs to AVFoundation, and a pick is logged and ignored rather than + /// turned into a reload the bypass has no audio override for. A host that wants another language on + /// that route loads a URL that carries it. public func selectAudioTrack(index: Int) { + if videoRoute == .remoteBypass { + EngineLog.emit( + "[AetherEngine] selectAudioTrack(\(index)) ignored: AVPlayer owns the audio selection on " + + "the remote-HLS bypass; audioTracks is informational there", + category: .engine + ) + return + } // Forward-only custom sources (incl. live HLS-ingest) can't rewind; rebuilding would re-consume a // drained FIFO and stall silently. Logged so a picker that does nothing is explainable. if isCustomSource && !customSourceIsSeekable { @@ -6439,21 +6966,35 @@ public final class AetherEngine: ObservableObject { ) return } - if activeAudioTrackIndex == index { return } + if activeAudioTrackIndex == index && audioSelectionTask == nil { return } EngineLog.emit( "[AetherEngine] selectAudioTrack: scheduling switch to stream \(index)", category: .engine ) - let gen = loadGeneration - Task { @MainActor [weak self] in - guard let self = self else { return } - await self.reloadWithAudioOverride( - url: url, - audioStreamIndex: Int32(index), - expectedGeneration: gen - ) + pendingAudioSelection = index + guard audioSelectionTask == nil else { return } + let epoch = audioSelectionEpoch + audioSelectionTask = Task { @MainActor [weak self] in + guard let self, self.audioSelectionEpoch == epoch else { return } + defer { + if self.audioSelectionEpoch == epoch { + self.audioSelectionTask = nil + } + } + while let selected = self.pendingAudioSelection { + self.pendingAudioSelection = nil + guard !Task.isCancelled, self.audioSelectionEpoch == epoch else { return } + if self.activeAudioTrackIndex == selected { continue } + let beforeReload = self.loadGeneration + let failure = await self.reloadWithAudioOverride( + url: url, audioStreamIndex: Int32(selected), expectedGeneration: beforeReload) + // This rebuild owns one stopInternal generation. Background teardown or + // another SDK recovery cannot lend its successor to queued audio work. + guard !Task.isCancelled, self.audioSelectionEpoch == epoch, failure == nil, + self.loadGeneration == beforeReload &+ 1 else { return } + } } } @@ -6560,12 +7101,23 @@ public final class AetherEngine: ObservableObject { /// - `!keepNativeHost`: belt and braces. A preserved host means audio keeps flowing into the next load. /// - `hostOptedIn`: `deactivatesAudioSessionOnStop`. The session is process-global state the engine /// mostly does not own, so releasing it is the host app's call. + /// Since Sodalite#175 the host opt-in is folded into `SharedOutputCoordinator.leave`, which owes it to the last engine out. nonisolated static func shouldDeactivateAudioSessionOnTeardown(finalTeardown: Bool, keepNativeHost: Bool, hostOptedIn: Bool) -> Bool { finalTeardown && !keepNativeHost && hostOptedIn } + /// Sodalite#175: the session goes only with the last engine out, and only when a release is owed. + nonisolated static func shouldDeactivateAudioSession(after outcome: SharedOutputCoordinator.LeaveOutcome) -> Bool { + outcome == .lastOut(releaseSession: true) + } + + /// Sodalite#175: a scheduled release is stale once another engine has joined in the meantime. + nonisolated static func releaseStillWanted(coordinatorIsEmpty: Bool) -> Bool { + coordinatorIsEmpty + } + #if os(iOS) || os(tvOS) /// Counterpart to the activations the engine takes part in (#215). The native path never activates the /// session itself (AVKit does it per playback, #24); the software and audio renderer paths do, in @@ -6625,11 +7177,26 @@ public final class AetherEngine: ObservableObject { audioSessionDeactivationTask = enqueueAudioSessionTransition { [weak self] in guard let self else { return } guard !Task.isCancelled, await self.loadGeneration == generation else { return } + guard await Self.releaseStillWanted(coordinatorIsEmpty: SharedOutputCoordinator.shared.isEmpty) else { + EngineLog.emit("[SharedOutput] release dropped: another engine joined before it ran", category: .engine) + return + } AetherEngine.deactivateSharedAudioSession() } } #endif + /// Sodalite#175: the panel mode belongs to every engine still playing, so a reset waits for the last one out. + private func resetDisplayCriteriaRespectingOthers() { + let id = ObjectIdentifier(self) + if SharedOutputCoordinator.shared.othersActive(besides: id) { + let controller = displayCriteria + SharedOutputCoordinator.shared.deferCriteriaReset(for: id) { controller.reset() } + } else { + displayCriteria.reset() + } + } + /// - Parameter resetDisplayCriteria: When `true` (default), release /// the `AVDisplayManager.preferredDisplayCriteria` so the panel /// returns to its default mode. Used by `load()` and the public @@ -6674,6 +7241,13 @@ public final class AetherEngine: ObservableObject { liveOutageSourceGivenUp = false liveReloadWatchdogTask?.cancel() liveReloadWatchdogTask = nil + // Audit CORE-1: the #65/#93 recovery rungs belong to the session being torn down. Left + // running across a background teardown, the item-death confirm could start a software + // rebuild seconds later with the app suspended. + stallReengageTask?.cancel() + stallReengageTask = nil + itemDeathConfirmTask?.cancel() + itemDeathConfirmTask = nil // #95: stop the tap reader before the session (and its SegmentCache) goes away. // #356: counterpart to the install line, because a session-preserving reload lands here // too. The host sees its stream finish and has to re-install; without this the device log @@ -6756,6 +7330,7 @@ public final class AetherEngine: ObservableObject { // #353: the picture belongs to the session. Left standing, the next source would be laid out // against this one's rectangle for as long as it takes its own first frame to arrive. softwareDisplaySize = nil + decodedVideoFormat = nil // #314: same detach on the software path, where the outgoing renderer's decode thread is what // can still hand a frame over while the next host comes up. softwareHost?.setVideoFrameTimeObserver(nil) @@ -6778,15 +7353,27 @@ public final class AetherEngine: ObservableObject { // #215: release the shared AVAudioSession once every render path above is quiesced. Scheduled // last so the item is unloaded, the AVPlayer released and the software/audio outputs stopped // before the session goes away, and scheduled rather than called because the release itself can - // block for ~0.5 s on a MAT passthrough route. Opt-in twice over: the caller must declare an - // actual final teardown, AND the host must have set deactivatesAudioSessionOnStop. - #if os(iOS) || os(tvOS) - if Self.shouldDeactivateAudioSessionOnTeardown(finalTeardown: finalTeardown, - keepNativeHost: keepNativeHost, - hostOptedIn: deactivatesAudioSessionOnStop) { - scheduleAudioSessionDeactivation() + // block for ~0.5 s on a MAT passthrough route. The caller must declare an actual final teardown; + // the host opt-in (deactivatesAudioSessionOnStop) goes to SharedOutputCoordinator.leave, and the + // release belongs to the last engine out. + if finalTeardown && !keepNativeHost { + let coordinator = SharedOutputCoordinator.shared + let label = coordinator.label(for: ObjectIdentifier(self)) + let outcome = coordinator.leave(ObjectIdentifier(self), releasesSession: deactivatesAudioSessionOnStop) + switch outcome { + case .othersRemain(let count): + EngineLog.emit("[SharedOutput] \(label) left, \(count) still active, session kept", category: .engine) + case .lastOut(let release): + EngineLog.emit("[SharedOutput] \(label) left last, release=\(release)", category: .engine) + case .notMember: + break + } + #if os(iOS) || os(tvOS) + if Self.shouldDeactivateAudioSession(after: outcome) { + scheduleAudioSessionDeactivation() + } + #endif } - #endif // Close custom reader on final teardown. Internal reloads pass keepCustomReader=true to survive for reuse. if !keepCustomReader { @@ -6797,7 +7384,7 @@ public final class AetherEngine: ObservableObject { } if resetDisplayCriteria { - displayCriteria.reset() + resetDisplayCriteriaRespectingOthers() } playbackBackend = .none activeVideoDecoder = nil @@ -6877,6 +7464,7 @@ public final class AetherEngine: ObservableObject { activeDiscTitleID = nil sourceStartSeconds = 0 isLive = false + isSequentialOrigin = false liveWindow = nil liveBehindWhenLastAdvancing = 0 lastPublishedLivePlayhead = nil @@ -6912,48 +7500,8 @@ public final class AetherEngine: ObservableObject { ) { [weak self] _ in Task { @MainActor in guard let self = self else { return } - #if os(iOS) - self.isBackgrounded = true - // Keep the video pipeline alive for PiP / background audio while the app stays running. - // Wedge-safe: a pause while backgrounded tears down via pause() below, so nothing crosses - // an idle suspension. - let keepAlive = Self.shouldKeepVideoAlive(enabled: self.backgroundPlaybackEnabled, - pipActive: self.pictureInPictureActive, - state: self.state) - let supportsGrace = true - #else - self.isBackgrounded = true - // tvOS: only an active PiP window defers the wedge-safe teardown (the system keeps the app - // running while its PiP window lives); no grace window, no background-audio case. The - // pictureInPictureActive didSet tears down the moment PiP ends while still backgrounded. - let keepAlive = Self.shouldKeepVideoAliveTV(enabled: self.backgroundPlaybackEnabled, - pipActive: self.pictureInPictureActive) - let supportsGrace = false - #endif - let action = Self.backgroundAction( - isAudioBackend: self.audioAVPlayerActive || self.audioHost != nil, - hasSoftwareHost: self.softwareHost != nil, - keepVideoAlive: keepAlive, - pipActive: self.pictureInPictureActive, - state: self.state - ) - switch Self.backgroundStep( - action: action, - state: self.state, - supportsGraceWindow: supportsGrace, - graceSeconds: self.backgroundTeardownGraceSeconds - ) { - case .perform(.doNothing): - return - case .perform(.enterSoftwareAudioOnly): - self.softwareHost?.enterBackgroundAudioOnly() - case .perform(.teardownVideo): - await self.teardownVideoForBackground() - case .deferTeardown(let seconds): - #if os(iOS) - self.scheduleBackgroundGraceTeardown(afterSeconds: seconds) - #endif - } + self.noteDidEnterBackground() + await self.applyBackgroundPolicy() } } lifecycleObservers.append(bgObserver) @@ -6973,6 +7521,7 @@ public final class AetherEngine: ObservableObject { let returnedFromBackground = self.isBackgrounded self.isBackgrounded = false if returnedFromBackground { Self.clearPanelRefusalOnForegroundReturn() } + self.backgroundActionOwed = false } } lifecycleObservers.append(fgObserver) @@ -6996,6 +7545,20 @@ public final class AetherEngine: ObservableObject { lifecycleObservers.append(observer) } + #if os(iOS) || os(tvOS) + AudioRouteDescription.changeLogger + #endif + + #if os(iOS) + let routeObserver = nc.addObserver( + forName: AVAudioSession.routeChangeNotification, + object: AVAudioSession.sharedInstance(), queue: .main + ) { [weak self] _ in + Task { @MainActor in self?.recomputeAirPlayPictureStaysLocal() } + } + lifecycleObservers.append(routeObserver) + #endif + // Foreign-session interruption handling (Sodalite device-verify 2026-07-15): a live-camera // PiP re-claims the audio session on every play() and the system pauses AVPlayer ~10ms after // .playing (interruption BEGAN reason=default). The system pause never goes through pause(), @@ -7061,34 +7624,77 @@ public final class AetherEngine: ObservableObject { #endif } - #if os(iOS) || os(tvOS) - /// Release the video pipeline before tvOS suspension. - /// - /// stopInternal's replaceCurrentItem(nil) + VTDecompressionSession invalidation frees the shared - /// mediaserverd decode session synchronously. keepNativeHost=true preserves the NativeAVPlayerHost shell - /// for AVKit's Now-Playing registration (issue #15); keepCustomReader=true retains the byte-source reader. - /// clock.currentTime/loadedURL/loadedOptions are preserved so reloadAtCurrentPosition() resumes correctly. - /// - /// A UIApplication background-task assertion is held across teardown so the loopback server's detached - /// socket close (HLSVideoEngine.stop drains the producer up to 3 s) completes before suspension. - @MainActor - /// AE#597: `stopInternal` does not touch the proxy, so without this its loopback socket rides - /// the suspension. Both background teardowns call it, the awaited one and the synchronous - /// backstop the system takes when it reclaims the grace assertion early. - private func releaseSubtitleProxyForBackground() { - guard Self.shouldReleaseSubtitleProxyForBackground( - hasProxy: remoteHLSSubtitleProxy != nil, isCustomSource: isCustomSource) else { return } - remoteHLSSubtitleProxy?.tearDown() - remoteHLSSubtitleProxy = nil - injectedSubtitleRenditionNames = [:] + // MARK: Owed background decision (audit CORE-2, LIF-101) + + /// Audit LIF-101: whether a background decision can be made now. Not while a load is in flight: + /// a teardown there unwinds the host's `load()` with the `CancellationError` that means "a newer + /// load or stop took over", and the host treats it as a supersede (no observers, no start report, + /// a spinner that never clears). + nonisolated static func backgroundDecisionSettles(state: PlaybackState, loadInFlight: Bool) -> Bool { + !loadInFlight && !backgroundActionIsOwed(state: state) + } + + private var backgroundDecisionMustWait: Bool { + !Self.backgroundDecisionSettles(state: state, loadInFlight: loadInFlightGeneration == loadGeneration) + } + + func beginLoadInFlight(_ generation: UInt64) { + loadInFlightGeneration = generation + } + + /// Called from the load's own exit, which is the settle point a state it passes through is not. + func endLoadInFlight(_ generation: UInt64) { + guard loadInFlightGeneration == generation else { return } + loadInFlightGeneration = nil + settleOwedBackgroundAction() + } + + /// didEnterBackground's bookkeeping. Audit CORE-2: a load or seek in flight has a pipeline + /// (loopback server, AVIO connection, the item) but no settled state to judge it by, and it + /// settles into `.playing` in the background with nothing left to look again. Owe it the decision. + func noteDidEnterBackground() { + isBackgrounded = true + backgroundActionOwed = backgroundDecisionMustWait + } + + /// Audit LIF-105: the PiP window closed while backgrounded, so nothing keeps the app running and + /// the wedge-safe teardown is due. Through the owed funnel, so a load or seek in flight in the + /// window is torn down once it settles instead of never. + func settleBackgroundDecisionAfterPictureInPictureClosed() { + guard isBackgrounded, !audioAVPlayerActive, audioHost == nil, softwareHost == nil else { return } + backgroundActionOwed = true + settleOwedBackgroundAction() + } + + /// Called from the `state` didSet and from a load's exit. A decision owed since the app went to + /// the background is made once the session has settled into a transport state. + private func settleOwedBackgroundAction() { + guard backgroundActionOwed, !backgroundDecisionMustWait else { return } + backgroundActionOwed = false + guard isBackgrounded, state == .playing || state == .paused else { return } EngineLog.emit( - "[AetherEngine] #597 background teardown: remote-HLS subtitle proxy released", + "[AetherEngine] background: the decision owed while a load or seek was in flight settled " + + "(state=\(state)); applying the background policy now (audit CORE-2)", category: .engine) + Task { @MainActor [weak self] in await self?.runOwedBackgroundPolicy() } } - private func teardownVideoForBackground() async { - let app = UIApplication.shared - let bgTask = app.beginBackgroundTask(withName: "AetherEngine.bgVideoTeardown") + private func runOwedBackgroundPolicy() async { + if let policy = owedBackgroundPolicyForTesting { + await policy(self) + return + } + #if os(iOS) || os(tvOS) + await applyBackgroundPolicy() + #endif + } + + /// The synchronous half of the background teardown. stopInternal's replaceCurrentItem(nil) + + /// VTDecompressionSession invalidation frees the shared mediaserverd decode session. + /// keepNativeHost=true preserves the NativeAVPlayerHost shell for AVKit's Now-Playing registration + /// (issue #15); keepCustomReader=true retains the byte-source reader. + /// clock.currentTime/loadedURL/loadedOptions are preserved so reloadAtCurrentPosition() resumes correctly. + func releaseVideoPipelineForBackground() { // #357: park the selection first. The foreground reload snapshots at reload time, which on // this path is long after stopInternal wiped what it wants. captureBackgroundTeardownSelection() @@ -7111,6 +7717,83 @@ public final class AetherEngine: ObservableObject { + "\(String(format: "%.0f", teardownMs))ms; the audio session and the native host " + "shell stay", category: .engine) + } + + /// AE#597: `stopInternal` does not touch the proxy, so without this its loopback socket rides + /// the suspension. Both background teardowns call it, the awaited one and the synchronous + /// backstop the system takes when it reclaims the grace assertion early. + private func releaseSubtitleProxyForBackground() { + guard Self.shouldReleaseSubtitleProxyForBackground( + hasProxy: remoteHLSSubtitleProxy != nil, isCustomSource: isCustomSource) else { return } + remoteHLSSubtitleProxy?.tearDown() + remoteHLSSubtitleProxy = nil + injectedSubtitleRenditionNames = [:] + detachRemoteHLSCueClock() // AE#616 + EngineLog.emit( + "[AetherEngine] #597 background teardown: remote-HLS subtitle proxy released", + category: .engine) + } + + #if os(iOS) || os(tvOS) + /// What didEnterBackground does with the running session. Runs again when a load or seek that + /// was in flight at that moment settles while the app is still in the background (audit CORE-2). + private func applyBackgroundPolicy() async { + guard isBackgrounded else { return } + // Audit LIF-101: a load in flight settles this from its own exit (`endLoadInFlight`). + if backgroundDecisionMustWait { + backgroundActionOwed = true + return + } + #if os(iOS) + // Keep the video pipeline alive for PiP / background audio while the app stays running. + // Wedge-safe: a pause while backgrounded tears down via pause(), so nothing crosses + // an idle suspension. + let keepAlive = Self.shouldKeepVideoAlive(enabled: backgroundPlaybackEnabled, + pipActive: pictureInPictureActive, + state: state) + let supportsGrace = true + #else + // tvOS: only an active PiP window defers the wedge-safe teardown (the system keeps the app + // running while its PiP window lives); no grace window, no background-audio case. The + // pictureInPictureActive didSet tears down the moment PiP ends while still backgrounded. + let keepAlive = Self.shouldKeepVideoAliveTV(enabled: backgroundPlaybackEnabled, + pipActive: pictureInPictureActive) + let supportsGrace = false + #endif + let action = Self.backgroundAction( + isAudioBackend: audioAVPlayerActive || audioHost != nil, + hasSoftwareHost: softwareHost != nil, + keepVideoAlive: keepAlive, + pipActive: pictureInPictureActive, + state: state + ) + switch Self.backgroundStep( + action: action, + state: state, + supportsGraceWindow: supportsGrace, + graceSeconds: backgroundTeardownGraceSeconds + ) { + case .perform(.doNothing): + return + case .perform(.enterSoftwareAudioOnly): + softwareHost?.enterBackgroundAudioOnly() + case .perform(.teardownVideo): + await teardownVideoForBackground() + case .deferTeardown(let seconds): + #if os(iOS) + scheduleBackgroundGraceTeardown(afterSeconds: seconds) + #endif + } + } + + /// Release the video pipeline before tvOS suspension (`releaseVideoPipelineForBackground`). + /// + /// A UIApplication background-task assertion is held across teardown so the loopback server's detached + /// socket close (HLSVideoEngine.stop drains the producer up to 3 s) completes before suspension. + private func teardownVideoForBackground() async { + let app = UIApplication.shared + let bgTask = app.beginBackgroundTask(withName: "AetherEngine.bgVideoTeardown") + releaseVideoPipelineForBackground() // Wait for the loopback server's detached cleanup (<=3 s producer drain + socket shutdown) before releasing. try? await Task.sleep(nanoseconds: 3_500_000_000) if bgTask != .invalid { app.endBackgroundTask(bgTask) } @@ -7152,7 +7835,10 @@ public final class AetherEngine: ObservableObject { /// mid-window) and perform it without further deferral. private func fireBackgroundGraceTeardown() async { backgroundGraceTask = nil - if isBackgrounded { + if isBackgrounded, backgroundDecisionMustWait { + // Audit LIF-101: a load the host started inside the window gets the decision at its exit. + backgroundActionOwed = true + } else if isBackgrounded { switch currentBackgroundAction() { case .teardownVideo: EngineLog.emit("[AetherEngine] background grace window expired, tearing down paused pipeline (#127)", category: .engine) diff --git a/Sources/AetherEngine/Audio/AtmosConfirmationWorker.swift b/Sources/AetherEngine/Audio/AtmosConfirmationWorker.swift index 688ee1912..52f3cb542 100644 --- a/Sources/AetherEngine/Audio/AtmosConfirmationWorker.swift +++ b/Sources/AetherEngine/Audio/AtmosConfirmationWorker.swift @@ -60,7 +60,7 @@ extension AetherEngine { applyConfirmedAtmos() let candidates = Self.atmosConfirmationCandidates(in: audioTracks) guard !candidates.isEmpty else { return } - if isCustomSource, customReader?.makeIndependentReader() == nil { return } + if isCustomSource, !Self.canForkCustomReader(customReader) { return } let headers = loadedOptions.httpHeaders let formatHint = customFormatHint @@ -74,6 +74,15 @@ extension AetherEngine { } } + /// Whether the host's reader can hand out the independent cursor each pass needs. The probe builds a + /// real reader, which the `IOReader` contract leaves to the engine to close (audit DEC-109); a host + /// whose fork opens a connection would otherwise hold one per load for nothing. + nonisolated static func canForkCustomReader(_ reader: IOReader?) -> Bool { + guard let probe = reader?.makeIndependentReader() else { return false } + probe.close() + return true + } + /// Teardown. `markClosed()` on the side demuxer so a parked AVIO read cannot outlive the session, /// the same rule the native subtitle readers follow. func cancelAtmosConfirmation() { diff --git a/Sources/AetherEngine/Audio/AtmosDetectionProbe.swift b/Sources/AetherEngine/Audio/AtmosDetectionProbe.swift index 0697cc094..498712296 100644 --- a/Sources/AetherEngine/Audio/AtmosDetectionProbe.swift +++ b/Sources/AetherEngine/Audio/AtmosDetectionProbe.swift @@ -137,6 +137,15 @@ extension AetherEngine { /// Packet ceiling for the AVDISCARD_ALL fuse, saturating instead of trapping: `maxPackets` is a public /// option and `Int.max` is a plausible "no limit" value for a host to pass (`forwardBufferSegments` /// takes exactly that), which would overflow a plain multiply. + /// Source bytes the probe may consume across every stream. With the other streams at AVDISCARD_ALL + /// the demuxer reads and drops a video run inside one `av_read_frame`, where neither `maxBytes` nor + /// the packet fuse sees it. Wider than the HDR10+ budget because the audio sits behind interleaved + /// UHD video here: 16 x `maxBytes`, at least 64 MiB. Saturating: `maxBytes` is public. + nonisolated static func atmosInputByteBudget(maxBytes: Int64) -> Int64 { + let (product, overflowed) = max(0, maxBytes).multipliedReportingOverflow(by: 16) + return overflowed ? .max : max(product, 64 * 1024 * 1024) + } + nonisolated static func atmosForeignPacketFuse(maxPackets: Int) -> Int { let (product, overflowed) = maxPackets.multipliedReportingOverflow(by: foreignPacketFuseMultiplier) return overflowed ? Int.max : product @@ -228,6 +237,8 @@ extension AetherEngine { // exhaust the 8 MiB byte cap in well under a second of container data, and the probe returns // .byteCap having fed the decoder nothing -- reporting "not Atmos" for genuinely Atmos media. demuxer.discardAllStreamsExcept([targetIndex]) + demuxer.beginInputByteBudget(Self.atmosInputByteBudget(maxBytes: options.maxBytes)) + defer { demuxer.endInputByteBudget() } while true { let elapsed = Double(DispatchTime.now().uptimeNanoseconds - start.uptimeNanoseconds) / 1_000_000_000 @@ -242,10 +253,10 @@ extension AetherEngine { do { packet = try demuxer.readPacket() } catch { - return stopped(.demuxError) + return stopped(demuxer.inputByteBudgetExhausted ? .byteCap : .demuxError) } guard let pkt = packet else { - return stopped(.demuxEOF) + return stopped(demuxer.inputByteBudgetExhausted ? .byteCap : .demuxEOF) } let afterRead = Double(DispatchTime.now().uptimeNanoseconds - start.uptimeNanoseconds) / 1_000_000_000 if afterRead >= options.timeBudget { diff --git a/Sources/AetherEngine/Audio/AudioAVPlayerHost.swift b/Sources/AetherEngine/Audio/AudioAVPlayerHost.swift index a877bdefb..c763c024c 100644 --- a/Sources/AetherEngine/Audio/AudioAVPlayerHost.swift +++ b/Sources/AetherEngine/Audio/AudioAVPlayerHost.swift @@ -43,6 +43,9 @@ final class AudioAVPlayerHost { /// shared singletons is what produces the half-working state. let nowPlayingSession: MPNowPlayingSession #endif + /// Sodalite#175: false for a `.secondary` engine, which never claims Now Playing. Set per load, the + /// host outlives a role change. + var ownsNowPlaying: Bool // MARK: - Private state @@ -84,8 +87,10 @@ final class AudioAVPlayerHost { // MARK: - Init - init(diagnosticPool: ItemDiagnosticReadPool = .shared, + init(ownsNowPlaying: Bool = true, + diagnosticPool: ItemDiagnosticReadPool = .shared, diagnosticRead: @escaping AVPlayerItemDiagnostics.Read = ItemDiagnosticSnapshot.read) { + self.ownsNowPlaying = ownsNowPlaying self.diagnosticPool = diagnosticPool self.diagnosticRead = diagnosticRead #if os(tvOS) || os(iOS) @@ -98,7 +103,7 @@ final class AudioAVPlayerHost { // manual-publish design whose MPMediaItemArtwork closure was non-@Sendable and tripped // dispatch_assert_queue_fail when MediaPlayer requested the bitmap off-actor. nowPlayingSession.automaticallyPublishesNowPlayingInfo = true - nowPlayingSession.becomeActiveIfPossible(completion: { _ in }) + if ownsNowPlaying { nowPlayingSession.becomeActiveIfPossible(completion: { _ in }) } #endif } @@ -106,6 +111,7 @@ final class AudioAVPlayerHost { /// Home overlay + remote commands alive across a background pause). func becomeActiveNowPlaying() { #if os(tvOS) || os(iOS) + guard ownsNowPlaying else { return } nowPlayingSession.becomeActiveIfPossible(completion: { _ in }) #endif } diff --git a/Sources/AetherEngine/Audio/AudioBridge.swift b/Sources/AetherEngine/Audio/AudioBridge.swift index 5314350c4..2e0007ac4 100644 --- a/Sources/AetherEngine/Audio/AudioBridge.swift +++ b/Sources/AetherEngine/Audio/AudioBridge.swift @@ -169,6 +169,7 @@ final class AudioBridge: @unchecked Sendable { /// before overwrite and in cleanup. private var swrInFmt: AVSampleFormat = AV_SAMPLE_FMT_NONE private var swrInRate: Int32 = 0 + private var swrReconfigureFailures = 0 private var swrInLayout = AVChannelLayout() /// FIFO buffering resampled PCM until >= encoderCtx.frame_size samples. FLAC's wrapper has /// AV_CODEC_CAP_SMALL_LAST_FRAME but not VARIABLE_FRAME_SIZE, so non-final frames must hit frame_size exactly @@ -180,10 +181,17 @@ final class AudioBridge: @unchecked Sendable { /// source), the abandoned pump can still be inside feed() while the restart thread calls /// startSegment() on this otherwise lock-free bridge; concurrent libswresample/libavcodec calls on /// the same contexts are a data race. Uncontended in the normal single-pump case. Mirrors - /// AudioDecoder.stateLock. Diagnostic reads (fifoSampleCount/liveBytes) stay lock-free; the engine - /// lifecycle (restartLock ref-handoff + waitForFinish-gated cleanup) forecloses their races. + /// AudioDecoder.stateLock. Diagnostic reads (fifoSampleCount/liveBytes) never touch a context: they read + /// a snapshot published under `opLock` (audit DEC-104, Vpipeline-101), because feed() and startSegment() + /// free and replace `swrCtx` / `encoderCtx` mid-session and the lifecycle only forecloses that for cleanup(). private let opLock = NSLock() + /// The figures `liveBytes` and `fifoSampleCount` report, written at the end of every operation that + /// can change them while `opLock` is held. Its own lock, so a 1 Hz reader neither waits on a long + /// feed() nor sees a torn struct. + private let liveBytesLock = NSLock() + private var publishedLiveBytes = LiveBytes(fifoSamples: 0, fifoBytes: 0, swrDelaySamples: 0, swrDelayBytes: 0) + /// PCM intermediate format end-to-end (resampler -> FIFO -> encoder). S16 for lossy sources (EAC3/AC3); /// S32 @ bits_per_raw_sample=24 for lossless sources (TrueHD, DTS-HD MA, FLAC, ALAC, raw 24/32-bit PCM) so /// FLAC output stays bit-perfect (S16 would dither away the bottom 8 bits, audible in quiet passages). @@ -486,14 +494,12 @@ final class AudioBridge: @unchecked Sendable { opLock.lock() defer { opLock.unlock() } cleanup() + publishLiveBytes() } /// FIFO depth in samples/channel, for the engine memory probe. Steady-state below frame_size (~4608 @48kHz); /// a growing value means the encoder isn't keeping up with the resampler. - var fifoSampleCount: Int { - guard let f = fifo else { return 0 } - return Int(av_audio_fifo_size(f)) - } + var fifoSampleCount: Int { liveBytes.fifoSamples } /// Cumulative bytes of encoded audio the bridge has emitted this session (sum of every output packet's size). /// Monotonic across producer restarts and encoder rebuilds; the telemetry sampler diffs it into a live output @@ -576,9 +582,16 @@ final class AudioBridge: @unchecked Sendable { /// One-shot: a bridge that stays silent for an hour costs one line, not one per packet. private(set) var silentFeedReported = false + /// AE#641: called once, on the pump thread, when the silence is structural AND the decoder is the + /// arm that failed (`decodedNothing`). A live session has no muxer death to learn it from: a FLAC + /// sample entry is built from the encoder's extradata, so segments keep being cut with an audio + /// track that never carries a sample, and AVPlayer shows the first picture and waits on the audio + /// forever. Set before the producer starts feeding. + var onDecoderProducedNothing: (@Sendable (FeedStats) -> Void)? + /// AE#474: the DECODER arm's unit, and only its unit. Source went in and the FIFO got nothing /// back, so there is no sample count to bound anything with and packets are all there is. - private static let silentFeedPacketThreshold = 64 + static let silentFeedPacketThreshold = 64 /// AE#474: the ENCODER arm's unit. `drainFIFOIntoEncoder(requireFull: true)` cannot encode below /// `frame_size`, so nothing can be emitted until that many samples have been enqueued, and the @@ -615,37 +628,30 @@ final class AudioBridge: @unchecked Sendable { } var liveBytes: LiveBytes { - let fifoSamples: Int - if let f = fifo { - fifoSamples = Int(av_audio_fifo_size(f)) - } else { - fifoSamples = 0 - } - - let channels: Int - let bytesPerSample: Int = Int(pcmBytesPerSample) - if let enc = encoderCtx { - channels = Int(enc.pointee.ch_layout.nb_channels) - } else { - channels = 0 - } - - let fifoBytes = fifoSamples * channels * bytesPerSample + liveBytesLock.lock() + defer { liveBytesLock.unlock() } + return publishedLiveBytes + } - let swrDelaySamples: Int + /// Reads the FFmpeg contexts and publishes the result for `liveBytes`. Callers hold `opLock`, which + /// is what makes the reads safe against the swaps. + private func publishLiveBytes() { + let fifoSamples = fifo.map { Int(av_audio_fifo_size($0)) } ?? 0 + let channels = encoderCtx.map { Int($0.pointee.ch_layout.nb_channels) } ?? 0 + let bytesPerSample = Int(pcmBytesPerSample) + var swrDelaySamples = 0 if let swr = swrCtx, let enc = encoderCtx { swrDelaySamples = Int(swr_get_delay(swr, Int64(enc.pointee.sample_rate))) - } else { - swrDelaySamples = 0 } - let swrDelayBytes = swrDelaySamples * channels * bytesPerSample - - return LiveBytes( + let snapshot = LiveBytes( fifoSamples: fifoSamples, - fifoBytes: fifoBytes, + fifoBytes: fifoSamples * channels * bytesPerSample, swrDelaySamples: swrDelaySamples, - swrDelayBytes: swrDelayBytes + swrDelayBytes: swrDelaySamples * channels * bytesPerSample ) + liveBytesLock.lock() + publishedLiveBytes = snapshot + liveBytesLock.unlock() } /// Mark a producer restart boundary: drain the FIFO (drops the buffered partial frame, max ~96 ms @48kHz) and @@ -654,6 +660,7 @@ final class AudioBridge: @unchecked Sendable { func startSegment() { opLock.lock() defer { opLock.unlock() } + defer { publishLiveBytes() } // A prior pump reached EOF and flush() drained the encoder into its terminal state; rebuild it // before this restart feeds new frames (#99 failure mode B). if drainedAtEOF { @@ -689,6 +696,7 @@ final class AudioBridge: @unchecked Sendable { guard let dec = decoderCtx, let enc = encoderCtx, let swr = swrCtx, let fifoPtr = fifo else { return [] } drainedAtEOF = true + defer { publishLiveBytes() } var results: [UnsafeMutablePointer] = [] // 1. Drain the decoder's internal delay. @@ -825,6 +833,7 @@ final class AudioBridge: @unchecked Sendable { let fifoPtr = fifo else { return [] } + defer { publishLiveBytes() } stats.packetsFed += 1 stats.packetsFedSinceLastEnqueue += 1 @@ -965,6 +974,7 @@ final class AudioBridge: @unchecked Sendable { + "written, so this session will fail its first segment cut unless output starts.", category: .session ) + if stats.decodedNothing { onDecoderProducedNothing?(stats) } } /// Align swr's INPUT side to the frame the decoder actually produced. libswresample reads `extended_data` @@ -974,24 +984,30 @@ final class AudioBridge: @unchecked Sendable { /// probe), and reading S32 integers as FLTP floats is noise. Re-derive the input from the frame, keeping /// the output side pinned to the encoder, exactly as AudioDecoder configures its resampler from the frame. /// No-op in the common case where find_stream_info already resolved the format (frame == seed), so working - /// paths are untouched; only a wrong seed or a genuine mid-stream format change rebuilds. swr_alloc_set_opts2 - /// reuses the context pointer on success and frees it on failure (the caller re-binds swrCtx); swr_init drops + /// paths are untouched; only a wrong seed or a genuine mid-stream format change rebuilds. A rebuild drops /// the sub-frame resampler delay, as startSegment already does. Runs under feed()'s opLock (never re-lock). + /// + /// Audit DEC-6: built on a scratch context and swapped in only once it initialised. Rebuilding the + /// live one lost it for good on a rejected format (set-opts frees it), after which every feed + /// returned early, the bridge stayed mute, and not even the AE#396 detector could see it. A frame + /// the resampler cannot take is now dropped and counted, and the old context keeps serving the + /// format it was built for. Returns false when this frame must not reach `swr_convert`. private func reconfigureSwrInputIfNeeded( forFrame sf: UnsafeMutablePointer, enc: UnsafeMutablePointer - ) { + ) -> Bool { let frameFmtRaw = sf.pointee.format let frameRate = sf.pointee.sample_rate - guard frameFmtRaw >= 0, frameRate > 0, sf.pointee.ch_layout.nb_channels > 0 else { return } + guard frameFmtRaw >= 0, frameRate > 0, sf.pointee.ch_layout.nb_channels > 0 else { return true } let matchesCurrent = frameFmtRaw == swrInFmt.rawValue && frameRate == swrInRate && av_channel_layout_compare(&swrInLayout, &sf.pointee.ch_layout) == 0 - guard !matchesCurrent else { return } + guard !matchesCurrent else { return true } let frameFmt = AVSampleFormat(rawValue: frameFmtRaw) + var scratch: OpaquePointer? let setRet = swr_alloc_set_opts2( - &swrCtx, + &scratch, &enc.pointee.ch_layout, pcmSampleFmt, enc.pointee.sample_rate, @@ -1001,7 +1017,22 @@ final class AudioBridge: @unchecked Sendable { 0, nil ) - guard setRet >= 0, swrCtx != nil, swr_init(swrCtx) >= 0 else { return } + let initRet = setRet >= 0 && scratch != nil ? swr_init(scratch) : setRet + guard initRet >= 0 else { + swr_free(&scratch) + swrReconfigureFailures += 1 + if swrReconfigureFailures == 1 || swrReconfigureFailures % 500 == 0 { + EngineLog.emit( + "[AudioBridge] ERROR: resampler rejected decoded \(frameRate)Hz/" + + "\(sf.pointee.ch_layout.nb_channels)ch fmt=\(frameFmtRaw) (ret=\(initRet)); " + + "\(swrReconfigureFailures) frame(s) dropped", + category: .session + ) + } + return false + } + swr_free(&swrCtx) + swrCtx = scratch av_channel_layout_uninit(&swrInLayout) av_channel_layout_copy(&swrInLayout, &sf.pointee.ch_layout) @@ -1023,6 +1054,7 @@ final class AudioBridge: @unchecked Sendable { category: .session ) } + return true } /// Resample sf (decoded source frame) to encoder format and push into the FIFO (swr_convert may produce @@ -1047,10 +1079,8 @@ final class AudioBridge: @unchecked Sendable { // Align swr's INPUT to the frame the decoder actually produced before converting. No-op once the seed // matched (the usual case); only a wrong init seed or a genuine mid-stream format change rebuilds swr. - reconfigureSwrInputIfNeeded(forFrame: sf, enc: enc) - // The rebuild reuses the context pointer on success, but swr_alloc_set_opts2 frees it on a set-opts - // failure (swr_free(ps) -> swrCtx == nil), which would dangle the caller's `swr`. Re-bind to the live one. - guard let swr = swrCtx else { + // A successful rebuild replaces the context, which would dangle the caller's `swr`. Re-bind to the live one. + guard reconfigureSwrInputIfNeeded(forFrame: sf, enc: enc), let swr = swrCtx else { stats.framesDroppedBeforeFIFO += 1 return } diff --git a/Sources/AetherEngine/Audio/AudioDecoder.swift b/Sources/AetherEngine/Audio/AudioDecoder.swift index dafe84b1e..225345fa1 100644 --- a/Sources/AetherEngine/Audio/AudioDecoder.swift +++ b/Sources/AetherEngine/Audio/AudioDecoder.swift @@ -355,10 +355,7 @@ final class AudioDecoder: @unchecked Sendable { // First frame in a new accumulator captures the PTS; subsequent frames only extend the buffer. if pendingSampleCount == 0 { - let pts = frame.pointee.pts - pendingStartPTS = (pts != Int64.min) - ? CMTimeMake(value: pts * Int64(timeBase.num), timescale: Int32(timeBase.den)) - : .invalid + pendingStartPTS = SourceTimestampBounds.cmTime(ticks: frame.pointee.pts, timeBase: timeBase) } pendingBytes.append(tempBuffer, count: Int(convertedSamples) * bytesPerSample) diff --git a/Sources/AetherEngine/Audio/AudioOutput.swift b/Sources/AetherEngine/Audio/AudioOutput.swift index 56ca7d657..43fa2915b 100644 --- a/Sources/AetherEngine/Audio/AudioOutput.swift +++ b/Sources/AetherEngine/Audio/AudioOutput.swift @@ -19,6 +19,16 @@ final class AudioOutput: @unchecked Sendable { /// One line per offset change, not per buffer. Reset by `setPresentationOffset`. private var loggedOffsetInEffect = false + /// Audit DEC-106: retired by every `flush()` and `stop()`, under `lock`. A feed loop reads it before + /// it reads a packet and enqueues through `enqueue(sampleBuffer:ifEpoch:)`, which compares under the + /// same lock, so a buffer decided on before a seek's flush can no longer land after it. + private var _epoch: UInt64 = 0 + var epoch: UInt64 { + lock.lock() + defer { lock.unlock() } + return _epoch + } + init() { renderer = AVSampleBufferAudioRenderer() synchronizer = AVSampleBufferRenderSynchronizer() @@ -46,6 +56,23 @@ final class AudioOutput: @unchecked Sendable { synchronizer.rate } + /// AE#395: the renderer's own view of its queue, for the diagnostic line: `status/sufficient/error`. + /// A session that is silent on one route while its clock runs at 1.00 has nothing else that could + /// say whether the renderer is playing what it was given, and the renderer error was only ever + /// logged in DEBUG builds. + var diagRendererState: String { + let status: String + switch renderer.status { + case .rendering: status = "rendering" + case .failed: status = "failed" + case .unknown: status = "unknown" + @unknown default: status = "?" + } + let sufficient = renderer.hasSufficientMediaDataForReliablePlaybackStart ? "y" : "n" + let error = (renderer.error as NSError?).map { "\($0.domain)/\($0.code)" } ?? "-" + return "\(status)/\(sufficient)/\(error)" + } + /// AE#549: how often this renderer has flushed itself, for the diagnostic line. var automaticFlushCount: Int { lock.lock() @@ -56,6 +83,11 @@ final class AudioOutput: @unchecked Sendable { private var automaticFlushObserver: NSObjectProtocol? private var _automaticFlushCount = 0 + /// Where the AE#549 flush runs. The posting thread must never wait on `lock`: `enqueue` holds it + /// across `renderer.enqueue` (DEC-106), and a renderer that posts from inside that call would make the + /// feed thread wait on a lock it already holds, with every `flush()` / `stop()` / `seekClock()` behind it. + private let automaticFlushQueue = DispatchQueue(label: "engine.audio.autoflush") + /// AE#549: the renderer throws its queue away when the route changes under it, and posts the /// timestamp of the first sample it dropped. Nothing in the engine observed that, so the lead /// that was discarded was neither re-fed nor mentioned anywhere. @@ -80,18 +112,21 @@ final class AudioOutput: @unchecked Sendable { guard let self else { return } let flushedFrom = (note.userInfo?[AVSampleBufferAudioRendererFlushTimeKey] as? NSValue)? .timeValue.seconds - lock.lock() - _automaticFlushCount += 1 - let count = _automaticFlushCount - renderer.flush() - lock.unlock() - EngineLog.emit( - "[AudioOutput] AE#549 renderer flushed itself (#\(count)): " - + "dropped from \(flushedFrom.map { String(format: "%.3f", $0) } ?? "unknown")s, " - + "clock at \(String(format: "%.3f", currentTimeSeconds))s rate=\(rate); " - + "audio returns once the feed reaches the clock", - category: .swPlayback - ) + automaticFlushQueue.async { [weak self] in + guard let self else { return } + lock.lock() + _automaticFlushCount += 1 + let count = _automaticFlushCount + renderer.flush() + lock.unlock() + EngineLog.emit( + "[AudioOutput] AE#549 renderer flushed itself (#\(count)): " + + "dropped from \(flushedFrom.map { String(format: "%.3f", $0) } ?? "unknown")s, " + + "clock at \(String(format: "%.3f", currentTimeSeconds))s rate=\(rate); " + + "audio returns once the feed reaches the clock", + category: .swPlayback + ) + } } } @@ -140,6 +175,17 @@ final class AudioOutput: @unchecked Sendable { synchronizer.setRate(0.0, time: at) } + /// Pause the master clock, at `latest` if it has already run past it. A park deferred to the last + /// sample runs whenever its task is scheduled, and on a starved main actor that is after the clock + /// has walked on (#694: 1.149 s on a 1.0 s source on a CI runner). + func pause(notAfter latest: Double) { + let now = synchronizer.currentTime() + let seconds = SoftwareEndOfMediaClock.parkSeconds(clockSeconds: CMTimeGetSeconds(now), notAfter: latest) + let at = seconds.map { CMTime(seconds: $0, preferredTimescale: 90000) } ?? now + EngineLog.emit("[AudioOutput] pause at t=\(String(format: "%.3f", at.seconds))", category: .swPlayback) + synchronizer.setRate(0.0, time: at) + } + /// AE#464: set the audio presentation offset. Positive presents audio later than video, which on /// this path means stamping its samples further ahead on the synchronizer's timeline: at clock /// time t the renderer then plays what was recorded at t minus the offset, while the video layer @@ -155,13 +201,41 @@ final class AudioOutput: @unchecked Sendable { /// Enqueue a decoded audio CMSampleBuffer. Always enqueues (renderer buffers internally); gating on /// isReadyForMoreMediaData dropped early samples before the synchronizer started, giving silence. /// + /// Audit DEC-106: with `epoch`, the buffer is enqueued only if no flush has retired that epoch, and the + /// comparison and the enqueue happen under the lock `flush` takes. Returns false, enqueuing nothing, + /// for a buffer that predates a flush. + /// /// AE#464: this is where a lip-sync offset is applied, and the position is the point. It is past /// the audio tap (whose `sourceTime` is documented as the SOURCE axis and feeds transcription), /// past the decoder's gapless clock (which would absorb a sub-100 ms offset as rounding), and /// past the caller's `lastEnqueuedAudioPtsSec` bookkeeping (whose lead is measured against the /// synchronizer clock, i.e. against the source axis too). Only the renderer sees the shift. - func enqueue(sampleBuffer: CMSampleBuffer) { - renderer.enqueue(retimed(sampleBuffer)) + @discardableResult + func enqueue(sampleBuffer: CMSampleBuffer, ifEpoch epoch: UInt64? = nil) -> Bool { + lock.lock() + if let epoch, epoch != _epoch { + lock.unlock() + return false + } + let offset = presentationOffset + let delivered = offset == .zero ? sampleBuffer : Self.retimed(sampleBuffer, by: offset) + var offsetLine: String? + if offset != .zero, !loggedOffsetInEffect, delivered !== sampleBuffer { + loggedOffsetInEffect = true + // Release-visible, once per offset change: an offset that was set and an offset that is being + // DELIVERED are different claims, and without this line the difference is only measurable with + // a capture card. The two timestamps are the whole proof. + let source = CMSampleBufferGetPresentationTimeStamp(sampleBuffer).seconds + offsetLine = "[AudioOutput] AE#464 audio delay in effect: " + + String(format: "%+.0f ms", offset.seconds * 1000) + + String(format: " (sample at %.3fs delivered at %.3fs)", source, source + offset.seconds) + } + renderer.enqueue(delivered) + #if DEBUG + afterRendererEnqueueForTesting?() + #endif + lock.unlock() + if let offsetLine { EngineLog.emit(offsetLine, category: .swPlayback) } #if DEBUG // Once per session: first enqueue + any renderer rejection, to distinguish "nothing enqueued" from @@ -180,43 +254,21 @@ final class AudioOutput: @unchecked Sendable { EngineLog.emit("[AudioOutput] renderer error: \(err)", category: .swPlayback) } #endif + return true } #if DEBUG + /// Test-only: runs inside `enqueue`'s locked section, right after the renderer took the buffer, which + /// is where a renderer that posts its notifications from inside `enqueue` would post them. + var afterRendererEnqueueForTesting: (@Sendable () -> Void)? private var _loggedFirstEnqueue = false private var _loggedRendererError = false #endif - /// A copy of `sampleBuffer` shifted by the current offset, or the buffer itself when there is - /// none (the overwhelmingly common case, and one that must not cost an allocation). A copy that - /// cannot be made is delivered unshifted: an audible lip-sync error is a far better outcome than - /// a dropped buffer, which is silence. - private func retimed(_ sampleBuffer: CMSampleBuffer) -> CMSampleBuffer { - lock.lock() - let offset = presentationOffset - lock.unlock() - guard offset != .zero else { return sampleBuffer } - let shifted = Self.retimed(sampleBuffer, by: offset) - - // Release-visible, once per offset change: an offset that was set and an offset that is being - // DELIVERED are different claims, and without this line the difference is only measurable with - // a capture card. The two timestamps are the whole proof. - if !loggedOffsetInEffect, shifted !== sampleBuffer { - loggedOffsetInEffect = true - let source = CMSampleBufferGetPresentationTimeStamp(sampleBuffer).seconds - EngineLog.emit( - "[AudioOutput] AE#464 audio delay in effect: " - + String(format: "%+.0f ms", offset.seconds * 1000) - + String(format: " (sample at %.3fs delivered at %.3fs)", source, source + offset.seconds), - category: .swPlayback - ) - } - return shifted - } - - /// The timing half, pure so the shift can be checked without a renderer. Every timing entry moves + /// The shift itself, pure so it can be checked without a renderer. Every timing entry moves /// by `offset`, presentation and decode alike; an entry with no valid presentation stamp is left - /// alone rather than given one. + /// alone rather than given one. A copy that cannot be made is delivered unshifted: an audible + /// lip-sync error is a far better outcome than a dropped buffer, which is silence. static func retimed(_ sampleBuffer: CMSampleBuffer, by offset: CMTime) -> CMSampleBuffer { guard offset != .zero else { return sampleBuffer } var count: CMItemCount = 0 @@ -268,12 +320,14 @@ final class AudioOutput: @unchecked Sendable { func flush() { lock.lock() defer { lock.unlock() } + _epoch &+= 1 renderer.flush() } func stop() { lock.lock() defer { lock.unlock() } + _epoch &+= 1 synchronizer.setRate(0.0, time: .zero) renderer.flush() } diff --git a/Sources/AetherEngine/Audio/AudioPlaybackHost.swift b/Sources/AetherEngine/Audio/AudioPlaybackHost.swift index 2e41867c0..d804ecd9d 100644 --- a/Sources/AetherEngine/Audio/AudioPlaybackHost.swift +++ b/Sources/AetherEngine/Audio/AudioPlaybackHost.swift @@ -64,16 +64,42 @@ final class AudioPlaybackHost { private var lastRate: Float = 1.0 - /// Source-position seconds the host opened at; the demux loop aligns the master clock to the first - /// decoded sample's PTS. `.zero` on cold start, resume offset on a start-position load. + /// Source-position seconds the host opened at; the demux loop resolves the master clock's anchor from it + /// and the first decoded sample's PTS (`clockAnchor`). `.zero` on cold start, resume offset on a + /// start-position load. private var initialClockTime: CMTime = .zero + /// Where the clock is anchored for a first decoded sample at `firstPTS`, and the session zero the host + /// then subtracts from the raw clock. The anchor stays `initialClockTime` for a source that starts where + /// it was asked to (a resume lands within `SWClockAnchorPolicy.toleranceSeconds`), and moves to the + /// sample for one whose timestamps start far past it: a clock at 0 under buffers stamped 3600 s never + /// reaches them, and the back-pressure gate parks the loop against the gap (audit DEC-102). + nonisolated static func clockAnchor(initialClockTime: CMTime, firstPTS: CMTime) + -> (anchor: CMTime, sessionZeroSeconds: Double) { + let resolution = SWClockAnchorPolicy.resolve( + initialSeconds: initialClockTime.seconds, + firstSampleSeconds: firstPTS.isValid ? firstPTS.seconds : Double.nan) + let anchor = resolution.anchorSeconds == initialClockTime.seconds + ? initialClockTime + : CMTime(seconds: resolution.anchorSeconds, preferredTimescale: 90000) + return (anchor, resolution.sessionZeroSeconds) + } + + /// The position published for a raw synchronizer time. + nonisolated static func publishedTime(raw: Double, sessionZeroSeconds: Double) -> Double { + sessionZeroSeconds > 0 ? max(0, raw - sessionZeroSeconds) : raw + } + /// Latched once the first `play()` has spun up the demux loop. private var demuxLoopStarted: Bool = false /// True between pause() and next play() so play() resumes the synchronizer rate (mirrors SoftwarePlaybackHost.pausedByHost). private var pausedByHost: Bool = false + /// #694: latched when end of media parks the clock (AE#374 on this host). Keeps `play()` from + /// restarting a clock the source stopped; a seek clears it. + private var didParkClockAtEnd = false + /// Shared clock-armed latch (mirrors SoftwarePlaybackHost._clockArmed): demux loop arms once on first decoded /// packet; seek() anchors directly and sets this so the loop doesn't snap back to the stale initial anchor. nonisolated(unsafe) private var _clockArmed = false @@ -82,6 +108,16 @@ final class AudioPlaybackHost { set { flagsLock.lock(); _clockArmed = newValue; flagsLock.unlock() } } + /// Audit DEC-102: the offset between the source's own timestamps and the published position, set once by + /// the demux loop when it anchors the clock on a first sample that lies past the load anchor (an Ogg + /// radio stream's granule position, an audio-only TS that joined mid-broadcast). 0 for a source that + /// starts where it was asked to. Same meaning and same policy as `SoftwarePlaybackHost.clockSessionZero`. + nonisolated(unsafe) private var _sessionZeroSeconds: Double = 0 + nonisolated private var sessionZeroSeconds: Double { + get { flagsLock.lock(); defer { flagsLock.unlock() }; return _sessionZeroSeconds } + set { flagsLock.lock(); _sessionZeroSeconds = newValue; flagsLock.unlock() } + } + /// Bumped by every seek(); demux loop resets its enqueue high-water mark on change so the back-pressure /// gate can't park against a pre-seek mark after a backward seek. nonisolated(unsafe) private var _seekGeneration: UInt64 = 0 @@ -145,8 +181,9 @@ final class AudioPlaybackHost { self.audioDecoder = aDec self.audioStreamIndex = resolvedAudioIdx self.audioOutput = AudioOutput() + self.audioOutput?.volume = volume - if let start = startPosition, start > 0 { + if let start = startPosition, start.isFinite, start > 0 { // #254: same off-main, deadline-bounded reposition the transport seek uses. Also load()'s // only suspension point, so the only place a stop() can land mid-load. _ = await dem.seekBounded(to: start, timeout: Self.seekBudgetSeconds, on: seekQueue) @@ -172,9 +209,9 @@ final class AudioPlaybackHost { hostPaused: pausedByHost, clockArmed: clockArmed && demuxLoopStarted, synchronizerRate: audioOutput?.rate ?? 0, - // This host has neither a rebuffer that stops the clock nor an end-of-media park. + // This host has no rebuffer that stops the clock. rebuffering: false, - parkedAtEndOfMedia: false + parkedAtEndOfMedia: didParkClockAtEnd ) { case .resumeHostPause: pausedByHost = false @@ -216,6 +253,10 @@ final class AudioPlaybackHost { } var clockRateForTesting: Float? { audioOutput?.rate } + var clockSecondsForTesting: Double? { audioOutput?.currentTimeSeconds } + var isClockArmedForTesting: Bool { clockArmed } + var sessionZeroForTesting: Double { sessionZeroSeconds } + var outputVolumeForTesting: Float? { audioOutput?.volume } #endif func pause() { @@ -254,6 +295,7 @@ final class AudioPlaybackHost { // and it invalidates in-flight packets from the moment the seek starts rather than after it. bumpSeekGeneration() let generation = seekGeneration + didParkClockAtEnd = false // #292: inside another seek's window `isPlaying` is that seek's parked flag, not the transport's // intent. Inherit what it captured, and hand the same value on to whoever supersedes this one. let wasPlaying = SeekResumeIntent.resolve(isPlaying: isPlaying, @@ -265,10 +307,14 @@ final class AudioPlaybackHost { audioDecoder?.flush() audioOutput?.flush() + // `seconds` is the published axis; the demuxer and the synchronizer clock speak the source's own + // timestamps, so the target is carried over before it reaches either (audit DEC-102). + let sourceSeconds = SWClockAnchorPolicy.sourceSeconds( + forSession: seconds, sessionZeroSeconds: sessionZeroSeconds) currentTime = seconds seekInFlight = true let outcome = await dem.seekBounded( - to: seconds, timeout: Self.seekBudgetSeconds, on: seekQueue, + to: sourceSeconds, timeout: Self.seekBudgetSeconds, on: seekQueue, isSuperseded: { [weak self] in self?.seekGeneration != generation }) guard seekGeneration == generation, !stopRequested else { return .superseded } seekInFlight = false @@ -281,7 +327,7 @@ final class AudioPlaybackHost { } currentTime = seconds - let targetTime = CMTime(seconds: seconds, preferredTimescale: 90000) + let targetTime = CMTime(seconds: sourceSeconds, preferredTimescale: 90000) guard demuxLoopStarted else { // Cold seek (no play() yet): stash target so the loop's first decoded packet anchors there, not at .zero. initialClockTime = targetTime @@ -320,9 +366,9 @@ final class AudioPlaybackHost { isReady = false } - var volume: Float { - get { audioOutput?.volume ?? 1.0 } - set { audioOutput?.volume = newValue } + /// #660: held here, not only on the output, because the engine sets it before `load()` builds one. + var volume: Float = 1.0 { + didSet { audioOutput?.volume = volume } } // MARK: - Demux loop @@ -340,15 +386,18 @@ final class AudioPlaybackHost { let getClockArmed: @Sendable () -> Bool = { [weak self] in self?.clockArmed ?? true } let setClockArmed: @Sendable () -> Void = { [weak self] in self?.clockArmed = true } let getSeekGeneration: @Sendable () -> UInt64 = { [weak self] in self?.seekGeneration ?? 0 } + let onClockAnchored: @Sendable (Double) -> Void = { [weak self] zero in self?.sessionZeroSeconds = zero } let onError: @Sendable (String) -> Void = { [weak self] msg in Task { @MainActor [weak self] in self?.failure = PlaybackErrorInfo(kind: .audioSessionFailed, message: msg) } } - let onEnd: @Sendable () -> Void = { [weak self] in + let onEnd: @Sendable (UInt64, Double) -> Void = { [weak self] generation, lastEnqueuedEnd in Task { @MainActor [weak self] in - self?.didReachEnd = true - self?.isPlaying = false + guard let self, self.seekGeneration == generation else { return } + self.parkClockAtEndOfMedia(lastEnqueuedEnd: lastEnqueuedEnd) + self.didReachEnd = true + self.isPlaying = false } } @@ -365,6 +414,7 @@ final class AudioPlaybackHost { stopRequested: getStopRequested, clockArmed: getClockArmed, armClock: setClockArmed, + onClockAnchored: onClockAnchored, seekGeneration: getSeekGeneration, onError: onError, onEnd: onEnd @@ -386,9 +436,10 @@ final class AudioPlaybackHost { stopRequested: @Sendable () -> Bool, clockArmed: @Sendable () -> Bool, armClock: @Sendable () -> Void, + onClockAnchored: @Sendable (Double) -> Void, seekGeneration: @Sendable () -> UInt64, onError: @Sendable (String) -> Void, - onEnd: @Sendable () -> Void + onEnd: @Sendable (UInt64, Double) -> Void ) { // Clock-armed latch is SHARED with the host: anchor the clock exactly once on the first decoded packet. // seekClock is NOT idempotent (re-sets rate+time), so per-packet calls would snap the clock back ~47x/sec @@ -400,7 +451,8 @@ final class AudioPlaybackHost { // playback end and bounds decoded-PCM memory in the renderer. let maxBufferAhead: Double = 8.0 // Source-time seconds of the last sample handed to the renderer. lastEnqueuedEnd and currentTimeSeconds - // share the source-PTS timeline (clock anchored to initialClockTime), so their difference is seconds queued ahead. + // share the source-PTS timeline (the clock is anchored on the source axis, see `clockAnchor`), so their + // difference is seconds queued ahead. var lastEnqueuedEnd: Double = 0 var seenSeekGeneration = seekGeneration() @@ -436,10 +488,16 @@ final class AudioPlaybackHost { if stopRequested() { return false } } + // Audit DEC-106: read before the packet, so a flush landing anywhere after this retires every + // buffer decided on below, inside `enqueue`, where the comparison is atomic with the enqueue. + let epochBeforeRead = audioOutput?.epoch ?? 0 let packet: UnsafeMutablePointer? do { packet = try demuxer.readPacket() } catch { + // Audit SEG-104: a stop closes the demuxer, which aborts a parked read. That is the + // stop arriving, not a playback failure. + if stopRequested() { return false } EngineLog.emit("[AudioHost] demux read failed: \(error)", category: .swPlayback) onError("Playback error: \(error.localizedDescription)") return false @@ -451,9 +509,17 @@ final class AudioPlaybackHost { // tail. flush() alone dropped the final ~21ms+ of every audio-only title. if let aDec = audioDecoder, let aOut = audioOutput, seekGeneration() == seenSeekGeneration { - let tail = aDec.drain() - for buf in tail { aOut.enqueue(sampleBuffer: buf) } - if let last = tail.last { + // Audit DEC-2: a seek's flush can land inside the drain, as in `decode` below. + let drained = aDec.drain() + let tail = seekGeneration() == seenSeekGeneration ? drained : [] + var tailAccepted = true + for buf in tail { + guard aOut.enqueue(sampleBuffer: buf, ifEpoch: epochBeforeRead) else { + tailAccepted = false + break + } + } + if tailAccepted, let last = tail.last { let end = CMTimeGetSeconds(CMSampleBufferGetPresentationTimeStamp(last)) + CMTimeGetSeconds(CMSampleBufferGetDuration(last)) if end.isFinite, end > lastEnqueuedEnd { lastEnqueuedEnd = end } @@ -478,7 +544,7 @@ final class AudioPlaybackHost { } if seekedAway { return true } if seekGeneration() != seenSeekGeneration { return true } - onEnd() + onEnd(seenSeekGeneration, lastEnqueuedEnd) return false } @@ -493,8 +559,19 @@ final class AudioPlaybackHost { if packet.pointee.stream_index == audioStreamIndex, let aDec = audioDecoder, let aOut = audioOutput { let buffers = aDec.decode(packet: packet) + // Audit DEC-2 (the AE#491 rule on the audio-only host): the seek's flush can land + // inside `decode`, so the buffers are checked out again. + if seekGeneration() != seenSeekGeneration { + av_packet_unref(packet) + av_packet_free_safe(packet) + return true + } for buf in buffers { - aOut.enqueue(sampleBuffer: buf) + guard aOut.enqueue(sampleBuffer: buf, ifEpoch: epochBeforeRead) else { + av_packet_unref(packet) + av_packet_free_safe(packet) + return true + } } if let last = buffers.last { let end = CMTimeGetSeconds(CMSampleBufferGetPresentationTimeStamp(last)) @@ -502,7 +579,21 @@ final class AudioPlaybackHost { if end.isFinite, end > lastEnqueuedEnd { lastEnqueuedEnd = end } } if !clockArmed(), !buffers.isEmpty { - aOut.seekClock(to: initialClockTime, rate: initialRate) + let resolved = clockAnchor( + initialClockTime: initialClockTime, + firstPTS: CMSampleBufferGetPresentationTimeStamp(buffers[0])) + if resolved.sessionZeroSeconds > 0 { + EngineLog.emit( + "[AudioHost] clock anchored at the first sample: " + + "anchor=\(String(format: "%.3f", resolved.anchor.seconds))s " + + "(load anchor \(String(format: "%.3f", initialClockTime.seconds))s, " + + "sessionZero=\(String(format: "%.3f", resolved.sessionZeroSeconds))s)", + category: .swPlayback + ) + // Before the clock moves, so no tick can publish the raw position in between. + onClockAnchored(resolved.sessionZeroSeconds) + } + aOut.seekClock(to: resolved.anchor, rate: initialRate) armClock() } } @@ -520,6 +611,50 @@ final class AudioPlaybackHost { } } + /// #694: stop the master clock on the last sample instead of letting it free-run past the end + /// (AE#374 on the software host). The playthrough wait in the demux loop releases up to 0.25 s + /// before the last enqueued sample, so the park is deferred by what is still queued: parking stops + /// the renderer too and an immediate park would cut that tail. Either way the clock stops on the + /// last sample, not where it stands when the park runs. + private func parkClockAtEndOfMedia(lastEnqueuedEnd: Double) { + guard !didParkClockAtEnd else { return } + didParkClockAtEnd = true + guard clockArmed, let aOut = audioOutput else { return } + let clock = aOut.currentTimeSeconds + let tail = SoftwareEndOfMediaClock.tailPlayoutSeconds( + clockSeconds: clock, + lastAudioPts: lastEnqueuedEnd + ) + guard tail > 0 else { return parkClockNow(notAfter: lastEnqueuedEnd) } + // The tail is source time: below 1x, or across a pause, it outlasts as many wall seconds, so + // the park waits for the clock to get there. `target` keeps the clamp on a broken PTS. + let target = clock + tail + let generation = seekGeneration + Task { @MainActor [weak self] in + var wait = tail + while true { + try? await Task.sleep(nanoseconds: UInt64(wait * 1_000_000_000)) + guard let self, self.seekGeneration == generation, self.didParkClockAtEnd, + let aOut = self.audioOutput else { return } + guard let next = SoftwareEndOfMediaClock.parkWaitSeconds( + clockSeconds: aOut.currentTimeSeconds, rate: aOut.rate, target: target + ) else { return self.parkClockNow(notAfter: lastEnqueuedEnd) } + wait = next + } + } + } + + private func parkClockNow(notAfter latest: Double) { + guard !stopRequested, let aOut = audioOutput else { return } + aOut.pause(notAfter: latest) + rate = 0 + EngineLog.emit( + "[AudioHost] end of media: clock parked at " + + "\(String(format: "%.3f", aOut.currentTimeSeconds))s", + category: .swPlayback + ) + } + // MARK: - Time updates private func startTimeUpdates() { @@ -532,7 +667,7 @@ final class AudioPlaybackHost { guard !self.seekInFlight else { return } let t = aOut.currentTimeSeconds if t.isFinite, t >= 0 { - self.currentTime = t + self.currentTime = Self.publishedTime(raw: t, sessionZeroSeconds: self.sessionZeroSeconds) } } } diff --git a/Sources/AetherEngine/Audio/AudioRouteDescription.swift b/Sources/AetherEngine/Audio/AudioRouteDescription.swift new file mode 100644 index 000000000..b1307a20c --- /dev/null +++ b/Sources/AetherEngine/Audio/AudioRouteDescription.swift @@ -0,0 +1,43 @@ +import Foundation +import AVFoundation + +/// The audio route as one log fragment, shared by every host that writes a route line, so the native and +/// the software path describe the same route in the same words and a diff between their logs is a diff +/// of routes, not of formats. +enum AudioRouteDescription { + + /// `output=… preferred=… max=… ports=[name[type, ch=n], …] latency=…ms io=…ms`, or nil where there is + /// no `AVAudioSession`. The output latency is the field AE#395 needs: a buffered AirPlay route delays + /// sound by seconds where HDMI delays it by milliseconds, and a feed that is fine on one is late on + /// the other. + static func current() -> String? { + #if os(iOS) || os(tvOS) + let session = AVAudioSession.sharedInstance() + let ports = session.currentRoute.outputs.map { port in + "\(port.portName)[\(port.portType.rawValue), ch=\(port.channels?.count ?? -1)]" + }.joined(separator: ", ") + return "output=\(session.outputNumberOfChannels) preferred=\(session.preferredOutputNumberOfChannels) " + + "max=\(session.maximumOutputNumberOfChannels) ports=[\(ports)] " + + "latency=\(String(format: "%.0f", session.outputLatency * 1000))ms " + + "io=\(String(format: "%.1f", session.ioBufferDuration * 1000))ms" + #else + return nil + #endif + } + + #if os(iOS) || os(tvOS) + /// AE#395: one line per route change, for the process rather than per engine (a multiview runs several). + /// A change mid-session leaves the session's start line describing a route that is gone, and a user + /// switching output to test a theory is exactly that case. Installed on first touch, never removed. + static let changeLogger: Void = { + _ = NotificationCenter.default.addObserver( + forName: AVAudioSession.routeChangeNotification, + object: AVAudioSession.sharedInstance(), queue: nil + ) { note in + let reason = (note.userInfo?[AVAudioSessionRouteChangeReasonKey] as? UInt).map(String.init) ?? "?" + guard let route = current() else { return } + EngineLog.emit("[AetherEngine] audioRoute changed reason=\(reason) \(route)", category: .engine) + } + }() + #endif +} diff --git a/Sources/AetherEngine/Audio/Spatial/SpatialAudioBridge.swift b/Sources/AetherEngine/Audio/Spatial/SpatialAudioBridge.swift index 431912f80..d5b344abd 100644 --- a/Sources/AetherEngine/Audio/Spatial/SpatialAudioBridge.swift +++ b/Sources/AetherEngine/Audio/Spatial/SpatialAudioBridge.swift @@ -72,6 +72,13 @@ final class SpatialAudioBridge: AudioTranscodingBridge, @unchecked Sendable { private var stats = AudioBridge.FeedStats() private(set) var outputBytesLifetime: Int64 = 0 + /// AE#641: called once, on the pump thread, when the object decoder has been fed + /// `AudioBridge.silentFeedPacketThreshold` packets in a row without returning a block. The APAC + /// sample entry is written up front, like FLAC's, so nothing downstream fails on its own and the + /// session would otherwise play the picture over silence. Set before the producer starts feeding. + var onDecoderProducedNothing: (@Sendable (AudioBridge.FeedStats) -> Void)? + private var decodedNothingReported = false + private static let avNoPTS: Int64 = -0x7FFFFFFFFFFFFFFF - 1 init( @@ -140,6 +147,8 @@ final class SpatialAudioBridge: AudioTranscodingBridge, @unchecked Sendable { defer { opLock.unlock() } stats.packetsFed += 1 stats.packetsFedSinceLastEnqueue += 1 + // Every exit: a decoder that rejects each packet has decoded nothing too. + defer { noteDecodedNothingIfNeeded() } if originFrame == nil, packet.pointee.pts != Self.avNoPTS { originFrame = av_rescale_q(packet.pointee.pts, srcTimeBase, encoderTimeBase) @@ -166,6 +175,17 @@ final class SpatialAudioBridge: AudioTranscodingBridge, @unchecked Sendable { return results } + /// Caller holds `opLock`. + private func noteDecodedNothingIfNeeded() { + guard !decodedNothingReported, stats.decodedNothing, + stats.packetsFedSinceLastEnqueue >= AudioBridge.silentFeedPacketThreshold else { return } + decodedNothingReported = true + EngineLog.emit( + "[SpatialAudioBridge] ERROR: AE#641 the TrueHD object decoder has produced nothing: " + + "\(stats.summary)", category: .session) + onDecoderProducedNothing?(stats) + } + func flush() -> [UnsafeMutablePointer] { opLock.lock() defer { opLock.unlock() } diff --git a/Sources/AetherEngine/Audio/Tap/AudioTapHLSFetcher.swift b/Sources/AetherEngine/Audio/Tap/AudioTapHLSFetcher.swift index d66211c34..4b8a3bfcb 100644 --- a/Sources/AetherEngine/Audio/Tap/AudioTapHLSFetcher.swift +++ b/Sources/AetherEngine/Audio/Tap/AudioTapHLSFetcher.swift @@ -21,12 +21,14 @@ final class AudioTapHLSFetcher: @unchecked Sendable { private let session: URLSession /// Same per-stream headers the player's AVURLAsset sends (#119); header-enforcing origins - /// 403 the tap's playlist / segment / key fetches without them. - private let httpHeaders: [String: String] + /// 403 the tap's playlist / segment / key fetches without them. The credential headers go only + /// to the origin of `credentialOrigin`, the URL the host loaded: every other URL here is one a + /// playlist named, on any host or scheme (audit DEC-103, the NET-7 rule). + private let credentials: CredentialScope private let keyCacheLock = NSLock() private var keyCache: [String: Data] = [:] - init(session: URLSession? = nil, httpHeaders: [String: String] = [:]) { + init(session: URLSession? = nil, httpHeaders: [String: String] = [:], credentialOrigin: URL? = nil) { if let session { self.session = session } else { @@ -36,19 +38,29 @@ final class AudioTapHLSFetcher: @unchecked Sendable { self.session = URLSession( configuration: cfg, delegate: EngineTLS.sessionDelegate, delegateQueue: nil) } - self.httpHeaders = httpHeaders + credentials = CredentialScope(headers: httpHeaders, anchor: credentialOrigin) } - private func get(_ url: URL) async throws -> (Data, URLResponse) { + /// Audit DEC-103: every body is cut off at its cap while it arrives. Playlists take the ingest + /// readers' 8 MiB, keys 64 bytes, and a segment, which here is audio only, the floor of the + /// duration-derived video cap. + private static let maximumPlaylistBytes = 8 * 1024 * 1024 + private static let maximumSegmentBytes = BoundedFetch.segmentLimit(forDuration: 0) + + private func get(_ url: URL, limit: Int) async throws -> (Data, URLResponse) { var request = URLRequest(url: url) - for (field, value) in httpHeaders { + for (field, value) in credentials.headers(for: url) { request.setValue(value, forHTTPHeaderField: field) } - return try await session.data(for: request) + do { + return try await BoundedFetch.data(for: request, session: session, limit: limit) + } catch let error as BoundedFetch.Exceeded { + throw FetchError.invalidPlaylist("body exceeds \(error.limit) bytes") + } } func fetchPlaylist(_ url: URL) async throws -> (HLSPlaylist, URL) { - let (data, response) = try await get(url) + let (data, response) = try await get(url, limit: Self.maximumPlaylistBytes) let status = (response as? HTTPURLResponse)?.statusCode ?? -1 guard (200..<300).contains(status) else { throw FetchError.http(status) } guard let text = String(data: data, encoding: .utf8) else { @@ -58,7 +70,7 @@ final class AudioTapHLSFetcher: @unchecked Sendable { } func fetchSegment(_ url: URL, crypt: HLSSegmentCrypt?, base: URL) async throws -> Data { - let (data, response) = try await get(url) + let (data, response) = try await get(url, limit: Self.maximumSegmentBytes) let status = (response as? HTTPURLResponse)?.statusCode ?? -1 if status == 404 { return Data() } // slid out of window; caller advances guard (200..<300).contains(status) else { throw FetchError.http(status) } @@ -76,7 +88,7 @@ final class AudioTapHLSFetcher: @unchecked Sendable { private func fetchKey(_ url: URL) async throws -> Data { let cacheKey = url.absoluteString if let cached = keyCacheLock.withLock({ keyCache[cacheKey] }) { return cached } - let (data, response) = try await get(url) + let (data, response) = try await get(url, limit: BoundedFetch.keyLimit) let status = (response as? HTTPURLResponse)?.statusCode ?? -1 guard (200..<300).contains(status), data.count == 16 else { throw FetchError.http(status) } keyCacheLock.withLock { keyCache[cacheKey] = data } diff --git a/Sources/AetherEngine/BackgroundTeardownSelection.swift b/Sources/AetherEngine/BackgroundTeardownSelection.swift index aa2e4603a..314e54ffb 100644 --- a/Sources/AetherEngine/BackgroundTeardownSelection.swift +++ b/Sources/AetherEngine/BackgroundTeardownSelection.swift @@ -19,12 +19,12 @@ extension AetherEngine { /// Park the current selection for the reload that follows this teardown. Called by both #127 /// teardown paths (grace expiry and the synchronous assertion backstop) BEFORE `stopInternal`. + /// + /// Audit CORE-3: merged over what is already parked rather than replacing it. A second teardown + /// before the reload claims the first (an iOS `pause()` while backgrounded re-arms the grace + /// window) reads a session `stopInternal` has already wiped, and used to park that emptiness. func captureBackgroundTeardownSelection() { - backgroundTeardownSelection = BackgroundTeardownSelection( - subtitles: captureSubtitleSessionCarryover(), - audioTrackIndex: activeAudioTrackIndex, - discTitleID: activeDiscTitleID - ) + backgroundTeardownSelection = liveSelection(over: backgroundTeardownSelection) } /// The selection a session-preserving reload must restore, claiming any parked teardown @@ -32,7 +32,11 @@ extension AetherEngine { func consumeReloadSelection() -> BackgroundTeardownSelection { let parked = backgroundTeardownSelection backgroundTeardownSelection = nil - return BackgroundTeardownSelection( + return liveSelection(over: parked) + } + + private func liveSelection(over parked: BackgroundTeardownSelection?) -> BackgroundTeardownSelection { + BackgroundTeardownSelection( subtitles: Self.mergedSubtitleCarryover( live: captureSubtitleSessionCarryover(), snapshot: parked?.subtitles), audioTrackIndex: activeAudioTrackIndex ?? parked?.audioTrackIndex, diff --git a/Sources/AetherEngine/Decoder/ColorDescription.swift b/Sources/AetherEngine/Decoder/ColorDescription.swift index 009288ec7..0073fddf4 100644 --- a/Sources/AetherEngine/Decoder/ColorDescription.swift +++ b/Sources/AetherEngine/Decoder/ColorDescription.swift @@ -53,7 +53,9 @@ struct ColorDescription: Equatable { /// The description to act on: the bitstream wherever it committed to a value, the container for the /// fields it left open. Per field, because a partially filled VUI is the common case and the one /// that fails silently (a matrix without a transfer reads HDR to a gate and reads nothing to zimg). - /// Two silences stay silent: an untagged stream is not improved by inventing BT.709 for it. + /// Two silences stay silent: an untagged stream is not improved by inventing BT.709 for it here. + /// The BT.709 a display needs is chosen where the buffer meets CoreVideo, see + /// `ColorAttachments.presented` (AE#654), so the HDR gate and the tone mapper still see the gap. static func resolved(bitstream: ColorDescription, container: ColorDescription) -> ColorDescription { ColorDescription( primaries: bitstream.primaries == AVCOL_PRI_UNSPECIFIED ? container.primaries : bitstream.primaries, diff --git a/Sources/AetherEngine/Decoder/EmbeddedSubtitleDecoder.swift b/Sources/AetherEngine/Decoder/EmbeddedSubtitleDecoder.swift index 1fc274e42..6cb4293fb 100644 --- a/Sources/AetherEngine/Decoder/EmbeddedSubtitleDecoder.swift +++ b/Sources/AetherEngine/Decoder/EmbeddedSubtitleDecoder.swift @@ -641,21 +641,38 @@ final class EmbeddedSubtitleDecoder { // Malformed rect guard: stride < width would read past the plane allocation. guard stride >= width else { return nil } + // AE#628: resolve the palette once. Every pixel below is then one table read instead of a + // palette walk plus a premultiply; on a full-canvas plane that is the difference between 5 ms + // and 1.3 ms per display set in release, and 211 ms and 52 ms in a debug build. + // Premultiplied: straight alpha produces black-fringe edges in CGImage premultipliedLast. + // Packed so a little-endian store lays the bytes out as R, G, B, A. + let alphaThreshold: UInt8 = 8 + var visible = [Bool](repeating: false, count: 256) + var premultiplied = [UInt32](repeating: 0, count: 256) + for i in 0..<256 { + let b = UInt32(palettePtr[i * 4 + 0]) + let g = UInt32(palettePtr[i * 4 + 1]) + let red = UInt32(palettePtr[i * 4 + 2]) + let a = UInt32(palettePtr[i * 4 + 3]) + visible[i] = palettePtr[i * 4 + 3] >= alphaThreshold + premultiplied[i] = ((red * a + 127) / 255) + | (((g * a + 127) / 255) << 8) + | (((b * a + 127) / 255) << 16) + | (a << 24) + } + // Re-crop to non-zero-alpha bounding box: some Blu-ray-to-MKV conversions emit full 1920x1080 ODS bitmaps // with cropping params that FFmpeg's pgssubdec discards, carrying ~8 MB of transparent pixels per cue. - let alphaThreshold: UInt8 = 8 var minX = width, minY = height, maxX = -1, maxY = -1 - for y in 0..= alphaThreshold { - if x < minX { minX = x } - if y < minY { minY = y } - if x > maxX { maxX = x } - if y > maxY { maxY = y } - } + visible.withUnsafeBufferPointer { visible in + for y in 0..= minX, maxY >= minY else { return nil } @@ -665,27 +682,17 @@ final class EmbeddedSubtitleDecoder { let absX = Int(r.x) + minX let absY = Int(r.y) + minY - var rgba = [UInt8](repeating: 0, count: cropW * cropH * 4) - for cy in 0.. Void)? var onA53Captions: (@Sendable ([CCDataParser.CCTriplet], Double) -> Void)? + var onDecodedFormat: (@Sendable (DecodedVideoFormat) -> Void)? + private var streamColor = ColorDescription.unspecified + private var streamCodecID = AV_CODEC_ID_NONE + private var streamProfile = AV_PROFILE_UNKNOWN + private var reportedPixelBufferType: OSType = 0 /// Skip pre-seek RASL frames to avoid the "fast forward" effect; decoded for reference but not delivered. /// Guarded by `skipLock` not `lock`: close() holds `lock` across VTDecompressionSessionWaitForAsynchronousFrames, @@ -162,6 +167,9 @@ final class HardwareVideoDecoder: VideoDecodingPipeline, @unchecked Sendable { let isHDRTransfer = ColorAttachments.isHDRTransfer(codecpar.pointee.color_trc) let use10Bit = bitsPerSample > 8 || isHDRTransfer + streamColor = ColorDescription(codecpar: codecpar) + streamCodecID = codecpar.pointee.codec_id + streamProfile = codecpar.pointee.profile self.colorPrimaries = ColorAttachments.primaries(codecpar.pointee.color_primaries) self.colorTransfer = ColorAttachments.transfer(codecpar.pointee.color_trc) self.colorMatrix = ColorAttachments.matrix(codecpar.pointee.color_space) @@ -222,7 +230,7 @@ final class HardwareVideoDecoder: VideoDecodingPipeline, @unchecked Sendable { lock.lock() // AE#492: see `SoftwareVideoDecoder.decode`. Same rule, same lock as `flush()`. if let epoch, epoch != _feedEpoch { lock.unlock(); return } - guard let session = session, let formatDesc = formatDescription else { + guard session != nil, let formatDesc = formatDescription else { lock.unlock() return } @@ -253,19 +261,10 @@ final class HardwareVideoDecoder: VideoDecodingPipeline, @unchecked Sendable { copied.deallocate() return } - let ptsRaw = packet.pointee.pts - let dtsRaw = packet.pointee.dts - let durRaw = packet.pointee.duration - let timescale = max(timeBase.den, 1) - - let pts = (ptsRaw != Int64.min) - ? CMTimeMake(value: ptsRaw * Int64(timeBase.num), timescale: timescale) - : CMTime.invalid - let dts = (dtsRaw != Int64.min) - ? CMTimeMake(value: dtsRaw * Int64(timeBase.num), timescale: timescale) - : CMTime.invalid - let dur = (durRaw > 0) - ? CMTimeMake(value: durRaw * Int64(timeBase.num), timescale: timescale) + let pts = SourceTimestampBounds.cmTime(ticks: packet.pointee.pts, timeBase: timeBase) + let dts = SourceTimestampBounds.cmTime(ticks: packet.pointee.dts, timeBase: timeBase) + let dur = packet.pointee.duration > 0 + ? SourceTimestampBounds.cmTime(ticks: packet.pointee.duration, timeBase: timeBase) : CMTime.invalid var timing = CMSampleTimingInfo(duration: dur, presentationTimeStamp: pts, decodeTimeStamp: dts) @@ -304,6 +303,12 @@ final class HardwareVideoDecoder: VideoDecodingPipeline, @unchecked Sendable { } } + // Audit DEC-4: the epoch check and the send sit under one hold of `lock`, or a flush landing + // while the sample buffer is built lets a pre-seek packet into VT after it. Safe to hold: + // the output callback never takes `lock`, and `close()` already holds it across the VT wait. + lock.lock() + if let epoch, epoch != _feedEpoch { lock.unlock(); return } + guard let session = self.session else { lock.unlock(); return } // Async decode with temporal queueing; callback fires on VT's internal queue. var infoFlags = VTDecodeInfoFlags() let decodeStatus = VTDecompressionSessionDecodeFrame( @@ -313,9 +318,10 @@ final class HardwareVideoDecoder: VideoDecodingPipeline, @unchecked Sendable { frameRefcon: nil, infoFlagsOut: &infoFlags ) + lock.unlock() if decodeStatus != noErr { EngineLog.emit( - "[HardwareVideoDecoder] decode error \(decodeStatus) at pts=\(ptsRaw)", + "[HardwareVideoDecoder] decode error \(decodeStatus) at pts=\(packet.pointee.pts)", category: .swPlayback ) } @@ -406,8 +412,26 @@ final class HardwareVideoDecoder: VideoDecodingPipeline, @unchecked Sendable { CVBufferRemoveAttachment(imageBuffer, kCVImageBufferPixelAspectRatioKey) } + reportDecodedFormat(imageBuffer) onFrame?(imageBuffer, pts, nil) } + + /// VideoToolbox decodes straight into the display buffer, so the buffer IS the decoded picture; its + /// colour is what this decoder attached, which is the stream's declaration. + private func reportDecodedFormat(_ buffer: CVImageBuffer) { + guard let onDecodedFormat else { return } + let type = CVPixelBufferGetPixelFormatType(buffer) + guard type != reportedPixelBufferType else { return } + reportedPixelBufferType = type + onDecodedFormat(DecodedVideoFormat( + frame: VideoStreamFormat( + pixelFormat: DecodedVideoFormat.libavPixelFormat(forPixelBufferType: type), + declaredBitDepth: 0, + color: streamColor, + codecID: streamCodecID, + profile: streamProfile), + pixelBufferFormat: DecodedVideoFormat.fourCC(type))) + } } // MARK: - C callback diff --git a/Sources/AetherEngine/Decoder/SoftwareVideoDecoder.swift b/Sources/AetherEngine/Decoder/SoftwareVideoDecoder.swift index 9529fa8f7..0ea493f10 100644 --- a/Sources/AetherEngine/Decoder/SoftwareVideoDecoder.swift +++ b/Sources/AetherEngine/Decoder/SoftwareVideoDecoder.swift @@ -22,6 +22,20 @@ final class SoftwareVideoDecoder: VideoDecodingPipeline, @unchecked Sendable { var onFirstHDR10PlusDetected: (@Sendable () -> Void)? var onA53Captions: (@Sendable ([CCDataParser.CCTriplet], Double) -> Void)? + /// AE#658: fires (decode thread) on the first displayed frame and whenever the decoded format or the + /// display buffer's format changes. Guarded by `lock`, like everything `emit` reads. + var onDecodedFormat: (@Sendable (DecodedVideoFormat) -> Void)? + private var decodedFormatKey: DecodedFormatKey? + private var reportedFormatKey: DecodedFormatKey? + + /// The decoder's own output, taken before a deinterlace graph can rewrite the frame (the hardware + /// graph hands `emit` a VideoToolbox surface whose pixel format says nothing about the decode). + struct DecodedFormatKey: Equatable { + var pixelFormat: Int32 + var color: ColorDescription + var pixelBufferType: OSType = 0 + } + /// True when the source is >8-bit (HDR10, AV1 HDR). private var use10Bit = false @@ -41,6 +55,10 @@ final class SoftwareVideoDecoder: VideoDecodingPipeline, @unchecked Sendable { private var pixelBufferPool: CVPixelBufferPool? private var poolWidth = 0 private var poolHeight = 0 + private var poolFormat = OutputFormat(tenBit: false, fullRange: false) + /// The format the last converted frame asked for, so a change is logged once (release-visible) rather + /// than per frame. Guarded by `lock`, like the pool. + private var loggedOutputFormat: OutputFormat? /// Skip pre-seek frames; decoded for reference but not converted. /// Guarded by `skipLock` not `lock`: emit() runs with `lock` held, so a same-lock accessor would deadlock. @@ -98,6 +116,48 @@ final class SoftwareVideoDecoder: VideoDecodingPipeline, @unchecked Sendable { return max(1, min(16, activeProcessorCount)) } + /// The pixel buffer format this decoder hands the display layer, read off the decoded frame (audit + /// DEC-107). The container's declaration alone missed two cases: `color_range` was never read, so a + /// full-range AV1 / VP9 / HEVC picture was labelled video range and shown crushed, and + /// `bits_per_raw_sample` is 0 for libdav1d and the AV1 / VP9 parsers, so a 10-bit SDR stream was + /// dithered down to 8 bit. + struct OutputFormat: Equatable { + var tenBit: Bool + var fullRange: Bool + + var pixelFormatType: OSType { + switch (tenBit, fullRange) { + case (false, false): return kCVPixelFormatType_420YpCbCr8BiPlanarVideoRange + case (false, true): return kCVPixelFormatType_420YpCbCr8BiPlanarFullRange + case (true, false): return kCVPixelFormatType_420YpCbCr10BiPlanarVideoRange + case (true, true): return kCVPixelFormatType_420YpCbCr10BiPlanarFullRange + } + } + + var swscaleFormat: AVPixelFormat { tenBit ? AV_PIX_FMT_P010LE : AV_PIX_FMT_NV12 } + } + + /// `streamIs10Bit` is the open-time verdict (declared depth or an HDR transfer) and only ever widens + /// the answer. The range is a LABEL: the legacy `sws_scale` reads a source's range from the pixel + /// format alone (the yuvj family), so a full-range frame in a plain 4:2:0 format is copied across + /// untouched and only has to be tagged as what it is. Asking swscale to convert would take 4K off its + /// unscaled NV12 / P010 path, and a yuvj source IS converted to video range by swscale itself, so it + /// stays video range here. + nonisolated static func outputFormat(pixelFormat: Int32, colorRange: AVColorRange, + streamIs10Bit: Bool) -> OutputFormat { + let format = AVPixelFormat(rawValue: pixelFormat) + let depth = av_pix_fmt_desc_get(format)?.pointee.comp.0.depth ?? 8 + let convertedBySwscale: Bool + switch format { + case AV_PIX_FMT_YUVJ420P, AV_PIX_FMT_YUVJ422P, AV_PIX_FMT_YUVJ444P, AV_PIX_FMT_YUVJ440P: + convertedBySwscale = true + default: + convertedBySwscale = false + } + return OutputFormat(tenBit: streamIs10Bit || depth > 8, + fullRange: colorRange == AVCOL_RANGE_JPEG && !convertedBySwscale) + } + /// AE#499: what the container declared about colour, captured at `open` before a single frame /// exists. A decoded frame carries the VUI alone, and a remux whose VUI is empty would otherwise /// reach `attachColorSpace` as an untagged picture, so an HDR10 file decoded in software lost its @@ -254,6 +314,8 @@ final class SoftwareVideoDecoder: VideoDecodingPipeline, @unchecked Sendable { if Self.disposition(forSendResult: sendRet) == .drainAndRetry { drainDecodedFrames() lock.lock() + // Audit DEC-1: the drain drops the lock between frames, so a flush can land in it. + if let epoch, epoch != _feedEpoch { lock.unlock(); return } sendRet = codecContext == nil ? FFmpegErr.einval : avcodec_send_packet(ctx, packet) lock.unlock() } @@ -287,6 +349,7 @@ final class SoftwareVideoDecoder: VideoDecodingPipeline, @unchecked Sendable { // AE#499: fill the fields the VUI left open from the container's declaration BEFORE any // consumer reads the frame, for the same reason the timestamp repair below runs here. ColorDescription.backfill(frame: f, container: containerColor) + decodedFormatKey = DecodedFormatKey(pixelFormat: f.pointee.format, color: ColorDescription(frame: f)) // #407: repair the frame's own timestamp BEFORE anything reads it. A frame that reaches // the renderer with no PTS is unschedulable and gets dropped there, so every consumer @@ -386,12 +449,9 @@ final class SoftwareVideoDecoder: VideoDecodingPipeline, @unchecked Sendable { } private func emitInner(_ f: UnsafeMutablePointer, timeBase tb: AVRational) { - if let threshold = skipUntilPTS, f.pointee.pts != Int64.min { - let framePTS = CMTimeMake( - value: f.pointee.pts * Int64(tb.num), - timescale: Int32(tb.den) - ) - if CMTimeCompare(framePTS, threshold) < 0 { + let cmPTS = SourceTimestampBounds.cmTime(ticks: f.pointee.pts, timeBase: tb) + if let threshold = skipUntilPTS, cmPTS.isValid { + if CMTimeCompare(cmPTS, threshold) < 0 { return } // Compare-and-clear: a concurrent seek can install a new threshold; blindly nil-ing would discard it. @@ -427,16 +487,7 @@ final class SoftwareVideoDecoder: VideoDecodingPipeline, @unchecked Sendable { pixelBuffer = converted } - let pts = f.pointee.pts - let cmPTS: CMTime - if pts != Int64.min { - cmPTS = CMTimeMake( - value: pts * Int64(tb.num), - timescale: Int32(tb.den) - ) - } else { - cmPTS = .invalid - } + reportDecodedFormat(pixelBuffer: pixelBuffer) // HDR10+: read dynamic metadata from post-decode AVFrame side data (T.35 SEI bytes). // Can't reuse the VT path's packet-side stash; this decoder owns its own packet flow. @@ -514,6 +565,7 @@ final class SoftwareVideoDecoder: VideoDecodingPipeline, @unchecked Sendable { pixelBufferPool = nil poolWidth = 0 poolHeight = 0 + loggedOutputFormat = nil if let session = transferSession { VTPixelTransferSessionInvalidate(session) transferSession = nil @@ -532,16 +584,12 @@ final class SoftwareVideoDecoder: VideoDecodingPipeline, @unchecked Sendable { /// Create (or reuse) the decoder-owned CVPixelBufferPool for the given geometry. These /// attributes (IOSurface + Metal compatible, NV12/P010) are the ones the display path has /// always accepted; both the sws path and the hw-deinterlace transfer draw from here. - private func ensurePixelBufferPool(width: Int, height: Int) -> CVPixelBufferPool? { - let cvPixelFormat: OSType = use10Bit - ? kCVPixelFormatType_420YpCbCr10BiPlanarVideoRange - : kCVPixelFormatType_420YpCbCr8BiPlanarVideoRange - - if pixelBufferPool == nil || poolWidth != width || poolHeight != height { + private func ensurePixelBufferPool(width: Int, height: Int, format: OutputFormat) -> CVPixelBufferPool? { + if pixelBufferPool == nil || poolWidth != width || poolHeight != height || poolFormat != format { pixelBufferPool = nil let poolAttrs: NSDictionary = [kCVPixelBufferPoolMinimumBufferCountKey: 6] let pbAttrs: NSDictionary = [ - kCVPixelBufferPixelFormatTypeKey: cvPixelFormat, + kCVPixelBufferPixelFormatTypeKey: format.pixelFormatType, kCVPixelBufferWidthKey: width, kCVPixelBufferHeightKey: height, kCVPixelBufferMetalCompatibilityKey: true, @@ -550,6 +598,7 @@ final class SoftwareVideoDecoder: VideoDecodingPipeline, @unchecked Sendable { CVPixelBufferPoolCreate(kCFAllocatorDefault, poolAttrs, pbAttrs, &pixelBufferPool) poolWidth = width poolHeight = height + poolFormat = format } return pixelBufferPool } @@ -564,10 +613,12 @@ final class SoftwareVideoDecoder: VideoDecodingPipeline, @unchecked Sendable { guard status == noErr, let s = session else { return nil } transferSession = s } + // The transfer converts range and depth itself, so it asks for the stream's own default. guard let session = transferSession, let pool = ensurePixelBufferPool( width: CVPixelBufferGetWidth(src), - height: CVPixelBufferGetHeight(src) + height: CVPixelBufferGetHeight(src), + format: OutputFormat(tenBit: use10Bit, fullRange: false) ) else { return nil } var dst: CVPixelBuffer? guard CVPixelBufferPoolCreatePixelBuffer(kCFAllocatorDefault, pool, &dst) == kCVReturnSuccess, @@ -576,6 +627,21 @@ final class SoftwareVideoDecoder: VideoDecodingPipeline, @unchecked Sendable { return out } + private func reportDecodedFormat(pixelBuffer: CVPixelBuffer) { + guard let onDecodedFormat, var key = decodedFormatKey else { return } + key.pixelBufferType = CVPixelBufferGetPixelFormatType(pixelBuffer) + guard key != reportedFormatKey else { return } + reportedFormatKey = key + onDecodedFormat(DecodedVideoFormat( + frame: VideoStreamFormat( + pixelFormat: AVPixelFormat(rawValue: key.pixelFormat), + declaredBitDepth: 0, + color: key.color, + codecID: codecContext?.pointee.codec_id ?? AV_CODEC_ID_NONE, + profile: codecContext?.pointee.profile ?? AV_PROFILE_UNKNOWN), + pixelBufferFormat: DecodedVideoFormat.fourCC(key.pixelBufferType))) + } + // MARK: - AVFrame → CVPixelBuffer (sws_scale) private func convertFrameToPixelBuffer(_ frame: UnsafeMutablePointer) -> CVPixelBuffer? { @@ -585,7 +651,19 @@ final class SoftwareVideoDecoder: VideoDecodingPipeline, @unchecked Sendable { let srcFmt = AVPixelFormat(rawValue: frame.pointee.format) - let dstFmt = use10Bit ? AV_PIX_FMT_P010LE : AV_PIX_FMT_NV12 + let outputFormat = Self.outputFormat( + pixelFormat: frame.pointee.format, colorRange: frame.pointee.color_range, streamIs10Bit: use10Bit) + let dstFmt = outputFormat.swscaleFormat + if outputFormat != loggedOutputFormat { + loggedOutputFormat = outputFormat + EngineLog.emit( + "[SWDecoder] output \(outputFormat.tenBit ? "10-bit P010" : "8-bit NV12"), " + + "\(outputFormat.fullRange ? "full" : "video") range " + + "(frame \(av_get_pix_fmt_name(srcFmt).map { String(cString: $0) } ?? "?"), " + + "color_range=\(frame.pointee.color_range.rawValue))", + category: .swPlayback + ) + } swsContext = sws_getCachedContext( swsContext, @@ -598,7 +676,7 @@ final class SoftwareVideoDecoder: VideoDecodingPipeline, @unchecked Sendable { guard swsContext != nil else { return nil } var pixelBuffer: CVPixelBuffer? - guard let pool = ensurePixelBufferPool(width: width, height: height) else { return nil } + guard let pool = ensurePixelBufferPool(width: width, height: height, format: outputFormat) else { return nil } let status = CVPixelBufferPoolCreatePixelBuffer(kCFAllocatorDefault, pool, &pixelBuffer) guard status == kCVReturnSuccess, let pb = pixelBuffer else { return nil } @@ -654,20 +732,13 @@ final class SoftwareVideoDecoder: VideoDecodingPipeline, @unchecked Sendable { // MARK: - Color Space Metadata /// Map FFmpeg color metadata to CVPixelBuffer attachments for correct HDR10 rendering (BT.2020 + PQ). + /// Every field is written, a gap included (AE#654): VideoToolbox never hands the display layer an + /// untagged buffer, and neither may this decoder. private func attachColorSpace(from frame: UnsafeMutablePointer, to pb: CVPixelBuffer) { - let primaries = ColorAttachments.primaries(frame.pointee.color_primaries) - let transfer = ColorAttachments.transfer(frame.pointee.color_trc) - let matrix = ColorAttachments.matrix(frame.pointee.colorspace) - - if let primaries { - CVBufferSetAttachment(pb, kCVImageBufferColorPrimariesKey, primaries, .shouldPropagate) - } - if let transfer { - CVBufferSetAttachment(pb, kCVImageBufferTransferFunctionKey, transfer, .shouldPropagate) - } - if let matrix { - CVBufferSetAttachment(pb, kCVImageBufferYCbCrMatrixKey, matrix, .shouldPropagate) - } + let tags = ColorAttachments.presented(ColorDescription(frame: frame)) + CVBufferSetAttachment(pb, kCVImageBufferColorPrimariesKey, tags.primaries, .shouldPropagate) + CVBufferSetAttachment(pb, kCVImageBufferTransferFunctionKey, tags.transfer, .shouldPropagate) + CVBufferSetAttachment(pb, kCVImageBufferYCbCrMatrixKey, tags.matrix, .shouldPropagate) } // MARK: - Pixel Aspect Ratio (anamorphic SD) diff --git a/Sources/AetherEngine/Decoder/SubtitleDecoder.swift b/Sources/AetherEngine/Decoder/SubtitleDecoder.swift index 15efa8413..6d4363759 100644 --- a/Sources/AetherEngine/Decoder/SubtitleDecoder.swift +++ b/Sources/AetherEngine/Decoder/SubtitleDecoder.swift @@ -514,7 +514,13 @@ enum SubtitleDecoder { break } - decodersByStream[pkt.pointee.stream_index]?.decode(packet: pkt) + if let decoder = decodersByStream[pkt.pointee.stream_index] { + // Audit FEA-101 / SUB-105: this loop reads libavformat directly, past Demuxer's bound. + if let stream = fmt.pointee.streams[Int(pkt.pointee.stream_index)] { + SourceTimestampBounds.sanitize(pkt, timeBase: stream.pointee.time_base) + } + decoder.decode(packet: pkt) + } av_packet_unref(pkt) trackedPacketFree(&pktPtr) diff --git a/Sources/AetherEngine/Decoder/VideoDecoderTypes.swift b/Sources/AetherEngine/Decoder/VideoDecoderTypes.swift index 4246cc063..a1a87601d 100644 --- a/Sources/AetherEngine/Decoder/VideoDecoderTypes.swift +++ b/Sources/AetherEngine/Decoder/VideoDecoderTypes.swift @@ -1,6 +1,7 @@ import Foundation import CoreMedia import CoreVideo +import CoreGraphics import AetherLibavformat import AetherLibavcodec @@ -20,6 +21,8 @@ protocol VideoDecodingPipeline: AnyObject, Sendable { /// Only the software decoder produces it; VideoToolbox surfaces no A53 side data (H.264/HEVC /// never route through the SW host, so nothing is missed there). var onA53Captions: (@Sendable ([CCDataParser.CCTriplet], Double) -> Void)? { get set } + /// AE#658: the decoded format and its display buffer, reported on the first frame and on change. + var onDecodedFormat: (@Sendable (DecodedVideoFormat) -> Void)? { get set } var skipUntilPTS: CMTime? { get set } /// AE#492: the decoder's current feed epoch. A caller that decides a batch of packets is @@ -62,15 +65,23 @@ enum ColorAttachments { case AVCOL_PRI_BT709: kCVImageBufferColorPrimaries_ITU_R_709_2 case AVCOL_PRI_BT2020: kCVImageBufferColorPrimaries_ITU_R_2020 case AVCOL_PRI_SMPTE432: kCVImageBufferColorPrimaries_P3_D65 + case AVCOL_PRI_SMPTE431: kCVImageBufferColorPrimaries_DCI_P3 + case AVCOL_PRI_SMPTE170M, AVCOL_PRI_SMPTE240M: kCVImageBufferColorPrimaries_SMPTE_C + case AVCOL_PRI_BT470BG: kCVImageBufferColorPrimaries_EBU_3213 default: nil } } static func transfer(_ v: AVColorTransferCharacteristic) -> CFString? { switch v { - case AVCOL_TRC_BT709: kCVImageBufferTransferFunction_ITU_R_709_2 + // BT.601 and the SDR BT.2020 codepoints name the BT.709 curve, CoreVideo has one constant for all. + case AVCOL_TRC_BT709, AVCOL_TRC_SMPTE170M, AVCOL_TRC_BT2020_10, AVCOL_TRC_BT2020_12: + kCVImageBufferTransferFunction_ITU_R_709_2 case AVCOL_TRC_SMPTE2084: kCVImageBufferTransferFunction_SMPTE_ST_2084_PQ case AVCOL_TRC_ARIB_STD_B67: kCVImageBufferTransferFunction_ITU_R_2100_HLG + case AVCOL_TRC_IEC61966_2_1: kCVImageBufferTransferFunction_sRGB + case AVCOL_TRC_LINEAR: kCVImageBufferTransferFunction_Linear + case AVCOL_TRC_SMPTE240M: kCVImageBufferTransferFunction_SMPTE_240M_1995 default: nil } } @@ -79,10 +90,68 @@ enum ColorAttachments { switch v { case AVCOL_SPC_BT709: kCVImageBufferYCbCrMatrix_ITU_R_709_2 case AVCOL_SPC_BT2020_NCL, AVCOL_SPC_BT2020_CL: kCVImageBufferYCbCrMatrix_ITU_R_2020 + case AVCOL_SPC_SMPTE170M, AVCOL_SPC_BT470BG: kCVImageBufferYCbCrMatrix_ITU_R_601_4 + case AVCOL_SPC_SMPTE240M: kCVImageBufferYCbCrMatrix_SMPTE_240M_1995 default: nil } } + struct Tags: Equatable { + var primaries: CFString + var transfer: CFString + var matrix: CFString + } + + /// AE#654: the tags a software-decoded buffer is presented with, gaps filled the way VideoToolbox + /// fills them on the hardware path. A buffer without tags is not read as BT.709 by the display + /// layer, the report has it going out in the panel's own gamut, so one untagged file looked two + /// ways depending on which decoder the route picked. Measured + /// on VideoToolbox's output (`ColorInfoGuessedBy`): nothing declared reads as BT.709 in all three, + /// at 720x480 and 720x576 as much as at 1080p and for MPEG-2 as much as H.264, and a lone BT.601 + /// matrix gets SMPTE-C primaries and the BT.709 curve. + static func presented(_ d: ColorDescription) -> Tags { + let declaredPrimaries = primaries(d.primaries) + let declaredMatrix = matrix(d.matrix) + let bt709 = kCVImageBufferColorPrimaries_ITU_R_709_2 + let bt601Matrix = kCVImageBufferYCbCrMatrix_ITU_R_601_4 + let bt2020Matrix = kCVImageBufferYCbCrMatrix_ITU_R_2020 + let bt2020Primaries = kCVImageBufferColorPrimaries_ITU_R_2020 + let smpteC = kCVImageBufferColorPrimaries_SMPTE_C + let ebu = kCVImageBufferColorPrimaries_EBU_3213 + + let resolvedPrimaries: CFString = declaredPrimaries ?? { + switch declaredMatrix { + case bt601Matrix?: smpteC + case bt2020Matrix?: bt2020Primaries + default: bt709 + } + }() + let resolvedMatrix: CFString = declaredMatrix ?? { + switch declaredPrimaries { + case smpteC?, ebu?: bt601Matrix + case bt2020Primaries?: bt2020Matrix + default: kCVImageBufferYCbCrMatrix_ITU_R_709_2 + } + }() + return Tags( + primaries: resolvedPrimaries, + transfer: transfer(d.transfer) ?? kCVImageBufferTransferFunction_ITU_R_709_2, + matrix: resolvedMatrix) + } + + /// The colour space CoreVideo manages a buffer with these tags in, i.e. the one playback shows the + /// picture in. An RGB still converted from the same picture has to carry it: tagged sRGB instead, + /// a BT.709 still drew 8 levels darker than VideoToolbox's own conversion of the frame, and an SDR + /// BT.2020 one up to 57 levels off in red. + static func colorSpace(for tags: Tags) -> CGColorSpace? { + let attachments: NSDictionary = [ + kCVImageBufferColorPrimariesKey: tags.primaries, + kCVImageBufferTransferFunctionKey: tags.transfer, + kCVImageBufferYCbCrMatrixKey: tags.matrix, + ] + return CVImageBufferCreateColorSpaceFromAttachments(attachments)?.takeRetainedValue() + } + /// PQ (ST 2084) or HLG transfer means the stream is HDR. static func isHDRTransfer(_ trc: AVColorTransferCharacteristic) -> Bool { trc == AVCOL_TRC_SMPTE2084 || trc == AVCOL_TRC_ARIB_STD_B67 diff --git a/Sources/AetherEngine/Decoder/VideoRoutingPolicy.swift b/Sources/AetherEngine/Decoder/VideoRoutingPolicy.swift index c88cf06b9..e8b9b9b0a 100644 --- a/Sources/AetherEngine/Decoder/VideoRoutingPolicy.swift +++ b/Sources/AetherEngine/Decoder/VideoRoutingPolicy.swift @@ -14,6 +14,7 @@ enum VideoRoutingPolicy { /// True when a video codec must use the software decode path (SoftwarePlaybackHost) instead of /// native AVPlayer. `av1Available` is `VTCapabilityProbe.av1Available` (HW AV1 decode support). + /// Evaluated only for AV1; registering its decoder must not delay unrelated formats. /// #150: `spsIndicatesInterlaced` (SPS frame_mbs_only_flag == 0) breaks the tie when the demuxer's /// field_order probe stays UNKNOWN; a concrete PROGRESSIVE probe analyzed actual frames and wins. /// A false positive costs an unnecessary SW decode (deint=interlaced passes progressive frames @@ -30,7 +31,7 @@ enum VideoRoutingPolicy { static func requiresSoftwarePath( codecID: AVCodecID, fieldOrder: AVFieldOrder, - av1Available: Bool, + av1Available: @autoclosure () -> Bool, spsIndicatesInterlaced: Bool = false, stereo3DType: AVStereo3DType? = nil ) -> Bool { @@ -38,7 +39,7 @@ enum VideoRoutingPolicy { case AV_CODEC_ID_NONE, AV_CODEC_ID_HEVC: return false case AV_CODEC_ID_AV1: - return !av1Available + return !av1Available() case AV_CODEC_ID_H264: if routesSoftwareForMultiviewCarriage(codecID: codecID, stereo3DType: stereo3DType) { return true @@ -101,8 +102,8 @@ enum VideoRoutingPolicy { /// AVPlayer path reaches readyToPlay and then renders nothing (H.264 High 4:2:2/4:4:4/High-10, HEVC Rext /// on Intel Macs / older Apple TV). Pure so it is unit-testable; the impure VT probe /// (`VTCapabilityProbe.canHardwareDecode`) is injected as the `canHardwareDecode` closure and only runs - /// when the gate actually consults it. Only H.264 / HEVC consult this gate; AV1 / VP9 / etc. already - /// have their own routing above and must not be reclassified here. + /// when the gate actually consults it. H.264 / HEVC and HW-routed AV1 consult this gate (AV1 by + /// profile, see `av1FitsHardwareDecoder`); VP9 / etc. have their own routing above. /// /// #176: HEVC DV Profile 5 bypasses the gate entirely. The probe builds a plain-HEVC format description /// from the raw hvcC, which is not what the native path plays (dvh1 + dvcC, decoded by Apple's DV @@ -118,13 +119,35 @@ enum VideoRoutingPolicy { switch codecID { case AV_CODEC_ID_HEVC where dvProfile == 5: return false - case AV_CODEC_ID_H264, AV_CODEC_ID_HEVC: + case AV_CODEC_ID_H264, AV_CODEC_ID_HEVC, AV_CODEC_ID_AV1: return !canHardwareDecode() default: return false } } + /// Whether a load consults `forcesSoftwareForUndecodableFormat` at all. H.264 / HEVC keep it VOD-only: + /// broadcast live is hardware-decodable and forced-native live keeps its verified path. AV1 is + /// consulted on live too (audit HLS-5 follow-up): a live AV1 High / Professional stream on a + /// hardware-AV1 device has no native picture, and live AV1 already goes to the software host + /// wherever hardware AV1 is missing entirely. + static func consultsUndecodableFormatGate(codecID: AVCodecID, isLive: Bool) -> Bool { + !isLive || codecID == AV_CODEC_ID_AV1 + } + + /// Audit HLS-5: Apple's hardware AV1 decoders (A17 Pro, M3 and later) decode Main profile only, + /// 8/10-bit 4:2:0. `av1Available` is a codec-level answer, so High (4:4:4) and Professional + /// (4:2:2, 12-bit) reached AVPlayer and rendered nothing while dav1d would have played them. + /// The av1C states `seq_profile` in the top 3 bits of its second byte; the codecpar profile is the + /// fallback. Unknown on both counts keeps the native route, like the H.264 / HEVC probe gap. + static func av1FitsHardwareDecoder(av1C: [UInt8]?, codecparProfile: Int32) -> Bool { + if let av1C, av1C.count >= 4, av1C[0] & 0x80 != 0 { + return av1C[1] >> 5 == 0 + } + guard codecparProfile >= 0 else { return true } + return codecparProfile == 0 + } + /// AE#461: the decode path a session ends up on, given what the routing concluded and what the /// host asked for. Pure so the override is unit-testable next to the decisions it overrules. /// @@ -146,6 +169,32 @@ enum VideoRoutingPolicy { } } + /// A URL source that turned out forward-only (the origin ignores `Range` and names no length, so + /// the reader could only stream it front to back) is served the way a declared sequential origin + /// is, when its container states a duration: native path, one linear pass, seeks unavailable. + /// The software path it used to take cannot seek on such a source either, so the promotion costs + /// nothing it had, and it buys what only the native path has: hardware decode, AVPlayer's + /// buffering, and a picture on an AirPlay receiver (the software host renders on the device, + /// only its audio follows the route). + /// + /// Only when the routing had already chosen native and the host did not ask for software, so + /// the promotion never moves a session between hosts on its own. A custom reader is the host's + /// to describe. Without a duration the segment plan has nothing to stride over, so that source + /// keeps the software path. + static func promotesForwardOnlySourceToSequential( + isSourceSeekable: Bool, + isLive: Bool, + declaredSequential: Bool, + isCustomSource: Bool, + routedSoftware: Bool, + preferred: DecodePath, + containerDurationSeconds: Double + ) -> Bool { + !isSourceSeekable && !isLive && !declaredSequential && !isCustomSource + && !routedSoftware && preferred == .automatic + && containerDurationSeconds.isFinite && containerDurationSeconds > 0 + } + /// #176 follow-up: DV variants whose only signal is IPT-PQ-c2 (no compatible base layer) cannot be /// color-correctly decoded by the software path: libavcodec / dav1d hand the IPT signal on as YCbCr, /// which renders with a green/purple cast. That is HEVC P5 and AV1 P10.0 (compat 0). P7 / P8.x / diff --git a/Sources/AetherEngine/Demuxer/AVIOProvider.swift b/Sources/AetherEngine/Demuxer/AVIOProvider.swift index f39cd8aa6..207c733bb 100644 --- a/Sources/AetherEngine/Demuxer/AVIOProvider.swift +++ b/Sources/AetherEngine/Demuxer/AVIOProvider.swift @@ -38,6 +38,18 @@ protocol AVIOProvider: AnyObject { /// seek fallback when a timestamp seek times out on an index-less container. var resolvedByteSize: Int64? { get } + /// Ceiling on the bytes this provider hands libavformat until `endReadByteBudget`, for a pass whose + /// packet caps cannot see what the demuxer reads and drops inside one `av_read_frame` (the blocks of + /// AVDISCARD_ALL streams). A read past it fails like a deadline abort. Demux-thread-only, same + /// contract as the read deadline. + func beginReadByteBudget(_ bytes: Int64) + + /// Disarm the budget armed by `beginReadByteBudget`. + func endReadByteBudget() + + /// True when a read was refused because the byte budget was spent. + var readByteBudgetExhausted: Bool { get } + /// AE#460 follow-up: whether the byte source behind this provider outlives the provider, so a /// rebuild reopens onto a source that is already positioned. True only for the custom-reader /// bridge, which does not own its reader; a provider that opens its own transport per session @@ -74,3 +86,31 @@ extension AVIOProvider { var currentSourceOffset: Int64? { nil } var sourceSurvivesReopen: Bool { false } } + +/// The shared bookkeeping behind `AVIOProvider.beginReadByteBudget`. Not thread-safe: it lives on the +/// demux thread with the read callback that consults it. +struct ReadByteBudget { + private var remaining: Int64? + private(set) var exhausted = false + + mutating func begin(_ bytes: Int64) { + remaining = max(0, bytes) + exhausted = false + } + + mutating func end() { remaining = nil } + + /// The size a read may ask for, or nil (latching `exhausted`) once the budget is spent. + mutating func allowance(_ size: Int32) -> Int32? { + guard let remaining else { return size } + guard remaining > 0 else { + exhausted = true + return nil + } + return Int32(min(Int64(size), remaining)) + } + + mutating func consumed(_ count: Int32) { + if count > 0, let remaining { self.remaining = remaining - Int64(count) } + } +} diff --git a/Sources/AetherEngine/Demuxer/AVIOReader.swift b/Sources/AetherEngine/Demuxer/AVIOReader.swift index 522c46eaa..45f5e80a9 100644 --- a/Sources/AetherEngine/Demuxer/AVIOReader.swift +++ b/Sources/AetherEngine/Demuxer/AVIOReader.swift @@ -482,8 +482,15 @@ final class AVIOReader: AVIOProvider, @unchecked Sendable { private static let avioBufferSize: Int32 = 256 * 1024 // 256 KB private static let streamTrimThreshold = 1024 * 1024 // 1 MB, keep for small backward seeks // Backpressure: suspend the streaming task above highWater, resume below lowWater. - private static let streamHighWater = 64 * 1024 * 1024 - private static let streamLowWater = 32 * 1024 * 1024 + private static let streamHighWaterDefault = 64 * 1024 * 1024 + private let streamHighWater: Int + private var streamLowWater: Int { streamHighWater / 2 } + /// Audit DMX-7: the suspend above is advisory (#220 measured 911 MB arriving after one), and + /// this path cannot re-request at an offset the way the persistent reader does, so a transport + /// that keeps delivering past twice the high water is ended and the read fails with EIO once + /// the buffered bytes are drained. Audit DMX-107: that failure is typed + /// (`AVIOReaderError.originIgnoresFlowControl`, `lastReadFailure`), so it reads as what it is. + private var streamHardCap: Int { 2 * streamHighWater } private let bufferLock = NSLock() private var currentBuffer = Data() @@ -497,8 +504,13 @@ final class AVIOReader: AVIOProvider, @unchecked Sendable { // MARK: - Streaming Mode (sequential GET) - private var streamBuffer = Data() + /// Held as the delivered chunks, like the persistent window (AE#619): a read trims its consumed + /// head by advancing an offset, where `Data.subdata` copied the whole 32-64 MB still ahead of the + /// cut on every read (audit PERF-102). Guarded by `streamLock`. + private var streamBuffer = ChunkedByteWindow() private var streamBytesRead: Int64 = 0 + /// Most bytes `streamBuffer` has held at once, for the bound tests. Guarded by `streamLock`. + private var streamPeakBytes = 0 private var streamEnded = false private let streamLock = NSLock() private let streamDataReady = DispatchSemaphore(value: 0) @@ -508,6 +520,13 @@ final class AVIOReader: AVIOProvider, @unchecked Sendable { /// consumer treats EOF as "played to the end" and deliberately never retries it. Guarded by /// `streamLock`. private var streamExpectedBytes: Int64 = -1 + /// Audit DMX-6: the streaming GET ended in a transport error, or was ended at + /// `streamHardCap`. Either way the bytes that did not arrive are lost, not absent, so the read + /// that runs out reports EIO instead of end-of-media. Guarded by `streamLock`. + private var streamFailed = false + /// Audit DMX-107: why the streaming GET was ended by this reader when it was not the transport's + /// doing. Guarded by `streamLock`. + private var streamFailureCause: AVIOReaderError? /// The status the streaming GET was answered with when it was anything but 200/206, 0 while /// none. A status is not media: the delegate hangs up at the header, and `open()` fails typed /// on it rather than handing FFmpeg an empty stream to misreport as invalid data. Written on @@ -567,8 +586,9 @@ final class AVIOReader: AVIOProvider, @unchecked Sendable { private static let liveWinHighWaterDefault = 64 * 1024 * 1024 private static let winLowWater = 8 * 1024 * 1024 // #220: how much the persistent reader asks for at a time. Bounds a single request's - // exposure by construction (an origin cannot serve more than it was asked for, whatever - // it thinks of flow control) and keeps the request cadence modest on a healthy link: + // exposure: `appendPersistentData` drops anything past the requested end and ends the + // transfer (audit DMX-1), so an origin that ignores Range or answers wider than asked + // cannot grow the window past it. It also keeps the request cadence modest on a healthy link: // one request per range boundary while the consumer keeps up, one per // highWater-to-lowWater drain cycle when the origin outruns it. static let persistentRangeBytes: Int64 = 32 * 1024 * 1024 @@ -677,6 +697,9 @@ final class AVIOReader: AVIOProvider, @unchecked Sendable { // playback fast path never enters this code, so it carries zero overhead. private static let detourBlockSize = 4 * 1024 * 1024 private static let detourMaxBlocks = 8 // ~32 MB LRU ceiling + /// Audit PERF-108: a detour block nobody has read or filled for this long is a scrub's leftover, + /// not a parse's working set (a parser revisits its region within seconds). + private static let detourBlockIdleSecondsDefault: TimeInterval = 30 // Once detour reads turn sequential past this much, re-anchor the streaming connection // there so sustained playback returns to the cheap window path (e.g. after a backward scrub). private static let detourReanchorBytes: Int64 = 8 * 1024 * 1024 @@ -722,8 +745,9 @@ final class AVIOReader: AVIOProvider, @unchecked Sendable { /// edge-triggered condition variable for read waits and backpressure. private let winCond = NSCondition() /// Sliding window of bytes from the live connection, starting at `winStart`. - /// `position - winStart` is the read offset within `window`. - private var window = Data() + /// `position - winStart` is the read offset within `window`. Held as the delivered chunks + /// (AE#619), so trimming the consumed head copies nothing. + private var window = ChunkedByteWindow() private var winStart: Int64 = 0 // Connection state. private var connEnded = false @@ -781,6 +805,41 @@ final class AVIOReader: AVIOProvider, @unchecked Sendable { return activeTransfer != nil } + /// Audit DMX-107: the typed cause of the read error the streaming path reports, nil while the + /// source is healthy and for a plain transport loss (which has no cause of ours to name). + var lastReadFailure: AVIOReaderError? { + streamLock.lock() + defer { streamLock.unlock() } + return streamFailureCause + } + + /// Detour blocks currently resident. + var detourResidentBlocksForTesting: Int { detourCache.residentCount } + + /// The most the forward-only streaming buffer has held at once. + var streamPeakBufferBytesForTesting: Int { + streamLock.lock() + defer { streamLock.unlock() } + return streamPeakBytes + } + + #if DEBUG + /// Storage address of every chunk the streaming buffer still holds, first to last. A trim that + /// copied the buffer would move all of them. + var streamChunkAddressesForTesting: [UnsafeRawPointer?] { + streamLock.lock() + defer { streamLock.unlock() } + return streamBuffer.chunkStorageAddresses() + } + #endif + + /// Bytes the persistent window holds, behind and ahead of the cursor. + var windowBytesForTesting: Int { + winCond.lock() + defer { winCond.unlock() } + return window.count + } + /// #220/#310: set when WE ended the connection at `winHighWater` (ending is the only /// non-advisory flow control, and a task left suspended instead is a dormant flow — see /// the backpressure doc block above). The read loop re-requests at the frontier — at low @@ -817,6 +876,17 @@ final class AVIOReader: AVIOProvider, @unchecked Sendable { /// (live sources, and any source whose total size is not resolved yet). winCond-guarded. private var connRangeEnd: Int64? + /// Audit DMX-101: true only while `open()` waits on the cold data connection, so that response + /// alone is judged for range support. A 200 later in a session (a CDN cache miss on a refill) + /// stays what it always was: rejected and retried, never a verdict on the origin. winCond-guarded. + private var rangeVerdictPending = false + /// Audit DMX-101: Content-Length of the cold connection's range-ignoring 200, -1 when it stated + /// none, nil while the origin has not been caught ignoring. winCond-guarded. + private var rangeIgnoredAtOpenLength: Int64? + /// Audit DMX-101: the origin cannot address bytes, so this source plays forward-only on the + /// streaming path. Written once by `open()` on the demux thread, read by `seek` on the same one. + private(set) var originIgnoresRange = false + /// Sodalite#117: contiguous range starts are summarised instead of logged one by one. /// winCond-guarded. private var connStartLogGate = ConnStartLogGate() @@ -871,6 +941,12 @@ final class AVIOReader: AVIOProvider, @unchecked Sendable { /// math lives on the cache and is unit-tested without any network. private let detourCache = DetourBlockCache(blockSize: AVIOReader.detourBlockSize, maxBlocks: AVIOReader.detourMaxBlocks) + /// When the detour cache was last swept for idle blocks (audit PERF-108). Demux-thread-only. + private var lastDetourIdleSweepNs: UInt64 = 0 + /// How long a detour block may sit unread before the sweep releases it. An init parameter for + /// the same reason `connStallTimeout` is one: a process-wide hook would leak into whatever suite + /// runs concurrently. + private let detourBlockIdleSeconds: TimeInterval // Re-anchor run tracking (demux-thread-only): the file offset the next sequential detour read // would continue from, and how many contiguous bytes the current detour run has served. private var detourRunNextExpected: Int64 = -1 @@ -1022,7 +1098,7 @@ final class AVIOReader: AVIOProvider, @unchecked Sendable { private let probeDrainLock = NSLock() private var drainingProbeRequest = false - init(url: URL, extraHeaders: [String: String] = [:], label: String = "source", chunkSize: Int = 4 * 1024 * 1024, prefetchEnabled: Bool = true, isLive: Bool = false, chunkRequestTimeout: TimeInterval = 35, chunkMaxRetries: Int = 3, boundedInitialFetch: Int64? = nil, sequentialOnly: Bool = false, connStallTimeout: TimeInterval = AVIOReader.connStallTimeoutDefault, windowHighWater: Int? = nil, heldConnection: Bool = false, probeControl: ProbeControl? = nil, probeRequestSession: URLSession? = nil) { + init(url: URL, extraHeaders: [String: String] = [:], label: String = "source", chunkSize: Int = 4 * 1024 * 1024, prefetchEnabled: Bool = true, isLive: Bool = false, chunkRequestTimeout: TimeInterval = 35, chunkMaxRetries: Int = 3, boundedInitialFetch: Int64? = nil, sequentialOnly: Bool = false, connStallTimeout: TimeInterval = AVIOReader.connStallTimeoutDefault, windowHighWater: Int? = nil, streamHighWater: Int? = nil, detourIdleSeconds: TimeInterval? = nil, heldConnection: Bool = false, probeControl: ProbeControl? = nil, probeRequestSession: URLSession? = nil) { self.probeControl = probeControl self.probeRequestSession = probeControl == nil ? nil : probeRequestSession self.url = url @@ -1041,6 +1117,8 @@ final class AVIOReader: AVIOProvider, @unchecked Sendable { self.connStallTimeout = max(0.05, connStallTimeout) self.winHighWater = max(1, windowHighWater ?? (isLive ? Self.liveWinHighWaterDefault : Self.winHighWaterDefault)) + self.streamHighWater = max(1, streamHighWater ?? Self.streamHighWaterDefault) + self.detourBlockIdleSeconds = detourIdleSeconds ?? Self.detourBlockIdleSecondsDefault self.heldConnectionEnabled = heldConnection } @@ -1151,8 +1229,30 @@ final class AVIOReader: AVIOProvider, @unchecked Sendable { gotData = true openPrefix = Array(warm.head.data.prefix(16)) } else { + winCond.lock() + rangeVerdictPending = !isLive + winCond.unlock() startPersistentConnection(at: 0, boundedTo: boundedInitialFetch) - gotData = awaitFirstPersistentData() + var opened = awaitFirstPersistentData() + // AE#693: one refusal is the origin's answer to that request, not yet to the + // resource. The #378 path below takes a 401/403/404/410 here as final and settles + // the session forward-only, which costs it every seek and the resume position. An + // IPTV panel answered a ranged open 403 and the unranged GET right after it 200, and + // ranged requests worked again later. Asking the same range once more decides it: + // served, the source stays seekable; refused again, the #378 path runs as before. + if !opened, !isLive, let refusal = refusalOfOpenConnection() { + EngineLog.emit( + "[AVIOReader] \(label) ranged open refused status=\(refusal) at offset 0; " + + "asking once more before settling the source forward-only (AE#693)", + category: .demux) + try probeControl?.check() + winCond.lock() + rangeVerdictPending = true + winCond.unlock() + startPersistentConnection(at: 0, boundedTo: boundedInitialFetch) + opened = awaitFirstPersistentData() + } + gotData = opened openPrefix = firstWindowPrefix() } // AE#140: an HLS playlist URL misrouted onto the raw-byte live path. A live origin serves the @@ -1185,13 +1285,24 @@ final class AVIOReader: AVIOProvider, @unchecked Sendable { // a size; if not, abandon it (generation bump ignores a size landing in the // race window). fileSize is read under the lock because the delegate thread now // writes it (issue #70 review #4/#5). - let (haveSize, abandoned, pumpStatus) = resolveOptimisticOpen() + let (haveSize, abandoned, pumpStatus, caughtIgnoringRange) = resolveOptimisticOpen() abandoned?.cancelTransfer() abandoned?.releaseOriginTicket() - if !haveSize { + var ignoredRangeLength = caughtIgnoringRange + if let length = caughtIgnoringRange { + tookFallback = true + try probeControl?.check() + if confirmRangeIsIgnored(contentLength: length) == .honoured { ignoredRangeLength = nil } + try probeControl?.check() + } + if let ignoredRangeLength { + openForwardOnlyBecauseRangeIsIgnored(contentLength: ignoredRangeLength) + try failIfStreamingRefused(fallbackStatus: 0) + } else if !haveSize { tookFallback = true - // A 401/403/404/410 at byte 0 is the origin's answer to the RESOURCE, not to - // the range form: a HEAD or a `bytes=0-1` from the same client is answered + // A 401/403/404/410 at byte 0, given twice (AE#693: once can be the moment's), + // is the origin's answer to the RESOURCE, not to the range form: a HEAD or a + // `bytes=0-1` from the same client is answered // alike, and a size learned from either would only re-issue the refused range // on the persistent path (which then dies after one retry with the status // lost). Skip the ladder. The one request still worth making is the unranged @@ -1221,8 +1332,31 @@ final class AVIOReader: AVIOProvider, @unchecked Sendable { _ = streamDataReady.wait(timeout: .now() + .seconds(15)) try failIfStreamingRefused(fallbackStatus: pumpRefusal) } else { + // Audit DMX-101: a cold open that was refused or slow never judged the + // origin's range support, and the probe above takes a 200's Content-Length + // for a size. This connection is then the first to see the origin's answer, + // so it gets the same head classification and confirmation. + winCond.lock() + rangeVerdictPending = true + winCond.unlock() startPersistentConnection(at: 0) - if !awaitFirstPersistentData() { + let gotData = awaitFirstPersistentData() + if let caught = takeRangeVerdictAfterProbe() { + caught.abandoned?.cancelTransfer() + caught.abandoned?.releaseOriginTicket() + try probeControl?.check() + let confirmation = confirmRangeIsIgnored(contentLength: caught.contentLength) + try probeControl?.check() + if confirmation == .ignored { + openForwardOnlyBecauseRangeIsIgnored(contentLength: caught.contentLength) + try failIfStreamingRefused(fallbackStatus: 0) + } else { + startPersistentConnection(at: 0) + if !awaitFirstPersistentData() { + EngineLog.emit("[AVIOReader] Persistent open (post-probe): no data within 15s, proceeding to read-loop reconnect", category: .demux) + } + } + } else if !gotData { EngineLog.emit("[AVIOReader] Persistent open (post-probe): no data within 15s, proceeding to read-loop reconnect", category: .demux) } } @@ -1296,6 +1430,14 @@ final class AVIOReader: AVIOProvider, @unchecked Sendable { return gotData } + /// The refusal status the current persistent connection ended on, nil when it did not end on one + /// of the `isResolvedExpiryStatus` refusals. Demux thread, open-time only. + private func refusalOfOpenConnection() -> Int? { + winCond.lock() + defer { winCond.unlock() } + return (connEnded && Self.isResolvedExpiryStatus(connStatus)) ? connStatus : nil + } + /// The streaming GET was answered with a status instead of a body, or the ranged open was /// already refused with one and the unranged GET then delivered nothing either. Either way the /// demuxer would be handed an empty stream (or an error page) and report it as invalid data; @@ -1356,7 +1498,7 @@ final class AVIOReader: AVIOProvider, @unchecked Sendable { winCond.lock() defer { winCond.unlock() } let n = Swift.min(max, window.count) - return n > 0 ? Array(window.prefix(n)) : [] + return window.prefix(n) } /// True when a body's leading bytes are the HLS playlist tag `#EXTM3U`, tolerating a UTF-8 BOM and @@ -1383,18 +1525,105 @@ final class AVIOReader: AVIOProvider, @unchecked Sendable { /// the lock. Demux thread, open-time only; leaves the AVIO context intact (unlike close()). /// `pumpStatus` is the HTTP status the abandoned connection was answered with (0 when no /// response arrived), so the caller can tell a refused resource from a length-less one. - private func resolveOptimisticOpen() -> (haveSize: Bool, abandoned: (any PersistentTransfer)?, pumpStatus: Int) { + private func resolveOptimisticOpen() -> (haveSize: Bool, abandoned: (any PersistentTransfer)?, + pumpStatus: Int, caughtIgnoringRange: Int64?) { winCond.lock() defer { winCond.unlock() } - if fileSize > 0 { return (true, nil, connStatus) } + rangeVerdictPending = false + let ignoredLength = rangeIgnoredAtOpenLength + rangeIgnoredAtOpenLength = nil + if fileSize > 0 { return (true, nil, connStatus, nil) } let status = connStatus connGeneration &+= 1 let transfer = activeTransfer activeTransfer = nil - window = Data() + window.removeAll() + connEnded = true + winCond.broadcast() + return (false, transfer, status, ignoredLength) + } + + /// Audit DMX-101: the verdict of the post-probe connection, which `resolveOptimisticOpen` does + /// for the cold one. The size is already resolved by the probe, so nothing here keys off it: + /// a caught origin gets its connection abandoned under the lock, with the window emptied, and + /// a connection that was not caught is left running. The flag is cleared either way, so a + /// refill or reconnect later in the session is never read as a verdict. Demux thread, + /// open-time only. + private func takeRangeVerdictAfterProbe() -> (contentLength: Int64, abandoned: (any PersistentTransfer)?)? { + winCond.lock() + defer { winCond.unlock() } + rangeVerdictPending = false + guard let contentLength = rangeIgnoredAtOpenLength else { return nil } + rangeIgnoredAtOpenLength = nil + connGeneration &+= 1 + let transfer = activeTransfer + activeTransfer = nil + window.removeAll() connEnded = true winCond.broadcast() - return (false, transfer, status) + return (contentLength, transfer) + } + + private enum RangeConfirmation { case honoured, ignored } + + /// Audit DMX-101: the cold connection's 200 is one answer. Some CDNs send a 200 on a cache miss + /// and a 206 for every request after it, so the verdict waits for a second, one byte request. + /// Anything but a 206 that starts where asked (a 200, a refusal, a timeout, a misplaced range) + /// leaves the origin judged as it was caught: a source that cannot address bytes. The body a + /// 200 answer carries is cut after the one byte the request asked for (`ChunkFetchDelegate`). + /// Demux thread, open-time only. + private func confirmRangeIsIgnored(contentLength: Int64) -> RangeConfirmation { + var request = URLRequest(url: requestURL()) + request.setValue("bytes=1-1", forHTTPHeaderField: "Range") + let budget = min(10, chunkRequestTimeout) + request.timeoutInterval = budget + applyExtraHeaders(&request) + do { + let (_, response) = try syncRequest(request, budget: budget) + guard let http = response as? HTTPURLResponse else { return .ignored } + switch Self.rangeAnswer(http, requestedStart: 1, requestedEnd: 1) { + case .honoured: + EngineLog.emit( + "[AVIOReader] \(label) the origin answered the ranged open with a 200 " + + "(Content-Length \(contentLength)) but honours a one byte range; " + + "keeping the seekable path (audit DMX-101)", category: .demux) + return .honoured + case .overWide: + // A 206 is the origin addressing bytes, however wide it answered. + return .honoured + default: + return .ignored + } + } catch { + return .ignored + } + } + + /// Audit DMX-101: an origin that cannot address bytes is played forward-only, exactly what + /// `sequentialOnly` does by declaration: one unranged GET, `fileSize` unresolved so the pb is + /// non-seekable (#126 below), and the window bounded by the streaming path's high water. The + /// Content-Length it stated stays advisory and is what `AVSEEK_SIZE` answers, because FFmpeg + /// estimates an MPEG-TS duration from the size and a length-less source reports none. + /// Demux thread, open-time only. + private func openForwardOnlyBecauseRangeIsIgnored(contentLength: Int64) { + EngineLog.emit( + "[AVIOReader] \(label) origin does not honour Range (Content-Length \(contentLength)); " + + "playing forward-only from one unranged GET (audit DMX-101)", category: .demux) + originIgnoresRange = true + winCond.lock() + fileSize = -1 + winCond.unlock() + streamLock.lock() + streamExpectedBytes = contentLength > 0 ? contentLength : -1 + streamLock.unlock() + winCond.lock() + let tailTask = tailPrefetchTask + tailPrefetchTask = nil + tailPrefetchInFlight = false + winCond.unlock() + tailTask?.cancel() + startStreamingDownload() + _ = streamDataReady.wait(timeout: .now() + .seconds(15)) } // Close flags written on the teardown thread (markClosed / fullyClose) and read on the demux @@ -1450,6 +1679,11 @@ final class AVIOReader: AVIOProvider, @unchecked Sendable { readDeadline = .distantFuture } + private var readByteBudget = ReadByteBudget() + var readByteBudgetExhausted: Bool { readByteBudget.exhausted } + func beginReadByteBudget(_ bytes: Int64) { readByteBudget.begin(bytes) } + func endReadByteBudget() { readByteBudget.end() } + /// Deadline expired; latches `readDeadlineFired` at the check sites. private var readDeadlinePassedOrAborted: Bool { isPastReadDeadline } @@ -1526,7 +1760,7 @@ final class AVIOReader: AVIOProvider, @unchecked Sendable { streamLock.lock() streamEnded = true - streamBuffer = Data() + streamBuffer.removeAll() let sTask = streamingTask let sSession = streamingSession let wasSuspended = streamingTaskSuspended @@ -1545,7 +1779,7 @@ final class AVIOReader: AVIOProvider, @unchecked Sendable { connEnded = true let transfer = activeTransfer activeTransfer = nil - window = Data() + window.removeAll() // #281: both spans hold real bytes (up to headSpanMaxBytes + tailPrefetchBytes), so they // are released with the window rather than living until the reader is deallocated. headSpan = Data() @@ -1570,13 +1804,15 @@ final class AVIOReader: AVIOProvider, @unchecked Sendable { func read(into buf: UnsafeMutablePointer, size: Int32) -> Int32 { guard !isClosed else { return -1 } if readDeadlinePassedOrAborted { readDeadlineFired = true; return -1 } + guard let allowed = readByteBudget.allowance(size) else { return -1 } // Check usePersistentReader before isStreaming: live feeds without // Content-Length must use the reconnect-capable persistent path. let n: Int32 - if usePersistentReader { n = readPersistent(into: buf, size: size) } - else if isStreaming { n = readStreaming(into: buf, size: size) } - else { n = readSeekable(into: buf, size: size) } + if usePersistentReader { n = readPersistent(into: buf, size: allowed) } + else if isStreaming { n = readStreaming(into: buf, size: allowed) } + else { n = readSeekable(into: buf, size: allowed) } if n > 0 { applyThrottle(deliveredBytes: Int(n)) } + readByteBudget.consumed(n) return n } @@ -1656,14 +1892,21 @@ final class AVIOReader: AVIOProvider, @unchecked Sendable { if fetchSize <= 0 { break } - guard let data = fetchChunk(from: position, size: fetchSize), !data.isEmpty else { + let fetched = fetchChunk(from: position, size: fetchSize) + guard let data = fetched, !data.isEmpty else { // An aborted fetch (supersede/close/deadline) must report a read // error, not EOF (which would truncate the stream cleanly). issue #27. if isClosed || readDeadlinePassedOrAborted { if readDeadlinePassedOrAborted { readDeadlineFired = true } return totalRead > 0 ? Int32(totalRead) : -1 } - // nil = transport failure; empty = 2xx with no body (would loop forever otherwise). + // Audit DMX-10: nil is a transport failure short of a known size, which is the + // same loss #25 stopped reporting as EOF on the persistent path; a probe that + // read it as end-of-file reported a truncated duration instead of failing. + if fetched == nil, fileSize > 0 { + return totalRead > 0 ? Int32(totalRead) : FFmpegErr.eio + } + // Empty = 2xx with no body (would loop forever otherwise). break } @@ -1695,26 +1938,21 @@ final class AVIOReader: AVIOProvider, @unchecked Sendable { let toCopy = min(available, requestSize - totalRead) streamLock.lock() - streamBuffer.withUnsafeBytes { raw in - let src = raw.baseAddress!.advanced(by: posInBuffer) - .assumingMemoryBound(to: UInt8.self) - buf.advanced(by: totalRead).update(from: src, count: toCopy) - } + streamBuffer.copyBytes(to: buf.advanced(by: totalRead), from: posInBuffer, count: toCopy) streamLock.unlock() position += Int64(toCopy) totalRead += toCopy - // subdata (not removeFirst): removeFirst leaks backing storage (see trimWindowLocked). streamLock.lock() let consumed = Int(position - streamBytesRead) if consumed > Self.streamTrimThreshold { let trimAmount = consumed - Self.streamTrimThreshold - streamBuffer = streamBuffer.subdata(in: trimAmount.. 0 && received < expected) { - EngineLog.emit( - "[AVIOReader] sequential stream \(ended ? "ended short" : "stalled out") at " - + "\(received)\(expected > 0 ? "/\(expected)" : "") bytes; reporting EIO", - category: .demux - ) - return FFmpegErr.eio - } + streamLock.lock() + let ended = streamEnded + let failed = streamFailed + let cause = streamFailureCause + let received = streamBytesRead + Int64(streamBuffer.count) + let expected = streamExpectedBytes + streamLock.unlock() + // This path cannot be resumed at an offset, so a stalled-out wait, a transport error or a + // body that ended short of its advisory length is a LOST source: report EIO so the pump + // exits on a read error the session can surface. EOF here would read as end-of-media, + // which the consumer deliberately never retries. Audit DMX-6: this held for the + // sequential origin only, and a length-less source ended its film early on a Wi-Fi drop. + if !ended || failed || (expected > 0 && received < expected) { + EngineLog.emit( + "[AVIOReader] \(label) stream \(!ended ? "stalled out" : failed ? "failed" : "ended short") at " + + "\(received)\(expected > 0 ? "/\(expected)" : "") bytes" + + (cause.map { " (\($0))" } ?? "") + "; reporting EIO", + category: .demux + ) + return FFmpegErr.eio } return FFmpegErr.eof } @@ -1780,6 +2020,7 @@ final class AVIOReader: AVIOProvider, @unchecked Sendable { private func readPersistent(into buf: UnsafeMutablePointer, size: Int32) -> Int32 { let requestSize = Int(size) var totalRead = 0 + sweepIdleDetourBlocks() // #93 restart latency: accumulate where THIS read spends its time; one summary line fires // on completion when the whole call exceeded the threshold (see SlowReadDiagnostics). @@ -1978,6 +2219,7 @@ final class AVIOReader: AVIOProvider, @unchecked Sendable { // the cheap window path instead of fetching 4 MB blocks forever. if curPosition == detourRunNextExpected && detourRunBytes >= Self.detourReanchorBytes { detourResetRun() + sweepIdleDetourBlocks(force: true) timedReconnect(seek: true, at: curPosition) continue } @@ -2047,11 +2289,7 @@ final class AVIOReader: AVIOProvider, @unchecked Sendable { let available = window.count - posInWindow if available > 0 { let copyNow = min(available, requestSize - totalRead) - window.withUnsafeBytes { raw in - let src = raw.baseAddress!.advanced(by: posInWindow) - .assumingMemoryBound(to: UInt8.self) - buf.advanced(by: totalRead).update(from: src, count: copyNow) - } + window.copyBytes(to: buf.advanced(by: totalRead), from: posInWindow, count: copyNow) position = curPosition + Int64(copyNow) totalRead += copyNow trimWindowLocked() @@ -2242,7 +2480,7 @@ final class AVIOReader: AVIOProvider, @unchecked Sendable { let dropThreshold = Self.winLookback + Self.winTrimBatch if behind > dropThreshold { let drop = behind - Self.winLookback - window = window.subdata(in: drop.. DetourFetch { let budget = Self.effectiveDetourBudget(chunkRequestTimeout: chunkRequestTimeout) - let ticket = OriginRequestBudget.shared.acquire( - for: requestURL(), label: "\(label) detour", timeout: budget) - defer { OriginRequestBudget.shared.release(ticket) } let rangeEnd = offset + Int64(size) - 1 var request = URLRequest(url: requestURL()) request.setValue("bytes=\(offset)-\(rangeEnd)", forHTTPHeaderField: "Range") @@ -2486,7 +2725,9 @@ final class AVIOReader: AVIOProvider, @unchecked Sendable { request.timeoutInterval = budget applyExtraHeaders(&request) do { - let (data, response) = try syncRequest(request, budget: budget) + let (data, response) = try syncRequest( + request, budget: budget, slotLabel: "detour", + slotWait: min(budget, Self.shortFetchSlotWaitSeconds)) if let http = response as? HTTPURLResponse { let status = http.statusCode if Self.isRateLimitStatus(status) { @@ -2504,6 +2745,10 @@ final class AVIOReader: AVIOProvider, @unchecked Sendable { EngineLog.emit("[AVIOReader] detour: server ignored Range (200 for offset \(offset)); rejecting", category: .demux, level: .verbose) return .failed } + if !isLive, let served = Self.misplacedRangeStart(http, requestedOffset: offset) { + EngineLog.emit("[AVIOReader] detour: 206 starts at \(served), not \(offset); rejecting (audit DMX-5)", category: .demux) + return .failed + } } addBytesFetched(data.count) return .ok(data) @@ -2512,6 +2757,21 @@ final class AVIOReader: AVIOProvider, @unchecked Sendable { } } + /// Audit PERF-108: release detour blocks nothing has touched for `detourBlockIdleSeconds`, at most + /// once a second from the read path and unconditionally when playback re-anchors, which is the + /// moment a scrub's blocks stop being anyone's working set. Demux-thread-only. + private func sweepIdleDetourBlocks(force: Bool = false) { + let now = DispatchTime.now().uptimeNanoseconds + guard force || now &- lastDetourIdleSweepNs >= 1_000_000_000 else { return } + lastDetourIdleSweepNs = now + let evicted = detourCache.evictIdle(olderThan: detourBlockIdleSeconds) + if evicted > 0 { + EngineLog.emit( + "[AVIOReader] \(label) released \(evicted) idle detour block(s) " + + "(\(detourCache.residentCount) resident)", category: .demux, level: .verbose) + } + } + /// Tracks contiguity of detour reads for the re-anchor heuristic. Shared by BOTH the backward /// and far-forward branches; the re-anchor check itself lives only in the backward branch on /// purpose, so a forward-accumulated run later met by a contiguous backward read is an intended @@ -2740,16 +3000,25 @@ final class AVIOReader: AVIOProvider, @unchecked Sendable { /// Start offset of the bytes a 206 actually carries, from `Content-Range: bytes a-b/total`. /// Returns nil unless the header agrees with what arrived, so a proxy that answered a suffix /// request with some other region cannot install bytes at the wrong offset. + /// + /// Audit DMX-8: a suffix is the END of the source, so the span has to end on the byte before a + /// numeric total. Without that, `bytes <2^63-65536>-/*` installed a span whose `end` + /// overflows on the first read that reaches it. + /// + /// The total has to be positive before `total - 1` is taken: `Int64` parses a signed string, so + /// `bytes 0-0/-9223372036854775808` reached the subtraction with `Int64.min` and trapped. static func suffixRangeStart(_ http: HTTPURLResponse, expectedLength: Int) -> Int64? { guard let value = http.value(forHTTPHeaderField: "Content-Range") else { return nil } let scanner = value.replacingOccurrences(of: "bytes ", with: "") let parts = scanner.split(separator: "/", maxSplits: 1) - guard let range = parts.first else { return nil } - let bounds = range.split(separator: "-", maxSplits: 1) + guard parts.count == 2, + let total = Int64(parts[1].trimmingCharacters(in: .whitespaces)), + total > 0 else { return nil } + let bounds = parts[0].split(separator: "-", maxSplits: 1) guard bounds.count == 2, let start = Int64(bounds[0].trimmingCharacters(in: .whitespaces)), let end = Int64(bounds[1].trimmingCharacters(in: .whitespaces)), - start >= 0, end >= start, + start >= 0, end >= start, end == total - 1, end - start + 1 == Int64(expectedLength) else { return nil } return start } @@ -2823,8 +3092,8 @@ final class AVIOReader: AVIOProvider, @unchecked Sendable { winCond.lock() connGeneration &+= 1 let generation = connGeneration - // #220: VOD asks for a fixed amount at a time, so the origin cannot deliver more than - // was requested and the window is bounded by construction rather than by reaction. Two + // #220: VOD asks for a fixed amount at a time, and anything an origin delivers past it is + // dropped at the append (audit DMX-1), so the window is bounded by the ask. Two // cases keep the open-ended form: live, where the material is produced in real time so // the origin cannot outrun media rate and the end is moving, and a source whose total // size is not resolved yet, where there is nothing to clamp the last range against. An @@ -2859,10 +3128,10 @@ final class AVIOReader: AVIOProvider, @unchecked Sendable { // new range, which is correct if slightly wasteful, and never leaves a hole. if offset >= winStart, offset <= winStart + Int64(window.count) { let keep = Int(offset - winStart) - if keep < window.count { window = window.subdata(in: 0.. room { + data = data.prefix(Int(max(0, room))) + overDelivered = true + } + } lastDeliveryAt = DispatchTime.now() // #309: the delivery-gap watchdog's only input var firstDataMs: Double? = nil if !connFirstDataSeen { @@ -3149,14 +3435,7 @@ final class AVIOReader: AVIOProvider, @unchecked Sendable { } } let base = window.count - window.count = base + count - window.withUnsafeMutableBytes { dst in - data.withUnsafeBytes { src in - if let d = dst.baseAddress, let s = src.baseAddress { - (d + base).copyMemory(from: s, byteCount: count) - } - } - } + window.append(data) // #281 retest: retain the head of the file for the open phase, as it arrives. It cannot be // copied later out of the window, because `trimWindowLocked` drops it as the parse reads // forward, which is why the seek-time park misses on every layout whose parse reads more @@ -3169,9 +3448,14 @@ final class AVIOReader: AVIOProvider, @unchecked Sendable { addBytesFetched(count) // #220: the requested range has been delivered in full. That ends the connection on // purpose; the read loop re-requests at the frontier once the consumer has drawn down. + var overDeliveredTransfer: (any PersistentTransfer)? if let end = connRangeEnd, winStart + Int64(window.count) > end { connEndedAtRangeEnd = true connEnded = true + if overDelivered { + overDeliveredTransfer = activeTransfer + activeTransfer = nil + } } winCond.broadcast() // #310: past high water the connection is ENDED, not suspended. suspend() is @@ -3200,6 +3484,14 @@ final class AVIOReader: AVIOProvider, @unchecked Sendable { winCond.broadcast() } winCond.unlock() + if let overDeliveredTransfer { + EngineLog.emit( + "[AVIOReader] \(label) gen=\(generation) origin delivered past the requested range " + + "end; ending the connection (audit DMX-1)", + category: .demux) + overDeliveredTransfer.cancelTransfer() + overDeliveredTransfer.releaseOriginTicket() + } if let toCancel { EngineLog.emit( "[AVIOReader] \(label) window high water: \(ahead / 1024 / 1024)MB ahead; ending the " @@ -3264,6 +3556,16 @@ final class AVIOReader: AVIOProvider, @unchecked Sendable { // from byte 0 (silent corruption). Reject it. Live is exempt: transcode // reconnect legitimately answers 200 with "from now". let requestedOffset = (generation == connGeneration) ? connRequestedOffset : 0 + // Audit DMX-101: the cold open asked for a bounded range at byte 0. A 200 whose body is not + // that range is an origin that cannot address bytes, and this connection is abandoned at the + // head: nothing below may adopt its Content-Length as a size the persistent reader could + // seek by. + var rangeIgnoredAtOpen = false + if generation == connGeneration, !isLive, rangeVerdictPending, + Self.rangeAnswer(http, requestedStart: requestedOffset, requestedEnd: connRangeEnd) == .ignored { + rangeIgnoredAtOpen = true + rangeIgnoredAtOpenLength = http.expectedContentLength + } // #331: the origin just told us its live stream has no byte address to resume at. Latch // it, so every later request in this session is the join shape instead of repeating an // offset that can only ever be rejected again. Nonzero offsets only: a 416 at zero is a @@ -3295,7 +3597,7 @@ final class AVIOReader: AVIOProvider, @unchecked Sendable { fileSize = 0 adoptedWarmSize = nil } - if generation == connGeneration, !isLive, fileSize <= 0, + if generation == connGeneration, !isLive, fileSize <= 0, !rangeIgnoredAtOpen, let total = Self.sizeFromResponse(http, requestedOffset: requestedOffset) { fileSize = total // #112: share this resolved length so a later side demuxer on the same origin can skip a probe that @@ -3328,6 +3630,24 @@ final class AVIOReader: AVIOProvider, @unchecked Sendable { ) isOK = false } + if rangeIgnoredAtOpen { + EngineLog.emit( + "[AVIOReader] \(label) origin answered the ranged open with a 200 " + + "(Content-Length \(http.expectedContentLength)); hanging up at the head (audit DMX-101)", + category: .demux) + isOK = false + } + // Live is exempt for the same reason as above: its offset is bookkeeping, and the append + // anchors the bytes at the frontier whatever the origin calls them. + if isOK, !isLive, generation == connGeneration, + let served = Self.misplacedRangeStart(http, requestedOffset: requestedOffset) { + EngineLog.emit( + "[AVIOReader] \(label) gen=\(generation) 206 starts at \(served), not the requested " + + "\(requestedOffset); rejecting body (audit DMX-5)", + category: .demux + ) + isOK = false + } if isOK { recordResolvedURL(respondedBy) @@ -3364,7 +3684,7 @@ final class AVIOReader: AVIOProvider, @unchecked Sendable { // #310: our own high-water cancel completes here as NSURLErrorCancelled. That end was // already logged where it was decided; reporting it as an error too would make every // drain cycle against a fast origin read like a transport fault. - let deliberateEnd = isCurrentGen && connEndedByBackpressure + let deliberateEnd = isCurrentGen && (connEndedByBackpressure || connEndedAtRangeEnd) let windowAhead = isCurrentGen ? (window.count - max(0, Int(position - winStart))) : 0 winCond.broadcast() winCond.unlock() @@ -3422,9 +3742,9 @@ final class AVIOReader: AVIOProvider, @unchecked Sendable { let delegate = StreamingDelegate( extraHeaders: headers(for: request.url), onResponse: { [weak self] response in - // Advisory length for the sequential-origin EOF/EIO distinction; -1 (chunked / - // unknown) leaves the clean-end path as the only EOF source. - guard let self, self.sequentialOnly else { return } + // Advisory length for the EOF/EIO distinction; -1 (chunked / unknown) leaves the + // clean-end path as the only EOF source. + guard let self else { return } let expected = response.expectedContentLength guard expected > 0 else { return } self.streamLock.lock() @@ -3451,26 +3771,53 @@ final class AVIOReader: AVIOProvider, @unchecked Sendable { ) { [weak self] data in guard let self, !self.isClosed else { return } self.streamLock.lock() - self.streamBuffer.append(data) + if self.streamFailed { + self.streamLock.unlock() + return + } + // Owned bytes: the chunk is retained until it is consumed, and a delivery that still + // references the transport's dispatch_data would pin its buffer for that long. + self.streamBuffer.append(data.withUnsafeBytes { Data($0) }) + self.streamPeakBytes = max(self.streamPeakBytes, self.streamBuffer.count) + var toCancel: URLSessionDataTask? + if self.streamBuffer.count > self.streamHardCap { + self.streamFailed = true + self.streamFailureCause = .originIgnoresFlowControl + toCancel = self.streamingTask + } // Backpressure: park the transfer once the retained buffer // exceeds the high water mark; readStreaming resumes it when // the consumer drains below the low water mark (and before // any wait, so a far-forward seek can't deadlock against a // suspended producer). var toSuspend: URLSessionDataTask? - if !self.streamingTaskSuspended, self.streamBuffer.count > Self.streamHighWater { + if !self.streamingTaskSuspended, self.streamBuffer.count > self.streamHighWater { self.streamingTaskSuspended = true toSuspend = self.streamingTask } self.streamLock.unlock() - toSuspend?.suspend() + if let toCancel { + EngineLog.emit( + "[AVIOReader] \(self.label) streaming buffer passed \(self.streamHardCap / 1024)KB " + + "with the transfer suspended: the origin ignores flow control, so the source is " + + "ended here to keep memory bounded and the read will fail (audit DMX-7, DMX-107)", + category: .demux) + toCancel.cancel() + } else { + toSuspend?.suspend() + } self.addBytesFetched(data.count) self.streamDataReady.signal() - } onComplete: { [weak self] in - self?.streamLock.lock() - self?.streamEnded = true - self?.streamLock.unlock() - self?.streamDataReady.signal() + } onComplete: { [weak self] error in + guard let self else { + semaphore.signal() + return + } + self.streamLock.lock() + if error != nil, !self.isClosed { self.streamFailed = true } + self.streamEnded = true + self.streamLock.unlock() + self.streamDataReady.signal() semaphore.signal() } @@ -3558,7 +3905,12 @@ final class AVIOReader: AVIOProvider, @unchecked Sendable { /// the same reason `recordRateLimitAndShouldGiveUp` is not private. Callers outside the file /// are tests; production reaches this through `seekCallback`. func seek(offset: Int64, whence: Int32) -> Int64 { - if whence == AVSEEK_SIZE { return fileSize } + if whence == AVSEEK_SIZE { + guard fileSize <= 0, originIgnoresRange else { return fileSize } + streamLock.lock() + defer { streamLock.unlock() } + return streamExpectedBytes + } // For persistent mode, position is shared with the delegate thread; // read SEEK_CUR base under the window lock. let newPosition: Int64 @@ -3622,6 +3974,73 @@ final class AVIOReader: AVIOProvider, @unchecked Sendable { return URLSession(configuration: config, delegate: EngineTLS.sessionDelegate, delegateQueue: nil) }() + /// Audit DMX-5: where a 206 says its body starts, when that is not where it was asked to. + /// Every ranged path here places the body at the REQUESTED offset, so a 206 aligned to an + /// edge's own chunk boundary would shift every later read by the difference. Nil for any other + /// status and for a Content-Range this cannot read (those keep the lenient historical path). + static func misplacedRangeStart(_ http: HTTPURLResponse, requestedOffset: Int64) -> Int64? { + guard let served = contentRangeSpan(http) else { return nil } + return served.start == requestedOffset ? nil : served.start + } + + /// First and last byte a 206 says it carries, nil for any other status and for a Content-Range + /// this cannot read. + static func contentRangeSpan(_ http: HTTPURLResponse) -> (start: Int64, end: Int64?)? { + guard http.statusCode == 206, + let value = http.value(forHTTPHeaderField: "Content-Range") else { return nil } + let trimmed = value.trimmingCharacters(in: .whitespaces) + guard trimmed.lowercased().hasPrefix("bytes ") else { return nil } + let span = trimmed.dropFirst("bytes ".count).split(separator: "/", maxSplits: 1).first ?? "" + guard let dash = span.firstIndex(of: "-"), + let start = Int64(span[.. RangeAnswer { + switch http.statusCode { + case 206: + if let served = misplacedRangeStart(http, requestedOffset: requestedStart) { + return .misplaced(start: served) + } + if let end = requestedEnd { + if let servedEnd = contentRangeSpan(http)?.end, servedEnd > end { return .overWide } + if http.expectedContentLength > end - requestedStart + 1 { return .overWide } + } + return .honoured + case 200: + guard requestedStart == 0 else { return .ignored } + let length = http.expectedContentLength + guard length > 0 else { return .ignored } + if let end = requestedEnd, length > end + 1 { return .ignored } + return .wholeFile + default: + return .unjudged + } + } + /// Total size from a data-connection response: `Content-Range` total on a 206, or /// `Content-Length` on a from-0 2xx (origins that answer 200 ignoring Range). Nil when /// the origin gave no usable length (chunked, or an unknown `*` total). Issue #70. @@ -3740,14 +4159,18 @@ final class AVIOReader: AVIOProvider, @unchecked Sendable { // Budget mirrors a single sequential probe (its own 20-25 s ceiling) plus the stagger; // isClosed teardown breaks the individual probes, which then signal outstanding down. + // Sodalite#173: the wait polls isClosed itself, since a fallback still asleep in its stagger + // would otherwise hold a closed open for up to the full stagger. let deadline = Date(timeIntervalSinceNow: Self.sizeProbeStaggerSeconds + min(25, chunkRequestTimeout) + 2) state.cond.lock() - while state.resolvedSize <= 0 && state.outstanding > 0 { - if !state.cond.wait(until: deadline) { break } + while state.resolvedSize <= 0 && state.outstanding > 0 && !isClosed { + let now = Date() + if now >= deadline { break } + _ = state.cond.wait(until: min(deadline, now.addingTimeInterval(0.05))) } let size = state.resolvedSize state.cond.unlock() - if size <= 0 { + if size <= 0, !isClosed { EngineLog.emit("[AVIOReader] no probe resolved a size, streaming mode (forward-only)", category: .demux) } return size > 0 ? size : -1 @@ -3885,6 +4308,13 @@ final class AVIOReader: AVIOProvider, @unchecked Sendable { ) return nil } + if !isLive, let served = Self.misplacedRangeStart(http, requestedOffset: offset) { + EngineLog.emit( + "[AVIOReader] chunk fetch: 206 starts at \(served), not \(offset); rejecting (audit DMX-5)", + category: .demux + ) + return nil + } } addBytesFetched(data.count) return data @@ -4010,7 +4440,11 @@ final class AVIOReader: AVIOProvider, @unchecked Sendable { private func requestTicket(for url: URL, label: String, timeout: TimeInterval) throws -> OriginRequestBudget.Ticket? { guard let probeControl else { - return OriginRequestBudget.shared.acquire(for: url, label: label, timeout: timeout) + let ticket = OriginRequestBudget.shared.acquire( + for: url, label: label, timeout: timeout, + shouldAbort: { [weak self] in self?.isClosed ?? true }) + if ticket == nil, isClosed { throw CancellationError() } + return ticket } guard !isClosed else { throw CancellationError() } try probeControl.check() @@ -4050,13 +4484,15 @@ final class AVIOReader: AVIOProvider, @unchecked Sendable { completion.wait() } - private func syncRequest(_ request: URLRequest, budget: TimeInterval = 35) throws -> (Data, URLResponse) { + private func syncRequest(_ request: URLRequest, budget: TimeInterval = 35, + slotLabel: String = "fetch", + slotWait: TimeInterval = AVIOReader.shortFetchSlotWaitSeconds) throws -> (Data, URLResponse) { // #377: every short fetch the reader makes (detour blocks, size probes, HEAD) funnels // through here, so this is the one place that has to take an origin slot for all of them. // Scoped to the call: unlike the pump's, this request's life IS this function's. let slotURL = request.url ?? url let ticket = try requestTicket( - for: slotURL, label: "\(label) fetch", timeout: Self.shortFetchSlotWaitSeconds) + for: slotURL, label: "\(label) \(slotLabel)", timeout: slotWait) defer { OriginRequestBudget.shared.release(ticket) } let delegate = ChunkFetchDelegate(extraHeaders: headers(for: request.url), @@ -4112,12 +4548,18 @@ final class DetourBlockCache: @unchecked Sendable { private let lock = NSLock() private var blocks: [Int64: Data] = [:] private var lru: [Int64] = [] + /// When each resident block was last inserted or served, on `now`'s clock (audit PERF-108). + private var lastHit: [Int64: TimeInterval] = [:] private let maxBlocks: Int + private let now: @Sendable () -> TimeInterval let blockSize: Int - init(blockSize: Int, maxBlocks: Int) { + /// `now` is injectable so the idle test moves a clock instead of waiting one out. + init(blockSize: Int, maxBlocks: Int, + now: @escaping @Sendable () -> TimeInterval = { ProcessInfo.processInfo.systemUptime }) { self.blockSize = blockSize self.maxBlocks = maxBlocks + self.now = now } /// Returns the resident block for `idx` and bumps its recency, or nil on a miss. @@ -4128,6 +4570,7 @@ final class DetourBlockCache: @unchecked Sendable { lru.remove(at: i) lru.append(idx) } + lastHit[idx] = now() return data } @@ -4136,15 +4579,38 @@ final class DetourBlockCache: @unchecked Sendable { lock.lock(); defer { lock.unlock() } if blocks[idx] == nil { lru.append(idx) } blocks[idx] = data + lastHit[idx] = now() while lru.count > maxBlocks { - blocks.removeValue(forKey: lru.removeFirst()) + let evicted = lru.removeFirst() + blocks.removeValue(forKey: evicted) + lastHit[evicted] = nil + } + } + + /// Drops every block nothing has read or filled for `seconds`, and returns how many went. The + /// cache is otherwise only emptied at close, so the blocks of a backward scrub (up to 32 MB) + /// outlived the scrub for the whole session (audit PERF-108). Age rather than a clear at the + /// re-anchor, because a parser ping-ponging through a region keeps hitting its blocks. + @discardableResult + func evictIdle(olderThan seconds: TimeInterval) -> Int { + lock.lock(); defer { lock.unlock() } + let cutoff = now() - seconds + let idle = lru.filter { (lastHit[$0] ?? 0) < cutoff } + guard !idle.isEmpty else { return 0 } + let gone = Set(idle) + lru.removeAll { gone.contains($0) } + for idx in idle { + blocks.removeValue(forKey: idx) + lastHit[idx] = nil } + return idle.count } func clear() { lock.lock() blocks.removeAll() lru.removeAll() + lastHit.removeAll() lock.unlock() } @@ -4573,7 +5039,7 @@ private final class ChunkFetchDelegate: NSObject, URLSessionDataDelegate, @unche private final class StreamingDelegate: NSObject, URLSessionDataDelegate { let onData: @Sendable (Data) -> Void - let onComplete: @Sendable () -> Void + let onComplete: @Sendable (Error?) -> Void /// Response hook (advisory Content-Length capture on the sequential-origin path). let onResponse: (@Sendable (URLResponse) -> Void)? /// The origin answered with a status instead of media (anything but 200/206). Called at the @@ -4591,7 +5057,7 @@ private final class StreamingDelegate: NSObject, URLSessionDataDelegate { onResponse: (@Sendable (URLResponse) -> Void)? = nil, onRefused: (@Sendable (Int, URL?) -> Void)? = nil, onData: @escaping @Sendable (Data) -> Void, - onComplete: @escaping @Sendable () -> Void + onComplete: @escaping @Sendable (Error?) -> Void ) { self.extraHeaders = extraHeaders self.onResponse = onResponse @@ -4650,7 +5116,7 @@ private final class StreamingDelegate: NSObject, URLSessionDataDelegate { EngineLog.emit("[AVIOReader] Stream error: \(error.localizedDescription)", category: .demux) } #endif - onComplete() + onComplete(error) } } @@ -4871,7 +5337,10 @@ private final class TailPrefetchDelegate: NSObject, URLSessionDataDelegate, @unc func urlSession(_ session: URLSession, dataTask: URLSessionDataTask, didReceive data: Data) { guard buffer.count < expectedLength else { return } - buffer.append(data) + // Audit DMX-9: clamped, as `RangeFetchDelegate` does. Appended whole, an origin that sends a + // few bytes past its own Content-Range read as a SHORT body and latched the origin as one + // that declines suffix ranges for the rest of the process. + buffer.append(data.prefix(expectedLength - buffer.count)) } func urlSession(_ session: URLSession, task: URLSessionTask, didCompleteWithError error: Error?) { @@ -4941,6 +5410,15 @@ enum AVIOReaderError: Error, Equatable, CustomStringConvertible, LocalizedError /// the same reason `httpStatus` is: without it the open surfaces FFmpeg's invalid data and a /// self-signed origin reads as a corrupt file. case transportSecurityFailed(code: Int) + /// Audit NET-102: a remote disc image on an origin that answers a range request with the whole + /// source (a 200). A disc image is a filesystem, which no reader can walk forward-only, so this + /// fails at the first response head. A plain media URL on such an origin plays forward-only + /// instead (audit DMX-101). + case originIgnoresRange + /// Audit DMX-107: a forward-only source kept delivering after the reader paused it, past the + /// streaming buffer's hard cap, so the source was ended to keep memory bounded. The read that + /// follows fails with EIO, and `AVIOReader.lastReadFailure` says why. + case originIgnoresFlowControl var description: String { switch self { @@ -4952,6 +5430,10 @@ enum AVIOReaderError: Error, Equatable, CustomStringConvertible, LocalizedError case .httpStatus(let status): return "Origin answered HTTP \(status) for the source" case .transportSecurityFailed(let code): return TransportSecurityFailure.sentence(for: code) + case .originIgnoresRange: + return "The origin ignores HTTP Range requests, which a remote disc image needs" + case .originIgnoresFlowControl: + return "The origin kept sending after the reader paused it, past the streaming buffer's bound" } } diff --git a/Sources/AetherEngine/Demuxer/ChunkedByteWindow.swift b/Sources/AetherEngine/Demuxer/ChunkedByteWindow.swift new file mode 100644 index 000000000..9e9bcaafb --- /dev/null +++ b/Sources/AetherEngine/Demuxer/ChunkedByteWindow.swift @@ -0,0 +1,125 @@ +import Foundation + +/// AE#619: the persistent reader's byte window, held as the chunks the connection delivered. +/// +/// The reader used to keep one contiguous `Data` and drop its consumed head with `subdata(in:)`, +/// which copied everything still ahead of the cut: up to ~18 MB once per 4 MB consumed, several +/// times the stream's own rate in `memmove`, and each append after it reallocated the fresh buffer. +/// Here a drop advances an offset and releases the chunks it passed, and nothing ahead of it moves. +/// Callers see one contiguous run of `count` bytes; the chunking is invisible to them. +/// +/// Not thread-safe. The reader guards it with `winCond` like the `Data` it replaces. +struct ChunkedByteWindow { + private var chunks: [Data] = [] + /// Storage offset one past the end of `chunks[i]`. Storage offsets only grow until + /// `removeAll`, so a drop never rewrites them. + private var ends: [Int] = [] + /// Index of the first chunk still holding live bytes. + private var first = 0 + /// Storage offset of logical byte 0. + private var head = 0 + /// Storage offset one past the last byte. + private var tail = 0 + + var count: Int { tail - head } + var isEmpty: Bool { tail == head } + + mutating func append(_ data: Data) { + guard !data.isEmpty else { return } + chunks.append(data) + tail += data.count + ends.append(tail) + } + + /// Drops the first `n` bytes. `n` is clamped to `count`. + mutating func dropFirst(_ n: Int) { + head += min(max(0, n), count) + while first < chunks.count, ends[first] <= head { + chunks[first] = Data() + first += 1 + } + if first == chunks.count { + removeAll() + } else if first >= 64, first * 2 >= chunks.count { + chunks.removeFirst(first) + ends.removeFirst(first) + first = 0 + } + } + + /// Keeps the first `keep` bytes and drops the rest. `keep` is clamped to `0...count`. + mutating func truncate(to keep: Int) { + let newTail = head + min(max(0, keep), count) + guard newTail < tail else { return } + if newTail == head { + removeAll() + return + } + let i = chunkIndex(containing: newTail - 1) + let start = ends[i] - chunks[i].count + let c = chunks[i] + chunks[i] = c[c.startIndex.., from offset: Int, count n: Int) { + precondition(offset >= 0 && n >= 0 && offset + n <= count, "ChunkedByteWindow read out of range") + guard n > 0 else { return } + var position = head + offset + var written = 0 + var i = chunkIndex(containing: position) + while written < n { + let c = chunks[i] + let start = ends[i] - c.count + let local = position - start + let take = min(c.count - local, n - written) + let from = c.startIndex + local + c.copyBytes(to: UnsafeMutableBufferPointer(start: destination + written, count: take), + from: from.. [UInt8] { + let take = min(max(0, n), count) + var out = [UInt8](repeating: 0, count: take) + out.withUnsafeMutableBufferPointer { buffer in + if let base = buffer.baseAddress { copyBytes(to: base, from: 0, count: take) } + } + return out + } + + /// Index of the chunk holding storage offset `position`, which must be in `head.. Int { + var lower = first + var upper = ends.count - 1 + while lower < upper { + let middle = lower + (upper - lower) / 2 + if ends[middle] <= position { lower = middle + 1 } else { upper = middle } + } + return lower + } + + #if DEBUG + /// Test hook: the storage address of every live chunk, first to last. + func chunkStorageAddresses() -> [UnsafeRawPointer?] { + chunks[first...].map { $0.withUnsafeBytes { UnsafeRawPointer($0.baseAddress) } } + } + #endif +} diff --git a/Sources/AetherEngine/Demuxer/CustomIOReaderBridge.swift b/Sources/AetherEngine/Demuxer/CustomIOReaderBridge.swift index 6b6eabd0e..0032ffe57 100644 --- a/Sources/AetherEngine/Demuxer/CustomIOReaderBridge.swift +++ b/Sources/AetherEngine/Demuxer/CustomIOReaderBridge.swift @@ -40,6 +40,11 @@ final class CustomIOReaderBridge: AVIOProvider, @unchecked Sendable { readDeadline = .distantFuture } + private var readByteBudget = ReadByteBudget() + var readByteBudgetExhausted: Bool { readByteBudget.exhausted } + func beginReadByteBudget(_ bytes: Int64) { readByteBudget.begin(bytes) } + func endReadByteBudget() { readByteBudget.end() } + /// #112 round 9: the byte axis libavformat sees through this bridge (for a disc adapter, the /// virtual concat stream length via AVSEEK_SIZE), backing the byte-estimate seek fallback. var resolvedByteSize: Int64? { @@ -180,8 +185,10 @@ final class CustomIOReaderBridge: AVIOProvider, @unchecked Sendable { // -1 = forced abort (not EOF); mirrors AVIOReader.read so FFmpeg doesn't run EOS handling. guard !isClosed else { return -1 } if isPastReadDeadline { readDeadlineFired = true; return -1 } - let n = callingHost { reader.read(buf, size: size) } + guard let allowed = readByteBudget.allowance(size) else { return -1 } + let n = callingHost { reader.read(buf, size: allowed) } if n == 0 { return FFmpegErr.eof } // IOReader uses 0 for EOF; avio expects AVERROR_EOF. + readByteBudget.consumed(n) return n } diff --git a/Sources/AetherEngine/Demuxer/Demuxer.swift b/Sources/AetherEngine/Demuxer/Demuxer.swift index 7465a04b1..502205492 100644 --- a/Sources/AetherEngine/Demuxer/Demuxer.swift +++ b/Sources/AetherEngine/Demuxer/Demuxer.swift @@ -2,6 +2,7 @@ import Foundation import AetherLibavformat import AetherLibavcodec import AetherLibavutil +import os /// The stages an `open()` passes through, reported as each one finishes (#361). Deliberately the @@ -116,6 +117,17 @@ struct DemuxerOpenProfile: Sendable { auditsRecordlessDolbyVision: false ) + /// AE#682: the I-frame rendition's side reader. Reads one keyframe per request, so it takes the + /// still extractor's tuning under a name of its own (#240: a connection line without a name + /// cannot say which reader opened it). + static let iFrameSideDemuxer = stillExtraction.withReaderLabel("iframe") + + /// Readers that fetch single keyframes and publish no timestamp axis, so the #409 repair's + /// sample window would be packets read for nothing. + static let labelsWithoutCompositionRepair: Set = [ + stillExtraction.readerLabel, iFrameSideDemuxer.readerLabel, + ] + /// A copy of `self` with only the open-time probe budget overridden (#68). /// A non-nil `probesize` / `maxAnalyzeDuration` replaces the matching field; /// nil keeps the receiver's value. The AVIO tuning (prefetch, chunk size, @@ -225,9 +237,94 @@ public final class Demuxer: @unchecked Sendable { // concurrent access triggers assertion failures in matroskadec.c. private let accessLock = NSLock() - private var avioProvider: AVIOProvider? + /// Audit DMX-11: `markClosed()` is the lock-free cross-thread abort, so it loads this reference + /// while the demux thread may be clearing it in `close()` or a failed open. A leaf lock of its + /// own (never `accessLock`, which `av_read_frame` holds for a whole network read) makes that + /// load a retained snapshot instead of a race. + private let providerLock = NSLock() + private var _avioProvider: AVIOProvider? + private var avioProvider: AVIOProvider? { + get { + providerLock.lock() + defer { providerLock.unlock() } + return _avioProvider + } + set { + providerLock.lock() + _avioProvider = newValue + providerLock.unlock() + } + } + + /// Audit NET-110: the disc reader `openHTTP` builds for a remote disc image. The disc adapter's + /// `ConcatIOReader.close()` is a no-op and the bridge does not own its reader, so this is the only + /// owner left to end the reader's `URLSession`. Guarded by `providerLock`. + private var _ownedSourceReader: IOReader? + + private func adoptOwnedSourceReader(_ reader: IOReader) { + providerLock.lock() + let previous = _ownedSourceReader + _ownedSourceReader = reader + providerLock.unlock() + previous?.close() + } + + private func releaseOwnedSourceReader() { + providerLock.lock() + let reader = _ownedSourceReader + _ownedSourceReader = nil + providerLock.unlock() + reader?.close() + } + + /// Audit DMX-102: unblocks the disc reader while `DiscReader.wrap` is still reading through it, + /// before any provider exists for `markClosed()` to reach. + private func cancelOwnedSourceReader() { + providerLock.lock() + let reader = _ownedSourceReader + providerLock.unlock() + reader?.cancel() + } + + /// Audit HLS-2: `markClosed()` before the provider exists used to be a no-op, so a teardown + /// that raced an in-flight open let it finish its connect and probe. + private let closeRequestLock = NSLock() + private var closeRequested = false + private var isCloseRequested: Bool { closeRequestLock.withLock { closeRequested } } private var openProfile: DemuxerOpenProfile = .playback + /// Audit NAT-7: the stream pointers `stream(at:)` hands out, copied out of `formatContext` + /// under `accessLock`. MPEG-TS adds streams inside `av_read_frame`, which reallocates the + /// `streams` array, so a caller on another thread must not index the live array while a read + /// holds the lock. The `AVStream`s themselves live until `avformat_close_input`. Guarded by + /// `streamTableLock`, a leaf lock; `streamTableSize` is its length, guarded by `accessLock`. + /// + /// Audit DMX-108: also the gate `withStream` and `close()` meet at. A caller inside + /// `withStream` counts itself in `streamUsers`, and `close()` empties the table (so no new one + /// can start) and waits for the count to reach zero before `avformat_close_input` frees the + /// `AVStream`s. The condition is on the leaf lock, so the wait never holds `accessLock`'s + /// network reads against anyone. + private let streamTableLock = NSCondition() + private var streamTable: [UnsafeMutablePointer] = [] + private var streamTableSize = 0 + private var streamUsers = 0 + + /// Audit DMX-108: what the track accessors answer with when a read holds `accessLock`. + /// MPEG-TS adds streams inside `av_read_frame`, which reallocates `streams`, so the accessors + /// walk the live array only under the lock, and the lock can be held for a whole network read. + /// An accessor that finds it busy answers from the last snapshot instead of waiting, the same + /// contract NAT-7 gave `stream(at:)`. Guarded by `streamTableLock`. + private struct TrackSnapshot { + var videoStreamIndex: Int32 = -1 + var audioStreamIndex: Int32 = -1 + var firstAudioStreamIndexByType: Int32 = -1 + var audioTracks: [TrackInfo] = [] + var subtitleTracks: [TrackInfo] = [] + var subtitleStreamIndices: Set = [] + var splitDisplaySetSubtitleStreamIndices: Set = [] + } + private var trackSnapshot = TrackSnapshot() + /// The URL and headers this demuxer was opened from, kept for the recordless Dolby Vision audit's /// second open. nil for a custom reader (no second open to give) and for a live source. private var auditSource: (url: URL, headers: [String: String])? @@ -240,6 +337,12 @@ public final class Demuxer: @unchecked Sendable { private var compositionRepair: (any H264TimestampRepairSession)? private var compositionRepairEvaluated = false + /// Audit HLS-103: packets `peekPackets` read ahead of the cursor on a source that cannot rewind. + /// They are already through everything `readPacket` does to a packet (the #409 repair, the + /// timestamp bound), so `readPacket` hands them out first and as they are. Freed by a + /// reposition and by `close()`. Guarded by `accessLock`. + private var peekedPackets: [UnsafeMutablePointer] = [] + /// #407: video streams whose PTS `+genpts` invented out of decode order, because the container /// carries none of its own. Cleared on the way out of `readPacketLocked` so the decoder's reorder /// owns the presentation axis. Decided once at open, see `armGeneratedPTSSuppression`. @@ -307,6 +410,51 @@ public final class Demuxer: @unchecked Sendable { /// leaves it nil and emits nothing. var onOpenProgress: (@Sendable (DemuxerOpenStage) -> Void)? + /// AE#678: when this open started and when each stage finished, for the one timing line an open + /// emits. `#361`'s checkpoints carry the same boundaries to the host, but only as a ladder with no + /// durations, and a downstream integrator had to guess whether the connect, the container or the + /// stream analysis was the slow leg. + private var openStartedAt: DispatchTime? + private var openStageTimes: [DemuxerOpenStage: DispatchTime] = [:] + + private func beginOpenTiming() { + openStartedAt = DispatchTime.now() + openStageTimes = [:] + } + + private func reportOpenStage(_ stage: DemuxerOpenStage) { + openStageTimes[stage] = DispatchTime.now() + onOpenProgress?(stage) + if stage == .streamsProbed { emitOpenTimings(outcome: nil) } + } + + /// Stills and the subtitle side reader open far too often to narrate, and the latter skips the + /// stream analysis the line exists to time. + private var reportsOpenTimings: Bool { + openProfile.readerLabel != DemuxerOpenProfile.stillExtraction.readerLabel && !openProfile.skipStreamInfo + } + + private func emitOpenTimings(outcome: String?) { + guard reportsOpenTimings, let start = openStartedAt else { return } + openStartedAt = nil + func ms(_ from: DispatchTime?, _ to: DispatchTime?) -> String { + guard let from, let to, to.uptimeNanoseconds >= from.uptimeNanoseconds else { return "-" } + return "\((to.uptimeNanoseconds - from.uptimeNanoseconds) / 1_000_000)ms" + } + let source = openStageTimes[.sourceOpened] + let container = openStageTimes[.containerOpened] + let probed = openStageTimes[.streamsProbed] ?? (outcome == nil ? nil : DispatchTime.now()) + let streams = formatContext.map { Int($0.pointee.nb_streams) } ?? 0 + EngineLog.emit( + "[Demuxer] open timings (\(openProfile.readerLabel)): connect \(ms(start, source)), " + + "open_input \(ms(source ?? start, container)), find_stream_info \(ms(container, probed)), " + + "total \(ms(start, probed)); \(streams) stream(s), probesize \(openProfile.probesize / 1_048_576) MiB, " + + "analyzeduration \(openProfile.maxAnalyzeDuration / 1_000_000) s" + + (outcome.map { "; \($0)" } ?? ""), + category: .demux + ) + } + // MARK: - Disc titles / chapters (#67) private(set) var discTitles: [DiscTitle] = [] @@ -317,6 +465,11 @@ public final class Demuxer: @unchecked Sendable { /// language inside the stream, so `trackInfo` backfills undetermined tracks from this; empty for every /// non-disc source, where it is a no-op (#527). private(set) var discStreamLanguages: [Int: String] = [:] + /// #651: the selected DVD title's declared subpicture substream ids, created before the probe. + private var discSubpictureStreamIDs: [Int]? + /// #651: one assembler per stream `declareDiscSubpictureStreams` created, keyed by stream index. + /// Those streams have no libavformat parser, so their fragments are joined here. Under `accessLock`. + private var subpictureAssemblers: [Int32: DVDSubpictureAssembler] = [:] /// Per-clip presentation-offset spans for a selected multi-clip Blu-ray title (empty otherwise). When /// non-empty, `readPacket` and `indexedKeyframes` fold each clip's timestamps onto one contiguous @@ -346,6 +499,7 @@ public final class Demuxer: @unchecked Sendable { discTitles = info.titles selectedDiscTitleIndex = info.selectedTitleIndex discStreamLanguages = info.selectedTitle?.streamLanguages ?? [:] + discSubpictureStreamIDs = info.selectedTitle?.dvdSubpictureStreamIDs clipTimeline = info.clipTimeline lastClipIndex = 0 lastReadClipIdx = -1 @@ -369,14 +523,53 @@ public final class Demuxer: @unchecked Sendable { /// Fold a raw timestamp onto the contiguous presentation timeline given its byte position and time base. /// Must be called under `accessLock`. - private func normalizedTimestamp(_ ts: Int64, pos: Int64, timeBase: AVRational) -> Int64 { + private func normalizedTimestamp(_ ts: Int64, pos: Int64, timeBase: AVRational) -> Int64? { guard ts != Int64.min, !clipTimeline.isEmpty else { return ts } - let sub = clipSubtractSeconds(forPos: pos) + return Self.foldedIndexTimestamp(ts, subtractSeconds: clipSubtractSeconds(forPos: pos), timeBase: timeBase) + } + + /// The AE#105 fold of one index entry, nil when the shift or the result leaves the tick range. The + /// shift comes from the disc's own playlist, and a wrapped entry is worse than a missing one: the + /// plan built from it lands wherever the wrap put it (audit HLS-102). + static func foldedIndexTimestamp(_ ts: Int64, subtractSeconds sub: Double, timeBase: AVRational) -> Int64? { guard sub != 0, timeBase.num > 0, timeBase.den > 0 else { return ts } - let subTicks = Int64((sub * Double(timeBase.den) / Double(timeBase.num)).rounded()) - return ts &- subTicks + let subTicks = (sub * Double(timeBase.den) / Double(timeBase.num)).rounded() + guard subTicks.isFinite, abs(subTicks) < Self.maxPlausibleIndexTicks else { return nil } + let (folded, overflow) = ts.subtractingReportingOverflow(Int64(subTicks)) + return overflow ? nil : folded } + /// The #409 decode-ladder offset applied to one index entry, nil on overflow (audit HLS-102). + static func offsetIndexTimestamp(_ ts: Int64, by offset: Int64?) -> Int64? { + guard let offset else { return ts } + let (placed, overflow) = ts.addingReportingOverflow(offset) + return overflow ? nil : placed + } + + /// Whether an index entry can be a real position (audit HLS-102), by the same rule the packet + /// funnel applies (`SourceTimestampBounds.plausible(_:timeBase:)`). libavformat rejects only NOPTS + /// and the relative-timestamp band near `Int64.max`; entries near `Int64.min` and just below the + /// band reach the plan builders otherwise. + static func isPlausibleIndexTimestamp(_ ts: Int64, timeBase: AVRational) -> Bool { + guard timeBase.num > 0, timeBase.den > 0 else { return false } + return SourceTimestampBounds.plausible(ts, timeBase: timeBase) != Int64.min + } + + static let maxPlausibleIndexTicks = Double(SourceTimestampBounds.demuxedMagnitude) + + /// Seconds as ticks on `timeBase`, nil unless the result is finite and under 2^62 ticks (audit + /// DMX-113, BIT-105). This is where every seconds-based reposition becomes an integer, and + /// `Int64(_:)` traps on NaN, infinity and anything past `Int64`; every caller already treats a + /// failed seek as one. Truncates like the plain conversion it replaces. + nonisolated static func ticks(forSeconds seconds: Double, timeBase: AVRational) -> Int64? { + guard timeBase.num > 0, timeBase.den > 0 else { return nil } + let ticks = seconds * Double(timeBase.den) / Double(timeBase.num) + guard ticks.isFinite, abs(ticks) < maxPlausibleIndexTicks else { return nil } + return Int64(ticks) + } + + nonisolated static let avTimeBase = AVRational(num: 1, den: AV_TIME_BASE) + /// True once a disc structure (BD/DVD/UDF) was recognized at open. Disc sources concat /// MPEG-TS / VOB clips and have no EOF cue index, so the MKV cue-index prewarm seek is /// useless there and a cold mid-disc range read is expensive on a remote ISO (#76). @@ -414,12 +607,23 @@ public final class Demuxer: @unchecked Sendable { /// everything - the non-seekable pb ran no tail estimate, so the container value is 0 or /// garbage from fabricated range data - then a custom time-seekable reader's own duration, /// then the disc/container resolution above. + /// + /// The result is clamped to `[0, MediaDurationCeiling.seconds]` (audit HLS-102): a corrupt header + /// can state about 9.2e12 s, which the segment plan turned into a trapping tick conversion or a + /// multi-terabyte reservation. Clamped rather than zeroed, so a slightly broken header still plays. static func effectiveDurationSeconds( declared: Double?, readerDuration: Double?, discTitle: Double?, container: Double ) -> Double { - if let declared, declared > 0 { return declared } - if let readerDuration, readerDuration > 0 { return readerDuration } - return effectiveDurationSeconds(discTitle: discTitle, container: container) + let resolved: Double + if let declared, declared > 0 { + resolved = declared + } else if let readerDuration, readerDuration > 0 { + resolved = readerDuration + } else { + resolved = effectiveDurationSeconds(discTitle: discTitle, container: container) + } + guard resolved.isFinite, resolved > 0 else { return 0 } + return min(resolved, MediaDurationCeiling.seconds) } /// Open a media URL and probe its streams. @@ -427,7 +631,9 @@ public final class Demuxer: @unchecked Sendable { /// - extraHeaders: Attached to every HTTP request (ignored for file:// URLs). /// - isLive: Suppresses EOF synthesis and surfaces terminal error on reconnect cap. func open(url: URL, extraHeaders: [String: String] = [:], profile: DemuxerOpenProfile = .playback, isLive: Bool = false, selectTitleID: Int? = nil) throws { + if isCloseRequested { throw DemuxerError.openFailed(code: -1) } self.openProfile = profile + beginOpenTiming() self.auditSource = isLive ? nil : (url, extraHeaders) let isHTTP = url.scheme == "http" || url.scheme == "https" @@ -469,10 +675,13 @@ public final class Demuxer: @unchecked Sendable { formatContext = openedCtx // #361: a local file has no connection to come up, so the source stage is credited by this // one rather than reported separately. - onOpenProgress?(.containerOpened) + reportOpenStage(.containerOpened) try probeStreams(openedCtx) - onOpenProgress?(.streamsProbed) + accessLock.lock() + refreshStreamTableLocked() + accessLock.unlock() + reportOpenStage(.streamsProbed) } /// Open a custom `IOReader` source. `formatHint` disambiguates probing when @@ -481,7 +690,9 @@ public final class Demuxer: @unchecked Sendable { /// DVD/BD ISOs to VOB/MPEGTS concat streams unless the reader opts out via /// `discImageProbeEnabled`. func open(reader: IOReader, formatHint: String? = nil, profile: DemuxerOpenProfile = .playback, isLive: Bool = false, selectTitleID: Int? = nil, discCacheKey: String? = nil) throws { + if isCloseRequested { throw DemuxerError.openFailed(code: -1) } self.openProfile = profile + beginOpenTiming() self.auditSource = nil if reader.discImageProbeEnabled, let discInfo = try DiscReader.wrap(reader, selectTitleID: selectTitleID, cacheKey: discCacheKey) { @@ -507,17 +718,39 @@ public final class Demuxer: @unchecked Sendable { // straight to libavformat fails to probe; it is a filesystem, not a media container, #64). // Gated on the disc-image extension so normal media URLs skip the range-probe entirely; if // the source is not a recognizable disc, fall through to the streaming reader. - if !isLive, Self.isDiscImageURL(url), - let discReader = HTTPDiscIOReader(url: url, extraHeaders: extraHeaders) { - if let discInfo = try DiscReader.wrap(discReader, selectTitleID: selectTitleID, cacheKey: url.absoluteString) { - adoptDiscInfo(discInfo) - auditSource = nil - let bridge = CustomIOReaderBridge(reader: discInfo.reader) - let inputFormat = av_find_input_format(discInfo.formatHint) - try openWithProvider(bridge, inputFormat: inputFormat, isLive: false) - return + if !isLive, Self.isDiscImageURL(url) { + let warm = HTTPDiscIOReader.takePrewarm(for: url, extraHeaders: extraHeaders) + if let discReader = try HTTPDiscIOReader.open(url: url, extraHeaders: extraHeaders, prewarmed: warm) { + adoptOwnedSourceReader(discReader) + if isCloseRequested { + releaseOwnedSourceReader() + throw DemuxerError.openFailed(code: -1) + } + let discInfo: DiscInfo? + do { + discInfo = try DiscReader.wrap(discReader, selectTitleID: selectTitleID, cacheKey: url.absoluteString) + } catch { + releaseOwnedSourceReader() + if let warm { SourcePrewarmStore.shared.store(warm, for: url) } + throw error + } + if let discInfo { + adoptDiscInfo(discInfo) + auditSource = nil + let bridge = CustomIOReaderBridge(reader: discInfo.reader) + let inputFormat = av_find_input_format(discInfo.formatHint) + do { + try openWithProvider(bridge, inputFormat: inputFormat, isLive: false) + } catch { + releaseOwnedSourceReader() + throw error + } + return + } + releaseOwnedSourceReader() } - discReader.close() + // Not a disc: hand the warm back so the streaming reader below adopts it (#647). + if let warm { SourcePrewarmStore.shared.store(warm, for: url) } } let reader = AVIOReader( url: url, @@ -537,6 +770,13 @@ public final class Demuxer: @unchecked Sendable { try openWithProvider(reader, isLive: isLive) } + /// A provider that connected (or tried to) for an open that is not going to finish. + private func abandonProvider(_ provider: AVIOProvider) { + provider.markClosed() + provider.close() + avioProvider = nil + } + /// Common AVIO open path. `inputFormat` forces a demuxer (custom sources with /// a format hint). `isLive` suppresses duration-estimate SEEK_END that latches /// EOF on unknown-length live sources. @@ -545,9 +785,26 @@ public final class Demuxer: @unchecked Sendable { inputFormat: UnsafePointer? = nil, isLive: Bool = false ) throws { - try provider.open() - avioProvider = provider - onOpenProgress?(.sourceOpened) // #361 + // Audit DMX-102: the provider is published BEFORE it connects, under the same lock that + // decides "already closed", so a `markClosed()` that lands during the connect (up to 15 s on + // a live source, longer for a remote disc probe) reaches the provider instead of finding + // nothing and leaving the open holding its origin slot. + closeRequestLock.lock() + let closedBeforeConnect = closeRequested + if !closedBeforeConnect { avioProvider = provider } + closeRequestLock.unlock() + if closedBeforeConnect { throw DemuxerError.openFailed(code: -1) } + do { + try provider.open() + } catch { + abandonProvider(provider) + throw error + } + if isCloseRequested { + abandonProvider(provider) + throw DemuxerError.openFailed(code: -1) + } + reportOpenStage(.sourceOpened) // #361 // AE#460 follow-up: a live source rebuilt on a RETAINED reader resumes where that reader // is, not at the host's base. A fresh AVIOContext starts its byte axis at 0 regardless, so @@ -624,10 +881,14 @@ public final class Demuxer: @unchecked Sendable { throw DemuxerError.openFailed(code: ret) } formatContext = ctxPtr // avformat_open_input may reallocate - onOpenProgress?(.containerOpened) // #361 + reportOpenStage(.containerOpened) // #361 + declareDiscSubpictureStreams(ctxPtr!) try probeStreams(ctxPtr!) - onOpenProgress?(.streamsProbed) // #361 + accessLock.lock() + refreshStreamTableLocked() + accessLock.unlock() + reportOpenStage(.streamsProbed) // #361 // #281: every parse seek this open performs has happened by now, so the provider can drop // the cold-start state that only exists to serve them. Deliberately after probeStreams: // find_stream_info is where the trailing-index ping-pong lives, not avformat_open_input. @@ -647,16 +908,67 @@ public final class Demuxer: @unchecked Sendable { private func applyProbeBudget(_ ctx: UnsafeMutablePointer) { ctx.pointee.probesize = openProfile.probesize ctx.pointee.max_analyze_duration = openProfile.maxAnalyzeDuration - if let probeControl { - ctx.pointee.interrupt_callback = AVIOInterruptCB( - callback: { opaque in - guard let opaque else { return 0 } - return Unmanaged.fromOpaque(opaque).takeUnretainedValue().isStopped ? 1 : 0 - }, - opaque: Unmanaged.passUnretained(probeControl).toOpaque()) + // Installed unconditionally: a local input's URLContext copies the callback at open, so the + // input byte budget could not reach it later. + ctx.pointee.interrupt_callback = AVIOInterruptCB( + callback: { opaque in + guard let opaque else { return 0 } + return Unmanaged.fromOpaque(opaque).takeUnretainedValue() + .shouldInterrupt() ? 1 : 0 + }, + opaque: Unmanaged.passUnretained(interrupt).toOpaque()) + } + + /// #651: the probe budget of a DVD title whose IFO declared its streams. + /// + /// MPEG-PS sets `AVFMTCTX_NOHEADER` and never clears it, so `find_stream_info` never takes its + /// "all info found" exit and reads the whole budget on every open: 50 MB, most of a first frame on + /// a remote ISO. The long window only ever existed to discover streams that start late, and on a + /// DVD those are the subpictures, which the IFO has already declared. What is left to resolve is + /// the continuous streams (MPEG-2 video's rate, the audio's parameters), which a few seconds give. + static let declaredDiscTitleProbesize: Int64 = 8 * 1024 * 1024 + static let declaredDiscTitleMaxAnalyzeDuration: Int64 = 5 * 1_000_000 + + /// #651: create the subpicture streams the selected DVD title's IFO declares, before anything is + /// read. `mpegps` looks a packet's stream up by `AVStream.id` before it creates one, so these + /// receive their packets exactly as if the demuxer had made them on the first one, and a subtitle + /// whose first packet is minutes in is a track from the first frame. Mirrors what `mpegps` sets + /// on a stream it creates, apart from `need_parsing`, which is internal: `dvdsubdec` reassembles + /// a subpicture split across packets itself. + private func declareDiscSubpictureStreams(_ ctx: UnsafeMutablePointer) { + guard let ids = discSubpictureStreamIDs, !ids.isEmpty, + let name = ctx.pointee.iformat?.pointee.name, String(cString: name) == "mpeg" else { return } + var existing = Set() + for i in 0.. 0 { + EngineLog.emit("[Demuxer] declared \(declared) DVD subpicture stream(s) from the IFO (#651)", + category: .demux) } } + /// #651: shrink the probe to `declaredDiscTitle*` once the IFO has declared the title's streams. + /// Never raises a budget: a tighter caller budget (#68) still wins. + private func boundProbeForDeclaredDiscTitle(_ ctx: UnsafeMutablePointer) { + guard discSubpictureStreamIDs != nil, + let name = ctx.pointee.iformat?.pointee.name, String(cString: name) == "mpeg" else { return } + ctx.pointee.probesize = min(ctx.pointee.probesize, Self.declaredDiscTitleProbesize) + ctx.pointee.max_analyze_duration = min(ctx.pointee.max_analyze_duration, + Self.declaredDiscTitleMaxAnalyzeDuration) + } + /// Demuxer fflags applied to every avformat_open_input. /// +genpts: libavformat regenerates missing pts/dts; per AetherEngine#4 this is /// what Jellyfin's server-side remux uses. Cuts 4K HDR HEVC RSS growth ~50% @@ -714,10 +1026,12 @@ public final class Demuxer: @unchecked Sendable { return } reclassifyAttachedPictures(ctx) + boundProbeForDeclaredDiscTitle(ctx) let parked = parkUnresolvableAudio(ctx) let findRet = avformat_find_stream_info(ctx, nil) unparkUnresolvableAudio(ctx, parked) guard findRet >= 0 else { + emitOpenTimings(outcome: "find_stream_info failed (\(findRet))") throw DemuxerError.streamInfoFailed(code: findRet) } logStreams(ctx) @@ -858,6 +1172,7 @@ public final class Demuxer: @unchecked Sendable { guard let ctx = formatContext else { return } reclassifyAttachedPictures(ctx) _ = avformat_find_stream_info(ctx, nil) + refreshStreamTableLocked(rebuildTracks: true) } /// True if the stream at `index` is missing or carries no resolved codec yet (`AV_CODEC_ID_NONE`). @@ -924,8 +1239,7 @@ public final class Demuxer: @unchecked Sendable { /// Clamped: av_find_best_stream returns AVERROR_STREAM_NOT_FOUND (-1381258232) /// on failure, not -1; normalize to -1 to avoid garbage in logs. var videoStreamIndex: Int32 { - guard let ctx = formatContext else { return -1 } - return max(-1, av_find_best_stream(ctx, AVMEDIA_TYPE_VIDEO, -1, -1, nil, 0)) + answeredFromStreams(\.videoStreamIndex) { Self.bestStreamIndex($0, AVMEDIA_TYPE_VIDEO) } } /// True if `index` names a video stream. Live producer uses this to detect @@ -942,15 +1256,51 @@ public final class Demuxer: @unchecked Sendable { /// GOTCHA: av_find_best_stream skips streams with no channels/sample_rate (live MPEG-TS /// probe may leave them that way). Use `firstAudioStreamIndexByType` as fallback. var audioStreamIndex: Int32 { - guard let ctx = formatContext else { return -1 } - return max(-1, av_find_best_stream(ctx, AVMEDIA_TYPE_AUDIO, -1, -1, nil, 0)) + answeredFromStreams(\.audioStreamIndex) { Self.bestStreamIndex($0, AVMEDIA_TYPE_AUDIO) } } /// First audio stream by codec_type regardless of codecpar completeness. /// Fallback for live MPEG-TS where av_find_best_stream skips empty-codecpar /// streams; the engine's live AAC codecpar repair fills them downstream. var firstAudioStreamIndexByType: Int32 { - guard let ctx = formatContext else { return -1 } + answeredFromStreams(\.firstAudioStreamIndexByType) { Self.firstAudioStreamIndexByType($0) } + } + + func audioTrackInfos() -> [TrackInfo] { + answeredFromStreams(\.audioTracks) { trackInfos(in: $0, ofType: AVMEDIA_TYPE_AUDIO) } + } + + func subtitleTrackInfos() -> [TrackInfo] { + answeredFromStreams(\.subtitleTracks) { trackInfos(in: $0, ofType: AVMEDIA_TYPE_SUBTITLE) } + } + + /// Runs `compute` over the live format context under `accessLock` when the lock is free, and + /// remembers the answer; answers from the last snapshot when a read holds it (audit DMX-108). + /// `try`, not `lock`, for the reason `stream(at:)` gives: a caller on the main actor must not + /// wait out a network read. + private func answeredFromStreams( + _ field: WritableKeyPath, + _ compute: (UnsafeMutablePointer) -> T + ) -> T { + if accessLock.try() { + defer { accessLock.unlock() } + let answer = formatContext.map(compute) ?? TrackSnapshot()[keyPath: field] + streamTableLock.lock() + trackSnapshot[keyPath: field] = answer + streamTableLock.unlock() + return answer + } + streamTableLock.lock() + defer { streamTableLock.unlock() } + return trackSnapshot[keyPath: field] + } + + private static func bestStreamIndex(_ ctx: UnsafeMutablePointer, + _ type: AVMediaType) -> Int32 { + max(-1, av_find_best_stream(ctx, type, -1, -1, nil, 0)) + } + + private static func firstAudioStreamIndexByType(_ ctx: UnsafeMutablePointer) -> Int32 { for i in 0.. [TrackInfo] { - guard let ctx = formatContext else { return [] } + private func trackInfos(in ctx: UnsafeMutablePointer, + ofType type: AVMediaType) -> [TrackInfo] { var tracks: [TrackInfo] = [] for i in 0.. [TrackInfo] { - guard let ctx = formatContext else { return [] } - var tracks: [TrackInfo] = [] + private static func subtitleStreamIndices(in ctx: UnsafeMutablePointer) -> Set { + var indices: Set = [] for i in 0.. Set { - guard let ctx = formatContext else { return [] } + private static func splitDisplaySetSubtitleStreamIndices( + in ctx: UnsafeMutablePointer + ) -> Set { + guard let formatName = ctx.pointee.iformat?.pointee.name, + String(cString: formatName).split(separator: ",").contains("mpegts") + else { return [] } var indices: Set = [] for i in 0.. TrackSnapshot { + guard let ctx = formatContext else { return TrackSnapshot() } + return TrackSnapshot( + videoStreamIndex: Self.bestStreamIndex(ctx, AVMEDIA_TYPE_VIDEO), + audioStreamIndex: Self.bestStreamIndex(ctx, AVMEDIA_TYPE_AUDIO), + firstAudioStreamIndexByType: Self.firstAudioStreamIndexByType(ctx), + audioTracks: trackInfos(in: ctx, ofType: AVMEDIA_TYPE_AUDIO), + subtitleTracks: trackInfos(in: ctx, ofType: AVMEDIA_TYPE_SUBTITLE), + subtitleStreamIndices: Self.subtitleStreamIndices(in: ctx), + splitDisplaySetSubtitleStreamIndices: Self.splitDisplaySetSubtitleStreamIndices(in: ctx)) + } + + /// #112: PGS subtitle streams whose display sets arrive split across PES packets and need + /// reassembly in the SubtitlePacketStore. Only the MPEG-TS demuxer splits them (Blu-ray + /// authoring: PCS|WDS|PDS|ODS|END as separate PES packets, some without a PTS); Matroska + /// carries one complete set per packet and must NOT be assembled (converters there strip + /// the trailing END, which the decoder's synthetic-END flush rescues per packet). + /// #151: every AVMEDIA_TYPE_SUBTITLE stream index; the forward prefetcher's route + keep set. + func subtitleStreamIndices() -> Set { + answeredFromStreams(\.subtitleStreamIndices) { Self.subtitleStreamIndices(in: $0) } + } + /// #230: the stream whose packets pace the subtitle side reader's forward park, or -1. /// /// The park can only be evaluated on a packet the loop actually receives, and with every @@ -1049,22 +1420,16 @@ public final class Demuxer: @unchecked Sendable { } func splitDisplaySetSubtitleStreamIndices() -> Set { - guard let ctx = formatContext, - let formatName = ctx.pointee.iformat?.pointee.name, - String(cString: formatName).split(separator: ",").contains("mpegts") - else { return [] } - var indices: Set = [] - for i in 0.. MediaMetadata { + accessLock.lock() + defer { accessLock.unlock() } guard let ctx = formatContext else { return MediaMetadata(title: nil, artist: nil, album: nil, artworkData: nil) } @@ -1204,6 +1569,7 @@ public final class Demuxer: @unchecked Sendable { // [Events] format line). Surfaced for LoadOptions.preserveASSMarkup hosts. var assHeader: String? = nil let codecID = codecpar.pointee.codec_id + let isAudio = codecpar.pointee.codec_type == AVMEDIA_TYPE_AUDIO if codecID == AV_CODEC_ID_ASS || codecID == AV_CODEC_ID_SSA, let extradata = codecpar.pointee.extradata, codecpar.pointee.extradata_size > 0 { @@ -1226,10 +1592,28 @@ public final class Demuxer: @unchecked Sendable { isHearingImpaired: isHearingImpaired, isCommentary: isCommentary, isAtmos: isAtmos, - assHeader: assHeader + assHeader: assHeader, + sampleRate: isAudio ? Int(codecpar.pointee.sample_rate) : 0, + bitsPerSample: isAudio ? Int(max(codecpar.pointee.bits_per_raw_sample, 0)) : 0, + sampleFormat: isAudio ? Self.sampleFormatName(codecpar.pointee.format) : nil, + channelLayout: isAudio ? Self.channelLayoutDescription(&codecpar.pointee.ch_layout) : nil, + profile: VideoStreamFormat.profileName(codecID: codecID, profile: codecpar.pointee.profile) ) } + static func sampleFormatName(_ raw: Int32) -> String? { + let fmt = AVSampleFormat(rawValue: raw) + guard fmt != AV_SAMPLE_FMT_NONE else { return nil } + return av_get_sample_fmt_name(fmt).map { String(cString: $0) } + } + + static func channelLayoutDescription(_ layout: UnsafePointer) -> String? { + guard layout.pointee.nb_channels > 0 else { return nil } + var buffer = [CChar](repeating: 0, count: 64) + guard av_channel_layout_describe(layout, &buffer, buffer.count) > 0 else { return nil } + return String(cString: buffer) + } + /// The language to publish for a stream. A disc keeps its track languages in its navigation data /// rather than in the streams, so an m2ts / VOB demuxed on its own reports every track as /// undetermined and no preferred-language selection can match. `discLanguages` (empty for every @@ -1254,6 +1638,8 @@ public final class Demuxer: @unchecked Sendable { /// MKV font attachments. Payload in codec extradata; filename/MIME in stream metadata. /// Non-font attachments filtered by isFontPayload. func fontAttachmentInfos() -> [FontAttachment] { + accessLock.lock() + defer { accessLock.unlock() } guard let ctx = formatContext else { return [] } var fonts: [FontAttachment] = [] for i in 0.. UnsafeMutablePointer? { - guard let ctx = formatContext, index >= 0, index < ctx.pointee.nb_streams else { + guard index >= 0 else { return nil } + // `try`, not `lock`: a read can hold `accessLock` for a whole network stall, and a caller + // on the main actor must not wait that out. The table a busy read leaves behind is current + // anyway, since `readPacketLocked` refreshes it whenever a read adds a stream. + if accessLock.try() { + refreshStreamTableLocked() + accessLock.unlock() + } + streamTableLock.lock() + defer { streamTableLock.unlock() } + return Int(index) < streamTable.count ? streamTable[Int(index)] : nil + } + + /// Caller holds `accessLock`. The track snapshot is rebuilt when the stream count changed, when + /// the context is gone, and when the caller says codec parameters moved under an unchanged count + /// (`rebuildTracks`, after a `find_stream_info`). + private func refreshStreamTableLocked(rebuildTracks: Bool = false) { + var table: [UnsafeMutablePointer] = [] + if let ctx = formatContext, let streams = ctx.pointee.streams { + let count = Int(ctx.pointee.nb_streams) + table.reserveCapacity(count) + for i in 0..(at index: Int32, _ body: (UnsafeMutablePointer) -> T) -> T? { + guard index >= 0 else { return nil } + streamTableLock.lock() + guard Int(index) < streamTable.count else { + streamTableLock.unlock() return nil } - return ctx.pointee.streams[Int(index)] + let stream = streamTable[Int(index)] + streamUsers += 1 + streamTableLock.unlock() + defer { + streamTableLock.lock() + streamUsers -= 1 + if streamUsers == 0 { streamTableLock.broadcast() } + streamTableLock.unlock() + } + return body(stream) } /// Sets AVDISCARD_ALL on streams outside `keep`. Without this, matroska reads @@ -1358,18 +1798,17 @@ public final class Demuxer: @unchecked Sendable { guard count > 0 else { return [] } let tb = stream.pointee.time_base var result: [Int64] = [] - result.reserveCapacity(Int(count)) for i in 0.. Bool = { true }) throws -> UnsafeMutablePointer? { accessLock.lock() defer { accessLock.unlock() } + if !peekedPackets.isEmpty { + guard isCurrent() else { throw CancellationError() } + return peekedPackets.removeFirst() + } + return try readPipelinePacketLocked(isCurrent: isCurrent) + } + + /// Shows the packets at the read position to `inspect`, in order, WITHOUT consuming them: what it + /// looked at is held and the next `readPacket()` calls return it first (audit HLS-103). For a source + /// that cannot rewind, where a probe that reads packets and seeks back has nothing to seek back + /// with. `inspect` returns true once it has seen enough. Stops at `maxPackets` held, at the end of + /// the source, or on a read error, which is thrown with the packets read so far still held. + func peekPackets(maxPackets: Int, + _ inspect: (UnsafeMutablePointer) -> Bool) throws { + accessLock.lock() + defer { accessLock.unlock() } + var index = 0 + while index < maxPackets { + if index == peekedPackets.count { + guard let packet = try readPipelinePacketLocked() else { return } + peekedPackets.append(packet) + } + if inspect(peekedPackets[index]) { return } + index += 1 + } + } + + /// Caller holds `accessLock`. + private func dropPeekedPacketsLocked() { + for held in peekedPackets { + var packet: UnsafeMutablePointer? = held + trackedPacketFree(&packet) + } + peekedPackets.removeAll() + } + + /// One packet through the demuxer's own pipeline, bypassing the peek queue. Caller holds `accessLock`. + private func readPipelinePacketLocked(isCurrent: @Sendable () -> Bool = { true }) throws -> UnsafeMutablePointer? { while true { // A read-ahead decision made before a seek cannot start a NEW-position read after // the seek releases this lock, then throw that first new packet away as stale. @@ -1385,16 +1862,18 @@ public final class Demuxer: @unchecked Sendable { // new read, so the container's own order survives the verdict. Checked every pass, not // once on entry: the packet that completes the sample flips the phase, and the queue // behind it has to drain before the read that follows it is emitted. - if let held = compositionRepair?.dequeue() { return held } + // Every exit is bounded again: #409 rewrites pts/dts after the funnel in + // `readDemuxedPacketLocked`, with wrapping arithmetic. + if let held = compositionRepair?.dequeue() { return boundTimestampsLocked(held) } guard let packet = try readPacketLocked() else { // EOF can arrive mid-sample on a very short source; the verdict has to be reached // now or the held packets would never be delivered. compositionRepair?.endOfStream() - if let held = compositionRepair?.dequeue() { return held } + if let held = compositionRepair?.dequeue() { return boundTimestampsLocked(held) } return nil } - guard let repair = armCompositionRepairIfNeeded() else { return packet } - if !repair.ingest(packet) { return packet } + guard let repair = armCompositionRepairIfNeeded() else { return boundTimestampsLocked(packet) } + if !repair.ingest(packet) { return boundTimestampsLocked(packet) } } } @@ -1408,6 +1887,16 @@ public final class Demuxer: @unchecked Sendable { decideCompositionRepairLocked() } + /// #409: a seek moves the read position, so the repair drops its picture-order anchor and + /// re-anchors on the landing keyframe. A seek that comes before the first read (the software + /// host's resume, #699) settles the verdict at the head first: sampled at the landing, the + /// stream would start on an open-GOP keyframe whose picture order is not 0, and the classifier + /// refuses that, so the whole session played in decode order. Caller holds `accessLock`. + private func noteCompositionRepairSeekLocked() { + if !compositionRepairEvaluated { decideCompositionRepairLocked() } + compositionRepair?.noteSeek() + } + /// #409: reads far enough into the source for the verdict, holding every /// packet it consumed so nothing is lost. Called before anything reads a timestamp axis off this /// demuxer: the container index and the packets must describe the same ladder, and only the @@ -1440,7 +1929,7 @@ public final class Demuxer: @unchecked Sendable { // reader, #104) has no pictures to sample at all, and a non-seekable source is a live feed, // where holding a dozen packets for a defect that lives in a VOD sample table is latency // spent for nothing. - guard openProfile.readerLabel != DemuxerOpenProfile.stillExtraction.readerLabel, + guard !DemuxerOpenProfile.labelsWithoutCompositionRepair.contains(openProfile.readerLabel), isSourceSeekable else { return nil } let index = max(-1, av_find_best_stream(ctx, AVMEDIA_TYPE_VIDEO, -1, -1, nil, 0)) guard index >= 0, index < Int32(ctx.pointee.nb_streams), @@ -1466,17 +1955,105 @@ public final class Demuxer: @unchecked Sendable { /// when this demuxer opened mid-file. Int64.min when the container has no index yet. private func firstIndexedTimestamp(of stream: UnsafeMutablePointer) -> Int64 { guard avformat_index_get_entries_count(stream) > 0, - let entry = avformat_index_get_entry(stream, 0) else { return Int64.min } + let entry = avformat_index_get_entry(stream, 0), + Self.isPlausibleIndexTimestamp(entry.pointee.timestamp, timeBase: stream.pointee.time_base) + else { return Int64.min } return entry.pointee.timestamp } - /// The read itself. Caller holds `accessLock`. + /// The read itself, with the fragments of a declared DVD subpicture stream joined (#651). + /// Caller holds `accessLock`. private func readPacketLocked() throws -> UnsafeMutablePointer? { + while true { + guard let read = try readDemuxedPacketLocked() else { return nil } + var packet: UnsafeMutablePointer? = read + let index = read.pointee.stream_index + guard subpictureAssemblers[index] != nil else { return read } + let timing = DVDSubpictureAssembler.Timing( + pts: read.pointee.pts, dts: read.pointee.dts, + pos: read.pointee.pos, duration: read.pointee.duration) + // In place (audit DMX-106): a copied-out assembler shares the dictionary's buffer, so + // every fragment's append copied the whole partial unit. + let unit = subpictureAssemblers[index]?.ingest( + UnsafeRawBufferPointer(start: read.pointee.data, count: Int(max(0, read.pointee.size))), + timing: timing) + guard let unit else { + trackedPacketFree(&packet) + continue + } + // The first fragment's props, then the joined payload in place of its own. + guard let joined = trackedPacketAlloc() else { trackedPacketFree(&packet); return nil } + var out: UnsafeMutablePointer? = joined + // `av_new_packet` resets every prop, so the copy comes after it. + guard let joinedSize = Int32(exactly: unit.data.count), + av_new_packet(joined, joinedSize) >= 0, + av_packet_copy_props(joined, read) >= 0 else { + trackedPacketFree(&out) + trackedPacketFree(&packet) + continue + } + unit.data.withUnsafeBytes { joined.pointee.data.update(from: $0.bindMemory(to: UInt8.self).baseAddress!, count: unit.data.count) } + joined.pointee.stream_index = index + joined.pointee.pts = unit.timing.pts + joined.pointee.dts = unit.timing.dts + joined.pointee.pos = unit.timing.pos + joined.pointee.duration = unit.timing.duration + trackedPacketFree(&packet) + return joined + } + } + + /// #651: drop half-joined subpicture units along with libavformat's own parser state. + private func resetSubpictureAssembly() { + for key in subpictureAssemblers.keys { subpictureAssemblers[key]?.reset() } + } + + /// An out-of-range source timestamp was logged once for this demuxer. Guarded by `accessLock`. + private var implausibleTimestampLogged = false + + /// Audit NAT-101 / DEC-101 / FEA-101 / SUB-105 / SEG-101: libavformat hands back whatever the + /// container wrote (matroskadec stores a uint64 cluster time into an int64 pts and clamps + /// BlockDuration to INT64_MAX, a live fMP4 tfdt is the origin's choice), and every consumer of + /// this demuxer does tick arithmetic on it. One bound here covers them all. Caller holds + /// `accessLock`. + @discardableResult + private func boundTimestampsLocked(_ packet: UnsafeMutablePointer) -> UnsafeMutablePointer { + guard let ctx = formatContext else { return packet } + let index = Int(packet.pointee.stream_index) + let timeBase = index >= 0 && index < Int(ctx.pointee.nb_streams) + ? ctx.pointee.streams[index]?.pointee.time_base : nil + let pts = packet.pointee.pts, dts = packet.pointee.dts, duration = packet.pointee.duration + guard SourceTimestampBounds.sanitize(packet, timeBase: timeBase ?? AVRational(num: 0, den: 0)), + !implausibleTimestampLogged else { return packet } + implausibleTimestampLogged = true + EngineLog.emit( + "[Demuxer] stream \(index) carries an out-of-range timestamp " + + "(pts=\(pts) dts=\(dts) duration=\(duration)), treated as unset", + category: .demux + ) + return packet + } + + /// What a failed `av_read_frame` means for a demuxer in this state: the code to throw, or nil for + /// the source's own end of file. + /// + /// Audit SEG-104: the abort of a parked read (`markClosed()`) reaches some demuxers as end of + /// file, and "the source ended" is a verdict every consumer acts on (tail adopt, bridge flush, + /// `onSequentialSourceEnded`). A closed demuxer has no verdict to give, so an EOF it reports is + /// the abort, and it says so. + static func readFailureCode(_ ret: Int32, closeRequested: Bool) -> Int32? { + guard ret == FFmpegErr.eof else { return ret } + return closeRequested ? FFmpegErr.exit : nil + } + + /// One `av_read_frame`, as libavformat delivers it. Caller holds `accessLock`. + private func readDemuxedPacketLocked() throws -> UnsafeMutablePointer? { guard let ctx = formatContext else { return nil } try probeControl?.willReadPacket() var packet: UnsafeMutablePointer? = trackedPacketAlloc() guard packet != nil else { return nil } let ret = av_read_frame(ctx, packet) + if Int(ctx.pointee.nb_streams) != streamTableSize { refreshStreamTableLocked() } do { try probeControl?.check() if ret >= 0, let packet { try probeControl?.receivedPacket(packet) } @@ -1486,12 +2063,10 @@ public final class Demuxer: @unchecked Sendable { } if ret < 0 { trackedPacketFree(&packet) - let isEOF = (ret == FFmpegErr.eof) - if isEOF { - return nil - } - throw DemuxerError.readFailed(code: ret) + guard let code = Self.readFailureCode(ret, closeRequested: isCloseRequested) else { return nil } + throw DemuxerError.readFailed(code: code) } + if let pkt = packet { boundTimestampsLocked(pkt) } // #407: before anything reads a timestamp off this packet. The PTS on these streams was // invented by `+genpts` out of decode order and transposes every B/P pair; dropping it leaves // the decoder's own reorder to place the picture. See `VFWDecodeOrderPTSRepair`. @@ -1570,16 +2145,22 @@ public final class Demuxer: @unchecked Sendable { func seek(to seconds: Double) -> Bool { accessLock.lock() defer { accessLock.unlock() } - guard let ctx = formatContext else { return false } + guard let ctx = formatContext, + let timestamp = Self.ticks(forSeconds: seconds, timeBase: Self.avTimeBase) else { return false } // #409: the read position moves, so the repair drops its picture-order anchor and // re-anchors on the next keyframe (a seek always lands on one). - compositionRepair?.noteSeek() + noteCompositionRepairSeekLocked() if let reader = timeSeekableReader { + dropPeekedPacketsLocked() guard repositionTimeSeekable(reader, toSourceSeconds: seconds, streamIndex: -1) else { return false } resetAfterTimeSeek(ctx) return true } - let timestamp = Int64(seconds * Double(AV_TIME_BASE)) + // Audit HLS-103: libavformat flushes its packet queue before it even tries a seek, so on a + // source that cannot rewind a refused seek is pure loss. A live AVIOReader reports itself + // seekable by design, so live callers keep their own rules. + guard isSourceSeekable else { return false } + dropPeekedPacketsLocked() let ret = avformat_seek_file(ctx, -1, Int64.min, timestamp, Int64.max, 0) if ret < 0 { #if DEBUG @@ -1587,6 +2168,7 @@ public final class Demuxer: @unchecked Sendable { #endif } avformat_flush(ctx) // prevents assertion failures in matroskadec.c + resetSubpictureAssembly() // #651: libavformat just dropped the parsers this stands in for lastReadClipIdx = -1 // AE#105: post-seek reads may land mid-clip; require a fresh clean crossing return ret >= 0 } @@ -1601,19 +2183,22 @@ public final class Demuxer: @unchecked Sendable { streamIndex < Int32(ctx.pointee.nb_streams) else { return false } // #409: the read position moves, so the repair drops its picture-order anchor and // re-anchors on the next keyframe (a seek always lands on one). - compositionRepair?.noteSeek() + noteCompositionRepairSeekLocked() if let reader = timeSeekableReader, let stream = ctx.pointee.streams[Int(streamIndex)] { let timeBase = stream.pointee.time_base guard timeBase.num > 0, timeBase.den > 0 else { return false } let seconds = Double(timestamp) * Double(timeBase.num) / Double(timeBase.den) + dropPeekedPacketsLocked() guard repositionTimeSeekable(reader, toSourceSeconds: seconds, streamIndex: streamIndex) else { return false } resetAfterTimeSeek(ctx) return true } + guard isSourceSeekable else { return false } // audit HLS-103, see `seek(to:)` + dropPeekedPacketsLocked() let ret = avformat_seek_file( ctx, streamIndex, @@ -1629,6 +2214,7 @@ public final class Demuxer: @unchecked Sendable { ) } avformat_flush(ctx) + resetSubpictureAssembly() // #651: libavformat just dropped the parsers this stands in for lastReadClipIdx = -1 return ret >= 0 } @@ -1640,6 +2226,7 @@ public final class Demuxer: @unchecked Sendable { pb.pointee.error = 0 } avformat_flush(ctx) + resetSubpictureAssembly() // #651: libavformat just dropped the parsers this stands in for lastReadClipIdx = -1 } @@ -1723,37 +2310,47 @@ public final class Demuxer: @unchecked Sendable { defer { accessLock.unlock() } guard let ctx = formatContext else { return false } guard probeControl?.isStopped != true else { return false } + // A stream-anchored seek carries the target in that stream's own time base; only the -1 + // form is expressed in AV_TIME_BASE units. + var anchor: Int32 = -1 + var timeBase = Self.avTimeBase + if anchorStreamIndex >= 0, anchorStreamIndex < Int32(ctx.pointee.nb_streams), + let tb = ctx.pointee.streams[Int(anchorStreamIndex)]?.pointee.time_base, + tb.num > 0, tb.den > 0 { + anchor = anchorStreamIndex + timeBase = tb + } + guard let timestamp = Self.ticks(forSeconds: seconds, timeBase: timeBase) else { return false } // #409: the read position moves, so the repair drops its picture-order anchor and // re-anchors on the next keyframe (a seek always lands on one). - compositionRepair?.noteSeek() + // A first seek's head sample runs outside `timeout` on purpose. Cut short, it either settles + // the verdict on a partial sample, and the session plays in decode order (#699), or leaves + // the seek unpositioned, and a resume that ignores the outcome then reads from the head. The + // sample has its own bounds instead: `sampleByteBudget`, `heldPacketCeiling` and the per-read + // stall timeout. + noteCompositionRepairSeekLocked() // #268: a time-seekable source repositions itself instead of paying libavformat's byte-space // binary search, which on an index-less MPEG-TS is either wedged or broken (round 10 below) and // over HTTP would additionally be a request storm. No read deadline is armed: the reader's // reposition is a bookkeeping operation, the refetch happens behind the FIFO. if let reader = timeSeekableReader { + dropPeekedPacketsLocked() guard repositionTimeSeekable(reader, toSourceSeconds: seconds, streamIndex: anchorStreamIndex) else { return false } resetAfterTimeSeek(ctx) return true } + guard isSourceSeekable else { return false } // audit HLS-103, see `seek(to:)` + dropPeekedPacketsLocked() // #112 round 9: the deadline lives on the provider protocol. Casting to AVIOReader here left a // disc-adapter source (CustomIOReaderBridge over HTTPDiscIOReader) unbounded: one positioning // seek on a remote ISO sat wedged ~230 s and every later re-arm queued behind it. avioProvider?.beginReadDeadline(secondsFromNow: timeout) defer { avioProvider?.endReadDeadline() } - // A stream-anchored seek carries the target in that stream's own time base; only the -1 - // form is expressed in AV_TIME_BASE units. - var anchor: Int32 = -1 - var timestamp = Int64(seconds * Double(AV_TIME_BASE)) - if anchorStreamIndex >= 0, anchorStreamIndex < Int32(ctx.pointee.nb_streams), - let tb = ctx.pointee.streams[Int(anchorStreamIndex)]?.pointee.time_base, - tb.num > 0, tb.den > 0 { - anchor = anchorStreamIndex - timestamp = Int64(seconds * Double(tb.den) / Double(tb.num)) - } let ret = avformat_seek_file(ctx, anchor, Int64.min, timestamp, Int64.max, 0) avformat_flush(ctx) + resetSubpictureAssembly() // #651: libavformat just dropped the parsers this stands in for lastReadClipIdx = -1 // AE#105: post-seek reads may land mid-clip; require a fresh clean crossing // matroska may return success with a partial index after abort; deadline flag // is authoritative, not ret. @@ -1870,7 +2467,15 @@ public final class Demuxer: @unchecked Sendable { ) -> Int64? { guard fileSize > 0, duration > 0, target >= 0 else { return nil } let fraction = min(1.0, max(0.0, (target - startOrigin - earlyBiasSeconds) / duration)) - return Int64(Double(fileSize) * fraction) + return clampedByteOffset(Double(fileSize) * fraction, fileSize: fileSize) + } + + /// `raw` as a byte offset in `[0, fileSize]` (audit DMX-103). The compare runs on the Double: + /// the total is whatever the origin wrote in `Content-Range`, and a `min(fileSize, ...)` after + /// `Int64(_:)` comes too late for a product that rounds up to 2^63 or past it. + nonisolated static func clampedByteOffset(_ raw: Double, fileSize: Int64) -> Int64 { + guard raw > 0 else { return 0 } + return raw >= Double(fileSize) ? fileSize : Int64(raw) } /// Landing verdict for one byte-estimate probe (#112 round 10). @@ -1895,7 +2500,7 @@ public final class Demuxer: @unchecked Sendable { let late = landed > target let farEarly = landed < target - byteEstimateAcceptEarlyWindowSeconds guard late || farEarly else { return .accept } - let corrected = min(fileSize, max(0, Int64(Double(currentByte) * (targetRel / landedRel)))) + let corrected = clampedByteOffset(Double(currentByte) * (targetRel / landedRel), fileSize: fileSize) guard corrected != currentByte else { return .accept } return .probe(corrected) } @@ -1939,9 +2544,12 @@ public final class Demuxer: @unchecked Sendable { accessLock.lock() defer { accessLock.unlock() } guard let ctx = formatContext else { return false } - compositionRepair?.noteSeek() // #409: re-anchor on the next keyframe + noteCompositionRepairSeekLocked() // #409: re-anchor on the next keyframe + guard isSourceSeekable else { return false } // audit HLS-103, see `seek(to:)` + dropPeekedPacketsLocked() let ret = avformat_seek_file(ctx, -1, Int64.min, byteTarget, Int64.max, AVSEEK_FLAG_BYTE) avformat_flush(ctx) + resetSubpictureAssembly() // #651: libavformat just dropped the parsers this stands in for lastReadClipIdx = -1 // AE#105: post-seek reads may land mid-clip; require a fresh clean crossing return ret >= 0 } @@ -2016,25 +2624,78 @@ public final class Demuxer: @unchecked Sendable { /// Fast lock-free unblock: AVIO read callback returns -1, av_read_frame returns /// at once. No resource freeing. Call before close() when cancelling a pump. func markClosed() { - avioProvider?.markClosed() + interrupt.requestClose() + closeRequestLock.lock() + closeRequested = true + let provider = avioProvider + closeRequestLock.unlock() + provider?.markClosed() + cancelOwnedSourceReader() } /// Static metadata probes only. Strong ownership outlives the native interrupt callback. - var probeControl: ProbeControl? + var probeControl: ProbeControl? { + get { interrupt.probeControl } + set { interrupt.probeControl = newValue } + } + + /// Target of the format context's interrupt callback, owned for the demuxer's whole life so the + /// unretained pointer libavformat holds cannot dangle. + private let interrupt = DemuxInterrupt() + + /// Caps the source bytes libavformat may consume until `endInputByteBudget`, enforced below + /// `av_read_frame`. A pass that sets AVDISCARD_ALL on other streams needs this: the demuxer reads + /// and drops their blocks inside one `av_read_frame`, where no packet or packet-byte cap sees them. + /// Overshoot is at most one read. Call from the thread that reads packets. + func beginInputByteBudget(_ bytes: Int64) { + accessLock.lock() + defer { accessLock.unlock() } + if let provider = avioProvider { + provider.beginReadByteBudget(bytes) + } else if let pb = formatContext?.pointee.pb { + interrupt.armInputCeiling(pb: pb, bytes: bytes) + } + } + + func endInputByteBudget() { + accessLock.lock() + defer { accessLock.unlock() } + avioProvider?.endReadByteBudget() + interrupt.disarmInputCeiling() + } + + /// True when a read was refused because the budget armed by `beginInputByteBudget` was spent. + var inputByteBudgetExhausted: Bool { + accessLock.lock() + defer { accessLock.unlock() } + return (avioProvider?.readByteBudgetExhausted ?? false) || interrupt.inputCeilingHit + } func close() { + interrupt.requestClose() avioProvider?.markClosed() // unblocks av_read_frame (tvOS suspends threads in background) accessLock.lock() + interrupt.disarmInputCeiling() + // Audit DMX-108: the table is emptied BEFORE the streams are freed, and close waits for + // every `withStream` caller already inside, so no thread is left holding a freed `AVStream*`. + streamTableLock.lock() + streamTable = [] + trackSnapshot = TrackSnapshot() + while streamUsers > 0 { streamTableLock.wait() } + streamTableLock.unlock() + dropPeekedPacketsLocked() if formatContext != nil { avformat_close_input(&formatContext) } formatContext = nil compositionRepair = nil compositionRepairEvaluated = false + refreshStreamTableLocked() accessLock.unlock() avioProvider?.close() avioProvider = nil + releaseOwnedSourceReader() } deinit { @@ -2060,3 +2721,43 @@ enum DemuxerError: Error, CustomStringConvertible, LocalizedError { var errorDescription: String? { description } } + +/// Answers libavformat's interrupt callback. `probeControl` is set before open; the input ceiling is +/// armed and read only on the thread that holds the demuxer's access lock for the native call. +private final class DemuxInterrupt: @unchecked Sendable { + var probeControl: ProbeControl? + private var pb: UnsafeMutablePointer? + private var ceiling: Int64 = .max + private(set) var inputCeilingHit = false + + /// Audit DMX-109: set by `Demuxer.markClosed()` from any thread, read by libavformat's callback + /// on the demux thread. A provider-backed input stops on its provider's own flag; a local-path + /// input has no provider, so this is the only way a close reaches a read parked on a slow volume. + private let closeRequested = OSAllocatedUnfairLock(initialState: false) + + func requestClose() { closeRequested.withLock { $0 = true } } + var isCloseRequested: Bool { closeRequested.withLock { $0 } } + + /// Local (URLContext) inputs only: a provider-backed input never consults this callback per read. + func armInputCeiling(pb: UnsafeMutablePointer, bytes: Int64) { + self.pb = pb + let (sum, overflow) = pb.pointee.bytes_read.addingReportingOverflow(max(0, bytes)) + ceiling = overflow ? .max : sum + inputCeilingHit = false + } + + func disarmInputCeiling() { + pb = nil + ceiling = .max + } + + func shouldInterrupt() -> Bool { + if isCloseRequested { return true } + if probeControl?.isStopped == true { return true } + if let pb, pb.pointee.bytes_read >= ceiling { + inputCeilingHit = true + return true + } + return false + } +} diff --git a/Sources/AetherEngine/Demuxer/HeldSourceConnection.swift b/Sources/AetherEngine/Demuxer/HeldSourceConnection.swift index b24fbf48c..87e9623c7 100644 --- a/Sources/AetherEngine/Demuxer/HeldSourceConnection.swift +++ b/Sources/AetherEngine/Demuxer/HeldSourceConnection.swift @@ -72,9 +72,11 @@ final class HeldSourceConnection: @unchecked Sendable { case malformedResponse(String) case tooManyRedirects case unsupportedURL(URL) + case invalidHeaderField(String) var errorDescription: String? { switch self { + case .invalidHeaderField(let name): return "a line break in the request field \(name)" case .malformedResponse(let detail): return "malformed HTTP response: \(detail)" case .tooManyRedirects: return "too many redirects" case .unsupportedURL(let url): return "unsupported URL for a held connection: \(url)" @@ -178,17 +180,24 @@ final class HeldSourceConnection: @unchecked Sendable { } private func openAndPump() throws { + let source = respondedBy var target = respondedBy var hops = 0 while true { if isCancelled { return } - let head = try open(target) + // Audit DMX-3: the headers were built for the source, so every hop gets the #126 + // policy every URLSession path applies: credentials only to the same origin, and + // never down from https to http. + let headers = RedirectHeaderPolicy.headersToReplay( + extraHeaders: extraHeaders, originalURL: source, redirectURL: target) + let head = try open(target, headers: headers) if let location = Self.redirectLocation(head), hops < Self.maxRedirects { guard let next = URL(string: location, relativeTo: target)?.absoluteURL else { throw ConnectionError.malformedResponse("unresolvable Location: \(location)") } hops += 1 closeSocket() + OriginRequestBudget.shared.noteRedirect(from: source, to: next) target = next continue } @@ -206,7 +215,7 @@ final class HeldSourceConnection: @unchecked Sendable { } /// Connect, write the request, and read until the head is complete. - private func open(_ target: URL) throws -> ResponseHead { + private func open(_ target: URL, headers: [String: String]) throws -> ResponseHead { guard let host = target.host, let scheme = target.scheme?.lowercased(), scheme == "http" || scheme == "https" else { throw ConnectionError.unsupportedURL(target) @@ -237,7 +246,7 @@ final class HeldSourceConnection: @unchecked Sendable { if secure { task.startSecureConnection() } try write(Self.requestBytes(target: target, host: host, port: port, secure: secure, - offset: offset, extraHeaders: extraHeaders, + offset: offset, extraHeaders: headers, userAgent: userAgent)) return try readHead() } @@ -441,11 +450,18 @@ extension HeldSourceConnection { secure: Bool, offset: Int64, extraHeaders: [String: String], - userAgent: String?) -> Data { - var path = target.path.isEmpty ? "/" : target.path - if let query = target.query, !query.isEmpty { path += "?" + query } + userAgent: String?) throws -> Data { + // Audit DMX-4: the path goes on the wire as the URL spells it. `URL.path` is decoded, so + // `%20`, `%3F` or `%2F` changed the resource asked for, a trailing slash was dropped, and a + // `%0D%0A` in a redirect target became a raw line break in the request. + let components = URLComponents(url: target, resolvingAgainstBaseURL: true) + let encodedPath = components?.percentEncodedPath ?? "" + var path = encodedPath.isEmpty ? "/" : encodedPath + if let query = components?.percentEncodedQuery, !query.isEmpty { path += "?" + query } + // An IPv6 literal is bracketed in Host, or its colons read as the port separator. + let hostName = host.contains(":") && !host.hasPrefix("[") ? "[\(host)]" : host // A non-default port belongs in Host, since the origin may route on it. - let hostHeader = (secure && port == 443) || (!secure && port == 80) ? host : "\(host):\(port)" + let hostHeader = (secure && port == 443) || (!secure && port == 80) ? hostName : "\(hostName):\(port)" var lines = ["GET \(path) HTTP/1.1", "Host: \(hostHeader)", @@ -461,6 +477,11 @@ extension HeldSourceConnection { lines.removeAll { $0.lowercased().hasPrefix(name.lowercased() + ":") } lines.append("\(name): \(value)") } + // Every field is written raw, so one carrying a line break would add lines of its own. + // `isNewline`, because CRLF is a single Swift Character that matches neither "\r" nor "\n". + if let bad = lines.first(where: { $0.contains(where: \.isNewline) }) { + throw ConnectionError.invalidHeaderField(String(bad.prefix(while: { $0 != ":" && $0 != " " }))) + } return Data((lines.joined(separator: "\r\n") + "\r\n\r\n").utf8) } } @@ -485,6 +506,16 @@ final class ChunkedBodyDecoder { private var buffer = Data() private var state: State = .size + /// Audit DMX-111: a size, terminator or trailer line longer than this is a malformed origin. A + /// line with no CRLF would otherwise be buffered and rescanned for one that is not coming. + static let maxLineBytes = 4096 + /// Framing bytes (size lines, terminators, blank lines, trailer fields) the decoder consumes + /// without one byte of body in between. Bounds the states that consume without producing: + /// endless blank lines, endless trailer fields. + static let maxFramingBytes = 64 * 1024 + private static let crlf = Data("\r\n".utf8) + private var framingBytes = 0 + var isComplete: Bool { if case .done = state { return true } return false @@ -503,7 +534,7 @@ final class ChunkedBodyDecoder { case .done: break loop case .size: - guard let line = takeLine() else { break loop } + guard let line = try takeLine() else { break loop } if line.isEmpty { continue } // tolerate the CRLF of a preceding chunk let sizeField = line.split(separator: ";", maxSplits: 1).first.map(String.init) ?? line guard let size = Int(sizeField.trimmingCharacters(in: .whitespaces), radix: 16), size >= 0 else { @@ -516,34 +547,48 @@ final class ChunkedBodyDecoder { if want == 0 { break loop } out.append(buffer.prefix(want)) buffer.removeFirst(want) + framingBytes = 0 state = remaining - want == 0 ? .dataTerminator : .data(remaining: remaining - want) case .dataTerminator: - guard let line = takeLine() else { break loop } + guard let line = try takeLine() else { break loop } guard line.isEmpty else { throw ChunkedError.expectedChunkTerminator(line) } state = .size case .trailer: - guard let line = takeLine() else { break loop } + guard let line = try takeLine() else { break loop } if line.isEmpty { state = .done } } } return out.isEmpty ? nil : out } - private func takeLine() -> String? { - guard let range = buffer.range(of: Data("\r\n".utf8)) else { return nil } + /// The next CRLF-terminated line, nil while there is not one yet. Only the first + /// `maxLineBytes + 2` bytes are searched, so a line with no end costs a bounded scan and is + /// refused instead of waited for. + private func takeLine() throws -> String? { + let window = buffer.startIndex..= Self.maxLineBytes + 2 { throw ChunkedError.lineTooLong } + return nil + } + framingBytes += buffer.distance(from: buffer.startIndex, to: range.upperBound) + if framingBytes > Self.maxFramingBytes { throw ChunkedError.framingTooLong } let line = String(decoding: buffer[.. Double? { + timeout: TimeInterval, shouldAbort: (() -> Bool)?) -> PacerWait { var pacingStarted: DispatchTime? let sleeper = DispatchSemaphore(value: 0) while true { + if shouldAbort?() == true { return .aborted } let instant = now() lock.lock() let key = headLocked(raw) @@ -306,7 +317,7 @@ final class OriginRequestBudget: @unchecked Sendable { guard let quietUntil = state.quietUntil else { origins[key] = state lock.unlock() - return pacedMilliseconds(since: pacingStarted, label: label, spent: false) + return .passed(milliseconds: pacedMilliseconds(since: pacingStarted, label: label, spent: false)) } let waitNeeded: TimeInterval @@ -318,7 +329,7 @@ final class OriginRequestBudget: @unchecked Sendable { state.tokens -= 1 origins[key] = state lock.unlock() - return pacedMilliseconds(since: pacingStarted, label: label, spent: false) + return .passed(milliseconds: pacedMilliseconds(since: pacingStarted, label: label, spent: false)) } waitNeeded = (1 - state.tokens) / Self.pacerRefillPerSecond } @@ -328,7 +339,7 @@ final class OriginRequestBudget: @unchecked Sendable { let remaining = timeout - Self.elapsedSeconds(from: started, to: DispatchTime.now()) guard remaining > 0 else { _ = pacedMilliseconds(since: pacingStarted, label: label, spent: true) - return nil + return .spent } if pacingStarted == nil { pacingStarted = DispatchTime.now() } @@ -359,14 +370,27 @@ final class OriginRequestBudget: @unchecked Sendable { /// not a correctness barrier, and blocking a read forever to honour a guess about someone /// else's rate limiter trades a slow session for a dead one. The un-granted case is counted /// and logged, because a budget that is being routinely overrun is worth seeing. - func acquire(for url: URL, label: String, timeout: TimeInterval) -> Ticket? { + /// + /// `shouldAbort` (audit DMX-112) is polled once per `pacerWaitSliceSeconds` in both waits, and a + /// caller that has been torn down while parked gets nil back with nothing consumed: no pacer + /// token, no slot, and its place in the FIFO given up, so it cannot take what the next session + /// is waiting for. nil is also what an unkeyable URL returns, so a caller that passes + /// `shouldAbort` tells the two apart by asking its own abort state. + func acquire(for url: URL, label: String, timeout: TimeInterval, + shouldAbort: (() -> Bool)? = nil) -> Ticket? { guard let raw = Self.originKey(for: url) else { return nil } let started = DispatchTime.now() // Rate before concurrency. The two rules bind different things and only this order lets a // caller wait for the rate rule without holding the concurrency one. - guard let pacedMs = waitForPacerToken(raw: raw, label: label, started: started, - timeout: timeout) else { + let pacedMs: Double + switch waitForPacerToken(raw: raw, label: label, started: started, + timeout: timeout, shouldAbort: shouldAbort) { + case .aborted: + return nil + case .passed(let milliseconds): + pacedMs = milliseconds + case .spent: // The pacer alone spent the caller's budget. Proceed uncounted-for, the same answer the // slot timeout below gives, and stay honest about being on the link. lock.lock() @@ -401,9 +425,39 @@ final class OriginRequestBudget: @unchecked Sendable { // What the pacer already spent comes off the slot's budget, so the caller's total wait is // the one figure it asked for rather than that figure per gate. let slotBudget = max(0, timeout - Self.elapsedSeconds(from: started, to: DispatchTime.now())) - let signalled = semaphore.wait(timeout: .now() + slotBudget) == .success + var signalled = false + var aborted = false + if let shouldAbort { + let deadline = DispatchTime.now() + slotBudget + while true { + let slice = min(Self.elapsedSeconds(from: DispatchTime.now(), to: deadline), + Self.pacerWaitSliceSeconds) + if semaphore.wait(timeout: .now() + slice) == .success { signalled = true; break } + if shouldAbort() { aborted = true; break } + if DispatchTime.now() >= deadline { break } + } + } else { + signalled = semaphore.wait(timeout: .now() + slotBudget) == .success + } let waitedMs = Double(DispatchTime.now().uptimeNanoseconds - started.uptimeNanoseconds) / 1_000_000 + if aborted { + lock.lock() + let keyNow = headLocked(raw) + var parked = origins[keyNow] ?? OriginState() + if let i = parked.waiters.firstIndex(where: { $0 === semaphore }) { + parked.waiters.remove(at: i) + origins[keyNow] = parked + lock.unlock() + return nil + } + lock.unlock() + // A release counted the slot in and signalled between the last slice and this lock, so + // the slot is ours already; it goes back rather than to a caller that has left. + release(Ticket(key: raw, label: label, waitedMs: waitedMs, granted: true)) + return nil + } + if signalled { // `release` counted us in before signalling, so the slot is already ours. return Ticket(key: raw, label: label, waitedMs: waitedMs, granted: true) diff --git a/Sources/AetherEngine/Demuxer/RedirectHeaderPolicy.swift b/Sources/AetherEngine/Demuxer/RedirectHeaderPolicy.swift index 185326261..d884d72b5 100644 --- a/Sources/AetherEngine/Demuxer/RedirectHeaderPolicy.swift +++ b/Sources/AetherEngine/Demuxer/RedirectHeaderPolicy.swift @@ -32,11 +32,21 @@ enum RedirectHeaderPolicy { headers.filter { !credentialHeaders.contains($0.key.lowercased()) } } + /// Audit NET-7: the headers a host handed over for `anchor` as they may be sent to `target`, a URI + /// some playlist named. A playlist can name any host and scheme, so it gets the rule a redirect + /// gets: credentials only to the same origin with no TLS downgrade, everything else as given. + static func scoped(_ headers: [String: String], grantedFor anchor: URL?, sentTo target: URL?) + -> [String: String] + { + headersToReplay(extraHeaders: headers, originalURL: anchor, redirectURL: target) + } + /// Builds the request actually handed back to URLSession on redirect: re-applies the - /// original Range (URLSession drops custom headers on cross-host redirect, and - /// Range-dependent proxies 400 without it), replays the policy-filtered extra - /// headers, and scrubs any credential header URLSession itself carried over when - /// the target is not credential-worthy. + /// original Range (Range-dependent proxies 400 without it), replays the policy-filtered + /// extra headers, and scrubs any credential header URLSession itself carried over when + /// the target is not credential-worthy. URLSession's default redirect copies every custom + /// header except `Authorization` to the new host (measured on CFNetwork 3896, audit + /// NET-108), so the scrub is what keeps a token and a cookie off a cross-origin target. static func redirectRequest( _ request: URLRequest, originalURL: URL?, @@ -63,7 +73,7 @@ enum RedirectHeaderPolicy { /// Same host and no TLS downgrade. Ports may differ only across an http -> https /// upgrade (Emby-style 8096 -> 8920); within the same scheme a port change is a /// different origin. - private static func credentialsAllowed(from original: URL?, to redirect: URL?) -> Bool { + static func credentialsAllowed(from original: URL?, to redirect: URL?) -> Bool { guard let original, let redirect, let fromHost = original.host?.lowercased(), let toHost = redirect.host?.lowercased(), @@ -82,3 +92,29 @@ enum RedirectHeaderPolicy { url.port ?? (scheme == "https" ? 443 : 80) } } + +/// Audit NET-109: which URLs may receive the headers a host handed over. Being allowed to fetch a +/// URL (a playlist named it, a redirect landed there) and being granted the host's credentials are +/// separate decisions: only the URLs the host itself passed in are anchors, and every other target +/// gets the same headers minus the credentials, the rule a redirect follows. One type so the relay, +/// the subtitle proxy, the audio tap and the live rendition fetch cannot each decide differently. +struct CredentialScope: Sendable { + let headers: [String: String] + let anchors: [URL] + + init(headers: [String: String], anchors: [URL]) { + self.headers = headers + self.anchors = anchors + } + + init(headers: [String: String], anchor: URL?) { + self.init(headers: headers, anchors: anchor.map { [$0] } ?? []) + } + + func headers(for target: URL) -> [String: String] { + if anchors.contains(where: { RedirectHeaderPolicy.credentialsAllowed(from: $0, to: target) }) { + return headers + } + return RedirectHeaderPolicy.scoped(headers, grantedFor: nil, sentTo: target) + } +} diff --git a/Sources/AetherEngine/Demuxer/SourcePrewarmPlan.swift b/Sources/AetherEngine/Demuxer/SourcePrewarmPlan.swift index 45c4852f5..323f5c9b9 100644 --- a/Sources/AetherEngine/Demuxer/SourcePrewarmPlan.swift +++ b/Sources/AetherEngine/Demuxer/SourcePrewarmPlan.swift @@ -141,8 +141,10 @@ enum SourcePrewarmPlan { let end = min(head.count, start + Int(clamping: size)) // A SeekHead truncated by the warm head's edge still yields the entries that fit. for position in seekPositions(head, from: start, to: end) { - let absolute = Int64(dataStart) + position - guard absolute >= headEnd else { continue } + // Audit DMX-2: an 8-byte SeekPosition reaches Int64.max; such an entry names + // nothing a source can hold, so it is skipped rather than added. + let (absolute, overflow) = Int64(dataStart).addingReportingOverflow(position) + guard !overflow, absolute >= headEnd else { continue } earliest = min(earliest ?? absolute, absolute) } } diff --git a/Sources/AetherEngine/Diagnostics/EngineLog.swift b/Sources/AetherEngine/Diagnostics/EngineLog.swift index 2d39341fa..9f24628af 100644 --- a/Sources/AetherEngine/Diagnostics/EngineLog.swift +++ b/Sources/AetherEngine/Diagnostics/EngineLog.swift @@ -28,6 +28,31 @@ public enum EngineLog { private static let handlerLock = NSLock() nonisolated(unsafe) private static var _handler: ((String) -> Void)? + /// Names a value that must never appear in a log line, such as an IPTV account password the host + /// holds. Every line then has that exact value (and its percent-encoded form) replaced before it + /// reaches OSLog or `handler`, wherever it sits. The engine already strips the credential shapes it + /// can recognise (named parameters, userinfo, encoded payloads, the Xtream Codes path layout); this + /// is for the ones only the host knows, such as a provider URL that carries the password as a bare + /// path segment. Returns false, and registers nothing, for a value shorter than four bytes, which + /// would black out ordinary text. Thread-safe. + @discardableResult + public static func registerSecret(_ value: String) -> Bool { + LogRedaction.register(value) + } + + /// Stops redacting a value passed to `registerSecret(_:)`, for example on logout. Registrations are + /// counted, so a value two owners registered stays redacted until both have unregistered it. + public static func unregisterSecret(_ value: String) { + LogRedaction.unregister(value) + } + + /// The line as `emit` would deliver it, with credentials stripped. For a tool or a host that prints + /// a line of its own next to the engine's, such as aetherctl's banners, so the line it composes does + /// not carry the token the engine's lines hide (audit OPS-106). + public static func redacted(_ line: String) -> String { + LogRedaction.redact(line) + } + public static let subsystem: String = "de.superuser404.AetherEngine" private static let loggers: [Category: Logger] = { @@ -76,3 +101,16 @@ public enum EngineLog { } } } + +extension EngineLog { + /// Audit NET-114: `"\(error)"` of a URLError prints its userInfo, which is the failing URL twice + /// with its whole query. The code and the host say what failed without handing the log the URL. + /// Any other error keeps its own description. + static func summary(of error: Error) -> String { + let bridged = error as NSError + guard bridged.domain == NSURLErrorDomain else { return "\(error)" } + let failing = (bridged.userInfo[NSURLErrorFailingURLErrorKey] as? URL) + ?? (bridged.userInfo[NSURLErrorFailingURLStringErrorKey] as? String).flatMap(URL.init(string:)) + return "NSURLError \(bridged.code)" + (failing?.host.map { " from \($0)" } ?? "") + } +} diff --git a/Sources/AetherEngine/Diagnostics/LiveTelemetry.swift b/Sources/AetherEngine/Diagnostics/LiveTelemetry.swift index a075d5783..8c5b89554 100644 --- a/Sources/AetherEngine/Diagnostics/LiveTelemetry.swift +++ b/Sources/AetherEngine/Diagnostics/LiveTelemetry.swift @@ -9,12 +9,20 @@ import Foundation /// interpret a sub-second decoded queue as the size of the compressed packet cache. public struct LiveTelemetry: Equatable, Sendable { // Enthusiast section + /// Rate of the media the playhead crossed over about the last 10 s of playback: the bytes of the + /// played video and audio packets presented in that span, over the media seconds it covered + /// (AE#514). Not the transfer, which is `networkThroughputMbps`: a read-ahead, a re-fetch after a + /// seek and a live source draining into its DVR window while paused all move bytes that nobody + /// played. Stands still through a pause, on live too. nil until the playhead has crossed a span the + /// session has bytes for. The remote-HLS bypass has no demuxer, so on that route this is the playing + /// variant's declared BANDWIDTH. public let instantBitrateMbps: Double? - /// Lifetime mean rate of the session, over the seconds it spent consuming media rather than over - /// wall-clock seconds since it started (AE#514). A pause therefore leaves this value standing - /// still instead of dragging it toward zero for as long as the pause lasts, and so does the tail - /// after end-of-media. nil until the session has both accrued active time and fetched something: - /// like `networkThroughputMbps`, a value that cannot be measured yet is a gap, never a zero. + /// Lifetime mean of the same quotient: every played byte over every media second played this + /// session (AE#514). A seek charges nothing for the span it jumped, a pause and the tail after + /// end-of-media charge nothing at all, and the prefetch counts once it is played, not when it + /// arrives. On the remote-HLS bypass it is the variant's declared AVERAGE-BANDWIDTH, or BANDWIDTH + /// where the master omits it. nil until something was played: like `networkThroughputMbps`, a value + /// that cannot be measured yet is a gap, never a zero. public let averageBitrateMbps: Double? /// Live bitrate of the audio bridge's encoded output, or nil when no bridge is active (stream-copy / /// AVPlayer-native path) or before the first delta. Measured from the bridge's cumulative output-byte diff --git a/Sources/AetherEngine/Diagnostics/LiveTelemetrySampler.swift b/Sources/AetherEngine/Diagnostics/LiveTelemetrySampler.swift index 45a121789..104f8e06b 100644 --- a/Sources/AetherEngine/Diagnostics/LiveTelemetrySampler.swift +++ b/Sources/AetherEngine/Diagnostics/LiveTelemetrySampler.swift @@ -66,6 +66,10 @@ struct NativeAVFReadings: Sendable { var contiguousForwardBufferSeconds: Double? = nil /// Sum over all access-log events, for the [LagDiag] tick-over-tick drop delta. var droppedFramesLifetimeSum: Int = 0 + /// The current variant's declared BANDWIDTH and AVERAGE-BANDWIDTH, in bits per second, from the + /// latest access-log entry. The log reports an undeclared value as negative. + var indicatedBitrate: Double = -1 + var indicatedAverageBitrate: Double = -1 var currentTimeSeconds: Double = .nan var timeControlStatus: AVPlayer.TimeControlStatus = .paused var rate: Float = 0 @@ -114,16 +118,16 @@ final class LiveTelemetrySampler { private var lastDemuxerBytes: Int64 = 0 private var lastBridgeBytes: Int64 = 0 private var lastFramesEnqueued: Int = 0 - private var sessionStartBytes: Int64 = 0 - /// AE#514: wall-clock seconds this session spent in a phase that consumes media, accumulated one - /// tick at a time. The divisor of the lifetime average, in place of the wall clock since start. - /// Readable so a test can pin that the tick charges it, not only that the fold over it is right. - private(set) var activeSeconds: Double = 0 + /// AE#514: both bitrate fields, metered over what the playhead crossed. Readable so a test can pin + /// that the tick feeds it, not only that the meter is right. + private(set) var playedMeter = PlayedBitrateMeter() - /// Timestamp of the previous tick, anchoring the delta charged above. nil until the first tick, - /// which therefore charges nothing: that is also the tick which seeds `sessionStartBytes`, so the - /// numerator and the divisor start counting at the same instant. + /// The ledger the meter was last advanced against. A different one is a different session, whose + /// playhead and bytes have nothing to do with the old meter's. + private var meterLedger: ObjectIdentifier? + + /// Timestamp of the previous tick: the wall time the playhead had to cover its step in. private var lastTickTime: Date? /// [LagDiag] tick-over-tick state (#93 post-recovery lag diagnosis). @@ -156,9 +160,9 @@ final class LiveTelemetrySampler { lastDemuxerBytes = engine?.demuxerBytesFetched ?? 0 lastBridgeBytes = engine?.audioBridgeOutputBytesLifetime ?? 0 lastFramesEnqueued = 0 - activeSeconds = 0 + playedMeter = PlayedBitrateMeter() + meterLedger = nil lastTickTime = nil - sessionStartBytes = 0 lagLastClock = nil lagLastDroppedSum = 0 eomParkFrozenTicks = 0 @@ -196,76 +200,79 @@ final class LiveTelemetrySampler { return Double(windowBytes) * 8.0 / Double(activeSeconds) / 1_000_000.0 } - /// AE#514: whether a tick's second belongs in the lifetime average's divisor. - /// - /// That average used to divide by wall-clock time since the session started, which made a pause - /// permanently wrong in one direction. `demuxerBytesFetched` stops advancing once the forward - /// buffer is full, the wall clock does not, so a 2.8 Mbps file left paused for three minutes - /// reported 0.4 Mbps and afterwards climbed back only asymptotically: the paused seconds never - /// left the divisor again. Same shape at the end of a source, where the sampler keeps ticking - /// until the host tears the session down. - /// - /// The line is drawn at "is the session consuming media", not at "is the picture moving". A seek - /// and a rebuffer are where the bytes arrive hardest, and a stalled reader is a real part of what - /// this session averaged, so charging their seconds is what keeps the quotient equal to the rate - /// the session pulled at. Only a pause and the three phases with no live session behind them - /// stand outside it. - static func chargesActiveTime(_ phase: PlaybackPhase) -> Bool { - switch phase { - case .paused, .idle, .ended, .error: - return false - case .loading, .playing, .seeking, .rebuffering, .stalled: - return true - } + /// Which cumulative counter the two bitrate fields are metered from. + enum BitrateCounter: Equatable { + /// Bytes the engine's demuxer pulled from the source (loopback and software). + case demuxer + /// The bitrate the playing variant declares, from AVPlayer's access log. The remote-HLS bypass has no + /// demuxer, and what AVPlayer transferred is not the stream's rate: a buffer filling after a start or + /// a seek pulls at link speed (22.6 Mbps measured for a 3.7 Mbps transcode). The transfer stays in the + /// network fields. + case declaredVariant } - /// Lifetime mean of the bytes the session fetched over the seconds it spent consuming them (AE#514). - /// - /// nil rather than zero until both halves are measurable, mirroring `observedTransferMbps`: a host - /// cannot tell a confident 0.00 Mbps from a session that has not fetched anything yet. The old - /// wall-clock form published exactly that on its first tick, where the lifetime delta is zero by - /// construction because the same tick seeds the baseline it then subtracts. - static func averageBitrateMbps(lifetimeBytes: Int64, activeSeconds: Double) -> Double? { - guard activeSeconds > 0, lifetimeBytes > 0 else { return nil } - return Double(lifetimeBytes) * 8.0 / activeSeconds / 1_000_000.0 + nonisolated static func bitrateCounter(for route: VideoRoute) -> BitrateCounter { + route == .remoteBypass ? .declaredVariant : .demuxer + } + + /// Both bitrate fields from the access log's declarations, in Mbps: instant is BANDWIDTH, average is + /// AVERAGE-BANDWIDTH, falling back to BANDWIDTH where the master omits it. Undeclared reads nil. + nonisolated static func declaredBitrates(indicated: Double, indicatedAverage: Double) + -> (instant: Double?, average: Double?) { + func mbps(_ bps: Double) -> Double? { bps.isFinite && bps > 0 ? bps / 1_000_000.0 : nil } + let instant = mbps(indicated) + return (instant, mbps(indicatedAverage) ?? instant) + } + + /// Whether the native branch has the loopback pipeline behind it. On the bypass the producer's A/V gap + /// does not exist (its getter reads 0, which would publish as a measured zero), and the #169 tail-park + /// synthesis stays with the route whose EXTINF overshoot it was built for. + nonisolated static func readsLoopbackPipeline(_ route: VideoRoute) -> Bool { + route != .remoteBypass + } + + /// AE#514: where the playhead stands on the axis the played-media ledger is keyed on, the source + /// axis the pumps read packets on. Native folds AVPlayer's clock back with the producer's shift + /// (`source_pts - playlistShiftSeconds`) rather than reading `sourceTime`, which publishes the item + /// axis on a sequential origin (#368). nil where there is no ledger-fed pipeline to read it off. + nonisolated static func ledgerPlayhead(backend: PlaybackBackend, nativeClock: Double?, playlistShift: Double, + softwareSourceClock: Double?) -> Double? { + switch backend { + case .native: + guard let nativeClock, nativeClock.isFinite else { return nil } + return nativeClock + playlistShift + case .software: + return softwareSourceClock + case .aether, .none, .audio: + return nil + } } private func tick() async { guard let engine = engine else { return } - // AE#514: this tick's wall-clock second goes into the lifetime average's divisor only if the - // session was consuming media in it. See `chargesActiveTime` for where the line runs. let tickTime = Date() - if let previous = lastTickTime, Self.chargesActiveTime(engine.playbackPhase) { - activeSeconds += tickTime.timeIntervalSince(previous) - } + let wallSeconds = lastTickTime.map { tickTime.timeIntervalSince($0) } ?? 0 lastTickTime = tickTime - // Instant + average bitrate from demuxer byte counters (both native and SW paths) + let route = engine.videoRoute + + // What the reader pulled from the source. Transfer, not media rate: it feeds the network fields + // and [LagDiag], never the bitrate fields (AE#514, see `playedMeter`). let demuxerBytes = engine.demuxerBytesFetched let bytesThisTick = max(0, demuxerBytes - lastDemuxerBytes) lastDemuxerBytes = demuxerBytes - if sessionStartBytes == 0 { sessionStartBytes = demuxerBytes } byteWindow.push(bytesThisTick) - let instantBitrateMbps: Double? - if byteWindow.count >= 2 { - let totalBytes = byteWindow.sum - let seconds = Double(byteWindow.count) - instantBitrateMbps = Double(totalBytes) * 8.0 / seconds / 1_000_000.0 - } else { - instantBitrateMbps = nil - } + let transferMbps: Double? = byteWindow.count >= 2 + ? Double(byteWindow.sum) * 8.0 / Double(byteWindow.count) / 1_000_000.0 + : nil let observedTransferMbps = Self.observedTransferMbps( windowBytes: byteWindow.sum, activeSeconds: byteWindow.activeCount, samples: byteWindow.count) - let averageBitrateMbps = Self.averageBitrateMbps( - lifetimeBytes: max(0, demuxerBytes - sessionStartBytes), - activeSeconds: activeSeconds) - // Live audio-bridge output bitrate from the bridge's cumulative encoded-byte counter. 0 on the // stream-copy / AVPlayer-native / video-only paths (no bridge), which surfaces as nil. let bridgeBytes = engine.audioBridgeOutputBytesLifetime @@ -298,7 +305,8 @@ final class LiveTelemetrySampler { observedFps = nil displayCushionSeconds = nil accumulatedFrameDelaySeconds = nil - avSyncGapMs = engine.lastAVGapMs // HLSSegmentProducer audio-gate-open vs video-gate-open (native path only) + // HLSSegmentProducer audio-gate-open vs video-gate-open, loopback only. + avSyncGapMs = Self.readsLoopbackPipeline(route) ? engine.lastAVGapMs : nil if let player = engine.currentAVPlayer, let item = player.currentItem { var readings = await readNativeOffMain(player: player, item: item) // stop() may have cancelled this tick, or a reload seam may have swapped the @@ -382,13 +390,45 @@ final class LiveTelemetrySampler { accumulatedFrameDelaySeconds = nil } + // AE#514: the bitrate fields are the media the playhead crossed, never the transfer. + var instantBitrateMbps: Double? + var averageBitrateMbps: Double? + if let ledger = engine.playedMediaLedger { + let identity = ObjectIdentifier(ledger) + if meterLedger != identity { + meterLedger = identity + playedMeter = PlayedBitrateMeter() + } + playedMeter.advance( + to: Self.ledgerPlayhead( + backend: engine.playbackBackend, + nativeClock: nativeReadings?.currentTimeSeconds, + playlistShift: engine.playlistShiftSeconds, + softwareSourceClock: engine.softwareHost?.sourceClockSeconds), + wallSeconds: wallSeconds, + ledger: ledger) + instantBitrateMbps = playedMeter.instantMbps + averageBitrateMbps = playedMeter.averageMbps + } + + // Remote-HLS bypass: no demuxer, so both rates are what the playing variant declares. + if Self.bitrateCounter(for: route) == .declaredVariant { + let declared = Self.declaredBitrates( + indicated: nativeReadings?.indicatedBitrate ?? -1, + indicatedAverage: nativeReadings?.indicatedAverageBitrate ?? -1) + instantBitrateMbps = declared.instant + averageBitrateMbps = declared.average + } + // Feed the extractor yield gate (#93 startup): nil on non-native paths keeps the // gate conservative there, but those paths have no active session to gate anyway. engine.extractorYieldState.setForwardBuffer(forwardBufferSeconds) if let readings = nativeReadings { - emitLagDiag(engine: engine, readings: readings, netMbps: instantBitrateMbps) - evaluateEndOfMediaPark(engine: engine, readings: readings) + emitLagDiag(engine: engine, readings: readings, netMbps: transferMbps) + if Self.readsLoopbackPipeline(route) { + evaluateEndOfMediaPark(engine: engine, readings: readings) + } } let softwareCache = engine.softwarePacketCacheSnapshot @@ -554,6 +594,8 @@ final class LiveTelemetrySampler { let observed = event.observedBitrate readings.networkThroughputMbps = observed.isFinite && observed > 0 ? observed / 1_000_000.0 : nil + readings.indicatedBitrate = event.indicatedBitrate + readings.indicatedAverageBitrate = event.indicatedAverageBitrate } // AE#443: the counters are not. They are totals PER ENTRY, and AVFoundation opens a new entry // whenever the playback session changes under it, so reading `.last` publishes a number that diff --git a/Sources/AetherEngine/Diagnostics/LogRedaction.swift b/Sources/AetherEngine/Diagnostics/LogRedaction.swift index d967f7086..469d491d0 100644 --- a/Sources/AetherEngine/Diagnostics/LogRedaction.swift +++ b/Sources/AetherEngine/Diagnostics/LogRedaction.swift @@ -13,6 +13,12 @@ import Foundation /// nothing in the URL names it. The list of names cannot be extended to reach that one, because the /// names live inside the payload, so the encoding itself has to be the signal (`encodedPayloadRange`). /// +/// IPTV panels speaking the Xtream Codes API put the account's password in the path with no name and +/// no encoding at all (`/live/{user}/{password}/{id}.ts`), so the layout is the only signal there +/// (`xtreamPathSecretRange`). Its short form (`/{user}/{password}/{id}`) has no layout to go by, and +/// neither does a provider shape nobody has reported yet; for those the host names the value itself +/// through `EngineLog.registerSecret(_:)`, which is matched literally (`registeredSecretRange`). +/// /// That makes it the engine's problem rather than each host's: the engine composes the line, it reaches /// three sinks a host does not control, and a host-side scrub only ever covers the one sink it owns. /// @@ -27,10 +33,14 @@ enum LogRedaction { /// Generic credential parameter and header names; the engine is not tied to one server product. /// Held as lowercase ASCII bytes and matched longest first, so `x-mediabrowser-token` wins over its /// `token` suffix. `token` alone is deliberately broad and only fires on a boundary, so identifiers - /// such as `hasToken` and `refreshTokenAt` are left alone. + /// such as `hasToken` and `refreshTokenAt` are left alone. Audit SUB-109 added the names other + /// backends use as whole keys (`api-key` covers `X-Api-Key` through the `-` boundary, `cookie` covers + /// `Set-Cookie`); bare `pass`, `key`, `auth` and `sid` stay out, because prose carries them. private static let keys: [[UInt8]] = [ - "x-mediabrowser-token", "x-emby-token", "access_token", "accesstoken", "connect.sid", - "signature", "password", "api_key", "apikey", "secret", "token", + "x-mediabrowser-token", "x-emby-token", "access_token", "accesstoken", "refreshtoken", + "sessiontoken", "auth_token", "authtoken", "connect.sid", "session_id", "sessionid", + "signature", "password", "api_key", "api-key", "apikey", "passwd", "secret", "cookie", + "token", "pwd", ].map { Array($0.utf8) } private static let placeholderBytes = Array(placeholder.utf8) @@ -42,19 +52,66 @@ enum LogRedaction { /// segment producer, so anything per-line here is per-line everywhere. static func redact(_ line: String) -> String { let bytes = Array(line.utf8) + let secrets = registeredSecrets + guard let view = DecodedView(bytes) else { return streamingRedact(line, bytes, secrets) } + + // Audit SUB-104: a URL carried percent-encoded inside another URL's query hides every shape + // that needs no key (`%2F` is not a `/`, `%40` is not an `@`, `%2F` ends in a base64 letter). + // The nameless matchers therefore also run over the decoded view, and each hit maps back to + // whole escapes. The key matcher stays raw-only: its depth-aware terminators are the NET-1 + // rule, and the view is produced once and never decoded again. + var spans: [Range] = [] + var i = 0 + while i < bytes.count { + guard let value = match(in: bytes, at: i, secrets: secrets, keys: true) else { + i += 1 + continue + } + spans.append(value) + i = value.upperBound + } + let decoded = view.decoded + var j = 0 + while j < decoded.count { + guard let value = match(in: decoded, at: j, secrets: secrets, keys: false) else { + j += 1 + continue + } + spans.append(view.rawStart[value.lowerBound] ..< view.rawStart[value.upperBound]) + j = value.upperBound + } + guard !spans.isEmpty else { return line } + + spans.sort { $0.lowerBound < $1.lowerBound } + var out: [UInt8] = [] + out.reserveCapacity(bytes.count) + var copiedUpTo = 0 + var current = spans[0] + for span in spans.dropFirst() { + if span.lowerBound <= current.upperBound { + current = current.lowerBound ..< max(current.upperBound, span.upperBound) + continue + } + out.append(contentsOf: bytes[copiedUpTo ..< current.lowerBound]) + out.append(contentsOf: placeholderBytes) + copiedUpTo = current.upperBound + current = span + } + out.append(contentsOf: bytes[copiedUpTo ..< current.lowerBound]) + out.append(contentsOf: placeholderBytes) + out.append(contentsOf: bytes[current.upperBound...]) + return String(decoding: out, as: UTF8.self) + } + + /// A line without a percent escape, which is nearly every line: one pass, and the output is + /// allocated only once something matches. + private static func streamingRedact(_ line: String, _ bytes: [UInt8], _ secrets: [[UInt8]]) -> String { var out: [UInt8]? var copiedUpTo = 0 var i = 0 while i < bytes.count { - // Three shapes, because a credential does not always arrive next to a name. The key - // matcher covers `api_key=…`, `X-Emby-Token: …` and the cookie; the payload matcher covers - // an encoded blob that no name points at, which is how a path segment carries one; the - // userinfo matcher covers `smb://user:secret@host`, where it sits in the authority. - guard let value = matchedKeyLength(in: bytes, at: i) - .flatMap({ valueRange(in: bytes, after: i + $0) }) - ?? encodedPayloadRange(in: bytes, at: i) - ?? userInfoSecretRange(in: bytes, at: i) else { + guard let value = match(in: bytes, at: i, secrets: secrets, keys: true) else { i += 1 continue } @@ -73,58 +130,240 @@ enum LogRedaction { return String(decoding: out, as: UTF8.self) } - /// Length of the key starting here, or nil. The key must start on a boundary, else `token` would - /// fire inside `hasToken`. A separator such as the `-` in `X-Emby-Token` or the `_` in `api_key` is - /// a boundary; an ASCII letter or digit is not. + /// Several shapes, because a credential does not always arrive next to a name. The key matcher + /// covers `api_key=…`, `X-Emby-Token: …` and the cookie; the scheme matcher covers + /// `Authorization: Bearer …` / `Basic …`; the payload matcher covers an encoded blob that no name + /// points at, which is how a path segment carries one; the userinfo matcher covers + /// `smb://user:secret@host`, where it sits in the authority; the path matcher covers the Xtream + /// layout; a registered secret is found wherever it sits. + @inline(__always) + private static func match(in bytes: [UInt8], at index: Int, secrets: [[UInt8]], keys: Bool) + -> Range? + { + let keyed = keys ? matchedKeyLength(in: bytes, at: index) + .flatMap({ valueRange(in: bytes, keyStart: index, keyEnd: index + $0) }) : nil + guard var span = keyed + ?? authorizationSchemeRange(in: bytes, at: index) + ?? encodedPayloadRange(in: bytes, at: index) + ?? userInfoSecretRange(in: bytes, at: index) + ?? xtreamPathSecretRange(in: bytes, at: index) else { + return registeredSecretRange(in: bytes, at: index, secrets: secrets) + } + // The scan resumes past the span, so a registered secret starting before its end goes unseen + // unless it is folded in, and returning that secret alone hides the value instead (`token` + // registered, `token=abc` logged). One span covers both. + var k = index + while !secrets.isEmpty, k < span.upperBound { + if let secret = registeredSecretRange(in: bytes, at: k, secrets: secrets) { + span = min(span.lowerBound, k) ..< max(span.upperBound, secret.upperBound) + } + k += 1 + } + return span + } + + /// Every logical character of a line once, escapes followed through `%25` layers by + /// `logicalByte`, with the raw offset each one starts at, so a range found in `decoded` maps back + /// to whole escapes. Nil for a line without a valid escape, which keeps the streaming pass. + private struct DecodedView { + let decoded: [UInt8] + /// `decoded.count + 1` entries; the last is the raw length. + let rawStart: [Int] + + init?(_ bytes: [UInt8]) { + guard bytes.withUnsafeBufferPointer({ memchr($0.baseAddress, 0x25, $0.count) }) != nil + else { return nil } + var k = 0 + var found = false + while k + 2 < bytes.count { + if bytes[k] == LogRedaction.percent, LogRedaction.hexValue(bytes[k + 1]) != nil, + LogRedaction.hexValue(bytes[k + 2]) != nil { + found = true + break + } + k += 1 + } + guard found else { return nil } + var decoded: [UInt8] = [] + decoded.reserveCapacity(bytes.count) + var rawStart: [Int] = [] + rawStart.reserveCapacity(bytes.count + 1) + var i = 0 + while i < bytes.count { + let char = LogRedaction.logicalByte(in: bytes, at: i) + decoded.append(char.byte) + rawStart.append(i) + i += char.width + } + rawStart.append(bytes.count) + self.decoded = decoded + self.rawStart = rawStart + } + } + + /// Raw length of the key starting here, or nil. The key must start on a boundary, else `token` + /// would fire inside `hasToken`. A separator such as the `-` in `X-Emby-Token` or the `_` in + /// `api_key` is a boundary; an ASCII letter or digit is not, unless it closes a percent escape + /// whose decoded character is a separator. + /// + /// Audit NET-1: the key is read through `logicalByte`, so a URL logged inside another URL's query + /// (`%26api%5Fkey%3D…`, or `%2526api%255Fkey%253D…` encoded twice) is matched like the plain form. private static func matchedKeyLength(in bytes: [UInt8], at index: Int) -> Int? { - if index > 0, isLetterOrDigit(bytes[index - 1]) { return nil } - for key in keys where index + key.count <= bytes.count { + let first = logicalByte(in: bytes, at: index) + guard keyInitials[Int(lowercased(first.byte))] else { return nil } + if precededByWordCharacter(bytes, at: index) { return nil } + for key in keys { + var j = index var matched = true - for offset in 0 ..< key.count where lowercased(bytes[index + offset]) != key[offset] { - matched = false - break + for keyByte in key { + guard j < bytes.count else { matched = false; break } + let char = logicalByte(in: bytes, at: j) + guard lowercased(char.byte) == keyByte else { matched = false; break } + j += char.width } - if matched { return key.count } + if matched { return j - index } } return nil } + /// A table rather than a `Set`: this is asked at every position of every line, and hashing the + /// byte cost more than the rest of an escape-free line's pass put together (measured). + private static let keyInitials: [Bool] = { + var table = [Bool](repeating: false, count: 256) + for key in keys { table[Int(key[0])] = true } + return table + }() + /// The span holding the secret, given the index just past the key. Covers the query form /// (`api_key=abc&next=1`), both header forms (`Token="abc"`, `X-Emby-Token: abc`) and the cookie /// form (`connect.sid=abc; Path=/`). Nil when there is no assignment or the value is empty, so /// `api_key=` and a bare mention in prose are left alone. - private static func valueRange(in bytes: [UInt8], after keyEnd: Int) -> Range? { + /// + /// Characters are read through `logicalByte`. A terminator ends the value only when it sits under + /// no more encoding layers than the `=` did: inside a plain query `%26` is part of the value, inside + /// an encoded one it is the `&` that ends it. With no escape in sight this is the byte scan it was. + private static func valueRange(in bytes: [UInt8], keyStart: Int, keyEnd: Int) -> Range? { var i = keyEnd - while i < bytes.count, bytes[i] == UInt8(ascii: " ") { i += 1 } - guard i < bytes.count, bytes[i] == UInt8(ascii: "=") || bytes[i] == UInt8(ascii: ":") else { + // Audit SUB-109: a JSON member or a dictionary description closes the key's quote before its + // separator (`{"api_key":"…"}`, `["X-Emby-Token": "…"]`). + if keyStart > 0, bytes[keyStart - 1] == UInt8(ascii: "\"") || bytes[keyStart - 1] == UInt8(ascii: "'"), + i < bytes.count, bytes[i] == bytes[keyStart - 1] { + i += 1 + } + while i < bytes.count, case let char = logicalByte(in: bytes, at: i), char.byte == space { + i += char.width + } + guard i < bytes.count else { return nil } + let separator = logicalByte(in: bytes, at: i) + guard separator.byte == UInt8(ascii: "=") || separator.byte == UInt8(ascii: ":") else { return nil } - let isHeaderSeparator = bytes[i] == UInt8(ascii: ":") - i += 1 + let isHeaderSeparator = separator.byte == UInt8(ascii: ":") + let depth = separator.depth + i += separator.width // Only a header separator may be followed by spaces. After `=` the value starts immediately: // a URL query and a cookie never space it out, and skipping here would let prose such as // "api_key= (missing)" read as a credential and swallow the rest of the line. var afterSpaces = i - while afterSpaces < bytes.count, bytes[afterSpaces] == UInt8(ascii: " ") { afterSpaces += 1 } + while afterSpaces < bytes.count, case let char = logicalByte(in: bytes, at: afterSpaces), + char.byte == space { + afterSpaces += char.width + } var quote: UInt8? - if afterSpaces < bytes.count, - bytes[afterSpaces] == UInt8(ascii: "\"") || bytes[afterSpaces] == UInt8(ascii: "'") { - quote = bytes[afterSpaces] - i = afterSpaces + 1 + if afterSpaces < bytes.count, case let char = logicalByte(in: bytes, at: afterSpaces), + char.depth <= depth, char.byte == UInt8(ascii: "\"") || char.byte == UInt8(ascii: "'") { + quote = char.byte + i = afterSpaces + char.width } else if isHeaderSeparator { i = afterSpaces } let start = i - if let quote { - while i < bytes.count, bytes[i] != quote { i += 1 } - } else { - while i < bytes.count, !isValueTerminator(bytes[i]) { i += 1 } + while i < bytes.count { + let char = logicalByte(in: bytes, at: i) + if char.depth <= depth { + if let quote, char.byte == quote { break } + if quote == nil, isValueTerminator(char.byte), + endsValue(char, at: i, in: bytes, afterHeaderSeparator: isHeaderSeparator) { + break + } + } + i += char.width } return start < i ? start ..< i : nil } + /// Audit SUB-108: `: ; , ) >` are legal unescaped inside a query value, and a user-chosen + /// password holds them, so after `=` one of them ends the value only where prose follows it (a + /// terminator, a blank or the end of the line): `…&api_key=abc: timeout` and + /// `connect.sid=abc; Path=/` still give their text back, `password=Pa:ss,word&…` goes whole. The + /// header form keeps the wide set, since a header value is not a query value. + private static func endsValue(_ char: (byte: UInt8, width: Int, depth: Int), at index: Int, + in bytes: [UInt8], afterHeaderSeparator: Bool) -> Bool { + guard !afterHeaderSeparator, isSoftTerminator(char.byte) else { return true } + let next = index + char.width + guard next < bytes.count else { return true } + return isValueTerminator(logicalByte(in: bytes, at: next).byte) + } + + // MARK: Percent escapes + + fileprivate static let percent = UInt8(ascii: "%") + private static let space = UInt8(ascii: " ") + + /// A value encoded more often than this is not one a URL builder produces by accident. + private static let maximumEncodingDepth = 4 + + /// One character as a URL decoder would see it: a raw byte, or a `%XX` escape, followed through + /// `%25` when the value was encoded more than once. `depth` is the number of layers (0 = raw). + fileprivate static func logicalByte(in bytes: [UInt8], at index: Int) + -> (byte: UInt8, width: Int, depth: Int) + { + let raw = bytes[index] + guard raw == percent, index + 2 < bytes.count, + let hi = hexValue(bytes[index + 1]), let lo = hexValue(bytes[index + 2]) else { + return (raw, 1, 0) + } + var value = hi << 4 | lo + var width = 3 + var depth = 1 + while value == percent, depth < maximumEncodingDepth, index + width + 1 < bytes.count, + let nextHi = hexValue(bytes[index + width]), let nextLo = hexValue(bytes[index + width + 1]) { + value = nextHi << 4 | nextLo + width += 2 + depth += 1 + } + return (value, width, depth) + } + + /// Whether the character in front of `index` is a letter or digit, reading a percent escape that + /// ends there (`%26`, `%2526`) as the character it decodes to. + private static func precededByWordCharacter(_ bytes: [UInt8], at index: Int) -> Bool { + guard index > 0, isLetterOrDigit(bytes[index - 1]) else { return false } + guard index >= 3, let hi = hexValue(bytes[index - 2]), let lo = hexValue(bytes[index - 1]) else { + return true + } + var k = index - 3 + var layers = 1 + while bytes[k] != percent { + guard layers < maximumEncodingDepth, k >= 2, + bytes[k - 1] == UInt8(ascii: "2"), bytes[k] == UInt8(ascii: "5") else { return true } + k -= 2 + layers += 1 + } + return isLetterOrDigit(hi << 4 | lo) + } + + fileprivate static func hexValue(_ b: UInt8) -> UInt8? { + switch b { + case UInt8(ascii: "0")...UInt8(ascii: "9"): return b - UInt8(ascii: "0") + case UInt8(ascii: "a")...UInt8(ascii: "f"): return b - UInt8(ascii: "a") + 10 + case UInt8(ascii: "A")...UInt8(ascii: "F"): return b - UInt8(ascii: "A") + 10 + default: return nil + } + } + /// The secret inside a URL's userinfo, given an index that may start `://`. `smb://user:pw@host` /// and `https://user:pw@host` put the credential in the authority, where no key precedes it, so the /// key matcher cannot see it and `load url=` logs `absoluteString` whole. The user name is left @@ -151,6 +390,139 @@ enum LogRedaction { return secretStart < i ? secretStart ..< i : nil } + // MARK: Registered secrets + + /// Shortest value `register` accepts. A literal match has no context to go by, so a one- or + /// two-byte value would black out every occurrence of that text in every line. + static let minimumSecretLength = 4 + + private static let secretsLock = NSLock() + nonisolated(unsafe) private static var _secrets: [String: (bytes: [UInt8], registrations: Int)] = [:] + + /// Snapshot taken once per line, so a register racing a redact sees the old set or the new one, + /// never half of it. Sorted longest first, so a secret that contains another goes whole. + private static var registeredSecrets: [[UInt8]] { + secretsLock.lock(); defer { secretsLock.unlock() } + return _secrets.isEmpty ? [] : _secrets.values.map(\.bytes).sorted { $0.count > $1.count } + } + + /// Registers the value and its percent-encoded form, which is how it appears inside a URL path or + /// query when it holds a character a URL cannot carry raw. Returns false for a value too short to + /// match literally. + /// + /// Counted (audit SUB-109): two owners of the same value, two profiles sharing a password or two + /// live servers, each unregister their own registration, and the value stays redacted until the + /// last one has. + @discardableResult + static func register(_ secret: String) -> Bool { + let forms = literalForms(of: secret) + guard !forms.isEmpty else { return false } + secretsLock.lock(); defer { secretsLock.unlock() } + for form in forms { + _secrets[form] = (Array(form.utf8), (_secrets[form]?.registrations ?? 0) + 1) + } + return true + } + + static func unregister(_ secret: String) { + let forms = literalForms(of: secret) + secretsLock.lock(); defer { secretsLock.unlock() } + for form in forms { + guard let entry = _secrets[form] else { continue } + _secrets[form] = entry.registrations > 1 ? (entry.bytes, entry.registrations - 1) : nil + } + } + + /// Whether `secret` is currently redacted as a registered literal. + static func isRegistered(_ secret: String) -> Bool { + secretsLock.lock(); defer { secretsLock.unlock() } + return _secrets[secret] != nil + } + + private static func literalForms(of secret: String) -> Set { + guard secret.utf8.count >= minimumSecretLength else { return [] } + var forms: Set = [secret] + if let encoded = secret.addingPercentEncoding(withAllowedCharacters: .urlPathAllowed) { forms.insert(encoded) } + if let encoded = secret.addingPercentEncoding(withAllowedCharacters: .alphanumerics) { forms.insert(encoded) } + return forms + } + + /// The span of a registered secret starting here, or nil. Exact bytes, no boundary rule: the host + /// said this value must never be logged, so it goes even inside a longer word. + private static func registeredSecretRange(in bytes: [UInt8], at index: Int, secrets: [[UInt8]]) -> Range? { + for secret in secrets where index + secret.count <= bytes.count && bytes[index] == secret[0] { + var matched = true + for offset in 1 ..< secret.count where bytes[index + offset] != secret[offset] { + matched = false + break + } + if matched { return index ..< index + secret.count } + } + return nil + } + + // MARK: Xtream Codes paths + + /// Path prefixes of the Xtream Codes stream layout, each paired with the zero-based positions of + /// the segments after it that hold a credential. `/live/`, `/movie/`, `/series/` and `/timeshift/` + /// carry `{user}/{password}/...`; the HLS redirect targets carry a session token first, and + /// `/hlsr/` then repeats `{user}/{password}` behind it. + private static let xtreamLayouts: [(prefix: [UInt8], secretSegments: [Int])] = [ + ("/live/", [1]), ("/movie/", [1]), ("/series/", [1]), ("/timeshift/", [1]), + ("/hls/", [0]), ("/hlsr/", [0, 2]), + ].map { (Array($0.0.utf8), $0.1) } + + /// The span from the first credential segment through the last one, given an index that may + /// start one of `xtreamLayouts`' prefixes, or nil. The user name in front of the password is left + /// readable for the same reason as in `userInfoSecretRange`; on `/hlsr/` it sits between the token + /// and the password and goes with them, since one line yields one span. + /// + /// A credential segment only counts when a further path segment follows it, because that is what + /// the layout guarantees and what an ordinary HLS path lacks: `/live/master.m3u8` and + /// `/live/channel1/index.m3u8` stay whole. Over-redacting some other three-deep `/live/` path is + /// the accepted cost. + private static func xtreamPathSecretRange(in bytes: [UInt8], at index: Int) -> Range? { + guard bytes[index] == UInt8(ascii: "/") else { return nil } + for layout in xtreamLayouts where hasPrefix(layout.prefix, in: bytes, at: index) { + var segments: [Range] = [] + var i = index + layout.prefix.count + let needed = layout.secretSegments.max()! + 2 + while segments.count < needed { + let start = i + while i < bytes.count, bytes[i] != UInt8(ascii: "/"), !isPathTerminator(bytes[i]) { i += 1 } + guard i > start else { break } + segments.append(start ..< i) + guard i < bytes.count, bytes[i] == UInt8(ascii: "/") else { break } + i += 1 + } + guard segments.count >= needed else { continue } + return segments[layout.secretSegments.min()!].lowerBound ..< segments[layout.secretSegments.max()!].upperBound + } + return nil + } + + private static func hasPrefix(_ prefix: [UInt8], in bytes: [UInt8], at index: Int) -> Bool { + guard index + prefix.count <= bytes.count else { return false } + for offset in 0 ..< prefix.count where bytes[index + offset] != prefix[offset] { return false } + return true + } + + /// Ends a URL path: the query, the fragment, or whatever the log line puts after the URL. + /// + /// `'` `,` and `)` are absent: they are legal raw in a path segment and a password holds them, so + /// `/live/u/pa,ss/1.ts` read as too few segments and kept its password. Where prose puts one after + /// the URL it lands in the last segment, which the layout never redacts, and the blank after it + /// still ends the path. + private static func isPathTerminator(_ b: UInt8) -> Bool { + switch b { + case UInt8(ascii: "?"), UInt8(ascii: "#"), UInt8(ascii: "\""), UInt8(ascii: ">"), + UInt8(ascii: " "), 0x09, 0x0A, 0x0D: + return true + default: + return false + } + } + /// Ends an authority component. `@` is deliberately absent: it is what the scan is looking for. private static func isAuthorityTerminator(_ b: UInt8) -> Bool { switch b { @@ -181,7 +553,9 @@ enum LogRedaction { /// very nearly every position, and only a run that survives that is ever decoded. private static func encodedPayloadRange(in bytes: [UInt8], at index: Int) -> Range? { guard bytes[index] == UInt8(ascii: "e") || bytes[index] == UInt8(ascii: "W") else { return nil } - if index > 0, isBase64URL(bytes[index - 1]) { return nil } + // A letter or digit in front means the run started earlier. A `-` or `_` does not: a blob + // glued to a version prefix (`/v1-eyJ…`) is still a blob (audit SUB-104). + if index > 0, isLetterOrDigit(bytes[index - 1]) { return nil } var end = index while end < bytes.count, isBase64URL(bytes[end]) { end += 1 } @@ -217,8 +591,55 @@ enum LogRedaction { isLetterOrDigit(b) || b == UInt8(ascii: "-") || b == UInt8(ascii: "_") } - /// `:` counts, so `…&api_key=abc: timeout` gives the token back and keeps the error text. None of - /// the credential shapes here (hex, base64url, percent-encoded cookie) contain a literal colon. + // MARK: Authorization schemes + + private static let authorizationSchemes: [[UInt8]] = ["bearer", "basic"].map { Array($0.utf8) } + + /// A credential after an `Authorization` scheme shorter than this is prose ("bearer token", + /// "basic realm"), not a credential. Sodalite's copy uses the same floor. + private static let minimumSchemeCredentialLength = 16 + + /// The credential after `Bearer ` or `Basic `, or nil (audit SUB-109). The scheme has no `=` or + /// `:` of its own, so the key matcher cannot see it, and `Authorization` is not a key: the + /// Jellyfin form of that header carries readable fields around its token. + private static func authorizationSchemeRange(in bytes: [UInt8], at index: Int) -> Range? { + let first = lowercased(bytes[index]) + guard first == UInt8(ascii: "b") else { return nil } + if index > 0, isLetterOrDigit(bytes[index - 1]) { return nil } + for scheme in authorizationSchemes where index + scheme.count < bytes.count { + var matched = true + for offset in 1 ..< scheme.count where lowercased(bytes[index + offset]) != scheme[offset] { + matched = false + break + } + guard matched else { continue } + var i = index + scheme.count + guard isBlank(bytes[i]) else { continue } + while i < bytes.count, isBlank(bytes[i]) { i += 1 } + let start = i + while i < bytes.count, isToken68(bytes[i]) { i += 1 } + if i - start >= minimumSchemeCredentialLength { return start ..< i } + } + return nil + } + + private static func isToken68(_ b: UInt8) -> Bool { + isBase64URL(b) || b == UInt8(ascii: ".") || b == UInt8(ascii: "~") || b == UInt8(ascii: "+") + || b == UInt8(ascii: "/") || b == UInt8(ascii: "=") + } + + private static func isBlank(_ b: UInt8) -> Bool { + b == space || b == 0x09 + } + + /// Audit SUB-108: the terminators that are also legal inside a query value. + private static func isSoftTerminator(_ b: UInt8) -> Bool { + b == UInt8(ascii: ":") || b == UInt8(ascii: ";") || b == UInt8(ascii: ",") + || b == UInt8(ascii: ")") || b == UInt8(ascii: ">") + } + + /// `:` counts, so `…&api_key=abc: timeout` gives the token back and keeps the error text, though + /// after `=` only where prose follows it (`endsValue`). private static func isValueTerminator(_ b: UInt8) -> Bool { switch b { case UInt8(ascii: "&"), UInt8(ascii: ";"), UInt8(ascii: ","), UInt8(ascii: ")"), diff --git a/Sources/AetherEngine/Diagnostics/PacketTimingProbe.swift b/Sources/AetherEngine/Diagnostics/PacketTimingProbe.swift index fde11c6af..33e12f976 100644 --- a/Sources/AetherEngine/Diagnostics/PacketTimingProbe.swift +++ b/Sources/AetherEngine/Diagnostics/PacketTimingProbe.swift @@ -26,6 +26,12 @@ public enum PacketTimingProbe { public var errorDescription: String? { description } } + /// nil when the step does not fit `Int64` (audit SUB-112): the timestamps are the probed file's own. + static func dtsDelta(_ dts: Int64, after last: Int64) -> Int64? { + let (delta, overflow) = dts.subtractingReportingOverflow(last) + return overflow ? nil : delta + } + public static func run( url: URL, seekSeconds: Double, @@ -107,9 +113,13 @@ public enum PacketTimingProbe { if isKey { keyCount += 1 } if dts != Int64.min { if let last = lastDts { - let d = dts - last - dtsDeltaHist[d, default: 0] += 1 - if d <= 0 { nonMonotonicDts += 1 } + if let d = dtsDelta(dts, after: last) { + dtsDeltaHist[d, default: 0] += 1 + if d <= 0 { nonMonotonicDts += 1 } + } else { + // A step no Int64 holds is a broken timeline, not a delta worth a bucket. + nonMonotonicDts += 1 + } } lastDts = dts } diff --git a/Sources/AetherEngine/Diagnostics/PlayedMediaLedger.swift b/Sources/AetherEngine/Diagnostics/PlayedMediaLedger.swift new file mode 100644 index 000000000..c6585e05e --- /dev/null +++ b/Sources/AetherEngine/Diagnostics/PlayedMediaLedger.swift @@ -0,0 +1,171 @@ +import Foundation + +/// AE#514 round 2: the bytes of the media the session plays, keyed by presentation time on the source +/// axis, so the bitrate fields can be metered from what the playhead crossed rather than from what the +/// reader transferred. +/// +/// The transfer counter and playback part ways on every route that reads ahead: VOD prefetches minutes +/// in front of the playhead, a seek discards a buffer and fetches it again, and a paused live session +/// keeps draining the origin into its DVR window (AE#443). A rate metered off that counter reported the +/// prefetch (35 Mbps for a 20 Mbps stream) and climbed without bound through a live pause. +/// +/// The pump records every packet of the played video and audio streams as it hands them on; the +/// sampler consumes the span the playhead crossed since its last tick. Thread-safe: the pump records +/// on its read thread, the sampler consumes on the main actor. +final class PlayedMediaLedger: @unchecked Sendable { + + enum Track: Int, CaseIterable { + case video = 0 + case audio = 1 + } + + private struct Entry { + let pts: Double + var bytes: Int + } + + /// A packet this far behind the newest one on its track is a re-read (a seek replaying a range the + /// reader already delivered), not B-frame reorder, which stays well under a second. + static let rereadThresholdSeconds: Double = 2.0 + + /// Entries held across both tracks. A VOD read-ahead of several minutes stays far below it; past + /// it the farthest-ahead packets go unrecorded, so the span they cover later reads as unmeasured + /// rather than as a lower rate. + static let maxEntries = 262_144 + + private let lock = NSLock() + private var tracks: [[Entry]] = Array(repeating: [], count: Track.allCases.count) + private var count = 0 + + /// Records one played packet. `pts` is in seconds on the source axis, the axis the playhead the + /// sampler hands `consume` is on. + func record(_ track: Track, pts: Double, bytes: Int) { + guard pts.isFinite, bytes > 0 else { return } + lock.lock() + defer { lock.unlock() } + var entries = tracks[track.rawValue] + tracks[track.rawValue] = [] + defer { tracks[track.rawValue] = entries } + + if let newest = entries.last?.pts, pts < newest - Self.rereadThresholdSeconds { + let cut = Self.lowerBound(entries, pts) + count -= entries.count - cut + entries.removeSubrange(cut...) + } + var index = entries.count + while index > 0, entries[index - 1].pts > pts { index -= 1 } + if index > 0, entries[index - 1].pts == pts { + entries[index - 1].bytes = bytes + return + } + guard count < Self.maxEntries else { return } + entries.insert(Entry(pts: pts, bytes: bytes), at: index) + count += 1 + } + + /// Bytes of every packet presented in `[from, to)`, which leave the ledger with everything before `to`. + func consume(from: Double, to: Double) -> Int64 { + lock.lock() + defer { lock.unlock() } + var total: Int64 = 0 + for track in tracks.indices { + let start = Self.lowerBound(tracks[track], from) + let end = Self.lowerBound(tracks[track], to) + if start < end { + for entry in tracks[track][start.. 0 else { return } + tracks[track].removeFirst(end) + count -= end + } + + private static func lowerBound(_ entries: [Entry], _ pts: Double) -> Int { + var low = 0, high = entries.count + while low < high { + let mid = (low + high) / 2 + if entries[mid].pts < pts { low = mid + 1 } else { high = mid } + } + return low + } +} + +/// AE#514 round 2: both bitrate fields over the media the playhead crossed. Advanced once per sampler +/// tick with the playhead on the ledger's axis. +/// +/// A step forward no larger than playback could have covered in the tick is charged: its bytes from the +/// ledger, its media seconds as the divisor. Anything else is a seek or a jump and charges nothing, and a +/// paused playhead does not move, so both values stand still through a pause on every route, live +/// included. A step whose span the ledger holds no bytes for is left out too (the ledger was not fed +/// there, or its axis does not line up with this playhead): an unmeasured span is a gap, never a zero. +struct PlayedBitrateMeter { + + /// Fastest playback the charge accepts, as media seconds per wall second, plus a fixed slack for + /// tick jitter. Past that the playhead jumped. + static let maxRate: Double = 4.0 + static let stepSlackSeconds: Double = 0.5 + + /// Charged ticks the instant value averages over, about the last ten seconds of playback. + static let windowTicks = 10 + + private var lastPlayhead: Double? + private var window: [(bytes: Int64, seconds: Double)] = [] + private(set) var lifetimeBytes: Int64 = 0 + private(set) var lifetimeSeconds: Double = 0 + + mutating func advance(to playhead: Double?, wallSeconds: Double, ledger: PlayedMediaLedger) { + guard let playhead, playhead.isFinite else { return } + defer { lastPlayhead = playhead } + guard let last = lastPlayhead else { + ledger.discard(below: playhead) + return + } + let step = playhead - last + guard step != 0 else { return } + guard step > 0, step <= max(0, wallSeconds) * Self.maxRate + Self.stepSlackSeconds else { + ledger.discard(below: playhead) + return + } + let bytes = ledger.consume(from: last, to: playhead) + guard bytes > 0 else { return } + lifetimeBytes += bytes + lifetimeSeconds += step + window.append((bytes, step)) + if window.count > Self.windowTicks { window.removeFirst(window.count - Self.windowTicks) } + } + + /// Mean rate of the last `windowTicks` charged ticks. nil before the first one. + var instantMbps: Double? { + let seconds = window.reduce(0) { $0 + $1.seconds } + let bytes = window.reduce(Int64(0)) { $0 + $1.bytes } + return Self.mbps(bytes: bytes, seconds: seconds) + } + + /// Mean rate of everything played this session. nil before the first charged tick. + var averageMbps: Double? { Self.mbps(bytes: lifetimeBytes, seconds: lifetimeSeconds) } + + private static func mbps(bytes: Int64, seconds: Double) -> Double? { + guard seconds > 0, bytes > 0 else { return nil } + return Double(bytes) * 8.0 / seconds / 1_000_000.0 + } +} diff --git a/Sources/AetherEngine/Disc/DVDIFOParser.swift b/Sources/AetherEngine/Disc/DVDIFOParser.swift index f64d4d44b..a8c5c7134 100644 --- a/Sources/AetherEngine/Disc/DVDIFOParser.swift +++ b/Sources/AetherEngine/Disc/DVDIFOParser.swift @@ -216,6 +216,34 @@ enum DVDIFOParser { return languages } + /// Every subpicture substream id the title's IFO declares, with or without a language, ascending. + /// + /// The same control-table reading as `parseStreamLanguages`: an entry the main PGC marks + /// unavailable is not in the VOBs, and each display-mode number an entry names is its own + /// MPEG-PS substream. Without a readable PGC the attribute position is the number (#651). nil when + /// the bytes are no VTS IFO, which is not the same answer as a title with no subpictures. + static func parseSubpictureStreamIDs(_ data: [UInt8]) -> [Int]? { + guard data.count >= 12, Array(data[0..<12]) == vtsMagic, + data.count > vtsSubpictureCountOffset else { return nil } + let pgc = mainPGCOffset(data) + let count = min(Int(data[vtsSubpictureCountOffset]), vtsMaxSubpictureStreams) + var ids = Set() + for n in 0..> 24) & 0x1F)) + for shift in [16, 8, 0] { + let number = (control >> shift) & 0x1F + if number != 0 { ids.insert(subpictureSubstreamBase + number) } + } + } + return ids.sorted() + } + /// Base substream id for a DVD audio coding mode; a stream's id is this plus its substream number. /// AC-3, DTS and LPCM ride in private_stream_1, where FFmpeg reports the substream byte; MPEG audio /// has its own PES stream id, where FFmpeg reports the full start code. nil for the reserved coding diff --git a/Sources/AetherEngine/Disc/DVDSubpictureAssembler.swift b/Sources/AetherEngine/Disc/DVDSubpictureAssembler.swift new file mode 100644 index 000000000..c70225745 --- /dev/null +++ b/Sources/AetherEngine/Disc/DVDSubpictureAssembler.swift @@ -0,0 +1,77 @@ +import Foundation + +/// #651: joins the PES fragments of one DVD subpicture unit back into the packet `dvdsubdec` expects. +/// +/// `mpegps` hands a subpicture stream it created itself to libavformat's `dvdsub` parser, which does +/// this join. A stream the engine declares from the IFO before the probe cannot ask for that parser +/// (`need_parsing` is internal to libavformat), and a unit larger than one 2 KB pack then reaches the +/// decoder as its first fragment alone, which does not decode. This is the same rule as +/// `libavcodec/dvdsub_parser.c`: the unit's size is its first 16 bits (or the 32 bits after a zero +/// there), fragments append until that size is reached, the unit carries the first fragment's +/// timing, and a fragment that would overrun the size drops the unit. +struct DVDSubpictureAssembler { + + /// Timing of the fragment that opened the unit, which the joined packet carries. + struct Timing: Equatable { + let pts: Int64 + let dts: Int64 + let pos: Int64 + let duration: Int64 + } + + struct Unit: Equatable { + let data: [UInt8] + let timing: Timing + } + + private var buffer: [UInt8] = [] + private var expected = 0 + private var timing: Timing? + + var isAssembling: Bool { timing != nil } + + var reservedCapacity: Int { buffer.capacity } + + static let maxUnitBytes = 1 << 20 + static let initialReservationBytes = 64 * 1024 + + /// Feed one demuxed fragment. Returns the unit once it is complete, nil while it is not (or when + /// the fragment was dropped). + mutating func ingest(_ fragment: UnsafeRawBufferPointer, timing fragmentTiming: Timing) -> Unit? { + if timing == nil { + let bytes = fragment.bindMemory(to: UInt8.self) + guard bytes.count >= 2 else { return nil } + var size = Int(bytes[0]) << 8 | Int(bytes[1]) + if size == 0 { + guard bytes.count >= 6 else { return nil } + size = Int(bytes[2]) << 24 | Int(bytes[3]) << 16 | Int(bytes[4]) << 8 | Int(bytes[5]) + } else if bytes.count < 6 { + return nil + } + // Audit NET-104: the 32-bit form stated up to 4 GiB, all of it reserved up front, and the + // unit then swallowed every later fragment of the stream. A DVD SPU is at most 53,220 + // bytes; 1 MiB leaves room for the HD-DVD units `dvdsubdec` decodes. + guard size > 0, size <= Self.maxUnitBytes else { return nil } + expected = size + timing = fragmentTiming + buffer.removeAll(keepingCapacity: true) + buffer.reserveCapacity(min(size, Self.initialReservationBytes)) + } + guard buffer.count + fragment.count <= expected, let unitTiming = timing else { + reset() + return nil + } + buffer.append(contentsOf: fragment.bindMemory(to: UInt8.self)) + guard buffer.count >= expected else { return nil } + let unit = Unit(data: buffer, timing: unitTiming) + reset() + return unit + } + + /// Drop a half-joined unit, for a seek: the next fragment read is from somewhere else. + mutating func reset() { + buffer.removeAll(keepingCapacity: true) + expected = 0 + timing = nil + } +} diff --git a/Sources/AetherEngine/Disc/DVDTitleSelector.swift b/Sources/AetherEngine/Disc/DVDTitleSelector.swift index 3516be7f8..9fe0a7f9b 100644 --- a/Sources/AetherEngine/Disc/DVDTitleSelector.swift +++ b/Sources/AetherEngine/Disc/DVDTitleSelector.swift @@ -4,13 +4,17 @@ import Foundation /// VTS_nn_0.VOB = menus; VTS_nn_1..9.VOB = content. Titles are ordered largest-first (total content /// VOB size, a proxy for duration) so id 0 is the main feature, matching the Blu-ray convention. enum DVDTitleSelector { + /// DVD-Video numbers its title sets 01 to 99. The number is an on-disc name, so anything else is + /// not a title set and must not become a group (audit NET-103). + static let vtsRange = 1...99 + /// One selectable title = one VTS's content VOBs (whole-VTS resolution; per-cell/episodic splitting /// is deferred). VOBs within a title are ordered by part; titles are ordered by total size, largest first. static func enumerateTitleVOBGroups(_ files: [DiscFile]) -> [(vtsn: Int, vobs: [DiscFile])] { struct Part { let title: Int; let part: Int; let file: DiscFile } var parts: [Part] = [] for f in files { - guard let (title, part) = parseVOBName(f.name), part >= 1 else { continue } + guard let (title, part) = parseVOBName(f.name), part >= 1, vtsRange.contains(title) else { continue } parts.append(Part(title: title, part: part, file: f)) } let byTitle = Dictionary(grouping: parts, by: \.title) diff --git a/Sources/AetherEngine/Disc/DiscMetadata.swift b/Sources/AetherEngine/Disc/DiscMetadata.swift index bc82a633b..96cc4957d 100644 --- a/Sources/AetherEngine/Disc/DiscMetadata.swift +++ b/Sources/AetherEngine/Disc/DiscMetadata.swift @@ -72,12 +72,19 @@ struct DiscTitle: Sendable, Equatable { /// every track as undetermined without this; `Demuxer.trackInfo` backfills from it. Empty when the /// disc declares none (#527). let streamLanguages: [Int: String] + /// DVD: every subpicture substream id (`0x20` + number) the VTS IFO declares for this title, + /// language or not, ascending. `Demuxer` creates these streams before the probe, so a subtitle + /// whose first packet comes late is a track from the start rather than whenever the probe + /// happens to reach it (#651). Empty when the IFO declares none, nil when it is unreadable: only a + /// title whose IFO was read gets the short probe, since only then is nothing left to discover. + let dvdSubpictureStreamIDs: [Int]? init(id: Int, durationTicks: UInt64, chapters: [DiscChapter] = [], bdClipIDs: [String]? = nil, bdClipSubtractTicks: [Int64]? = nil, bdClipCumulativeBeforeTicks: [UInt64]? = nil, dvdVTSN: Int? = nil, dvdTitleNumber: Int? = nil, - streamLanguages: [Int: String] = [:]) { + streamLanguages: [Int: String] = [:], + dvdSubpictureStreamIDs: [Int]? = nil) { self.id = id self.durationTicks = durationTicks self.chapters = chapters @@ -87,6 +94,7 @@ struct DiscTitle: Sendable, Equatable { self.dvdVTSN = dvdVTSN self.dvdTitleNumber = dvdTitleNumber self.streamLanguages = streamLanguages + self.dvdSubpictureStreamIDs = dvdSubpictureStreamIDs } } diff --git a/Sources/AetherEngine/Disc/DiscReader.swift b/Sources/AetherEngine/Disc/DiscReader.swift index 8cfc23fca..1b77a1f44 100644 --- a/Sources/AetherEngine/Disc/DiscReader.swift +++ b/Sources/AetherEngine/Disc/DiscReader.swift @@ -49,13 +49,9 @@ enum DiscReader { return nil } let playlistDir = (try? udf.list(path: ["BDMV", "PLAYLIST"])) ?? [] - var parsed: [MPLSPlaylist] = [] - for e in playlistDir where e.name.hasSuffix(".mpls") { - let exts = (try? udf.extents(of: e)) ?? [] - guard !exts.isEmpty else { continue } - let bytes = readAll(reader, exts) - if let pl = MPLSParser.parse(bytes) { parsed.append(pl) } - } + let parsed = scanPlaylists(playlistDir, + extents: { (try? udf.extents(of: $0)) ?? [] }, + read: { readAll(reader, $0) }) let titles = BDTitleSelector.enumerateTitles(parsed) guard !titles.isEmpty else { EngineLog.emit("[disc] BDMV present but no parseable .mpls (\(playlistDir.count) PLAYLIST entries, \(parsed.count) parsed); cannot select a title", category: .demux) @@ -64,24 +60,14 @@ enum DiscReader { let selectedIndex = selectTitleID.flatMap { titles.indices.contains($0) ? $0 : nil } ?? 0 let selected = titles[selectedIndex] let streamDir = (try? udf.list(path: ["BDMV", "STREAM"])) ?? [] - var allExtents: [(offset: Int64, length: Int64)] = [] - // One ClipSpan per resolved clip: byte start in the concat stream + how far to pull its - // timestamps back so it continues contiguously from clip 0 (AE#105 multi-clip position drift). - var clipTimeline: [ClipSpan] = [] - let subTicks = selected.bdClipSubtractTicks ?? [] - let cumBeforeTicks = selected.bdClipCumulativeBeforeTicks ?? [] - for (k, clip) in (selected.bdClipIDs ?? []).enumerated() { - guard let e = streamDir.first(where: { $0.name == "\(clip).m2ts" }), - let exts = try? udf.extents(of: e) else { continue } - let byteStart = allExtents.reduce(Int64(0)) { $0 + max(0, $1.length) } - let predictedShift = k < subTicks.count ? Double(subTicks[k]) / discTickRate : 0 - let cumBeforeSec = k < cumBeforeTicks.count ? Double(cumBeforeTicks[k]) / discTickRate : 0 - clipTimeline.append(ClipSpan(concatByteStart: byteStart, - cumulativeBeforeSec: cumBeforeSec, - predictedShiftSec: predictedShift)) - EngineLog.emit("[disc] AE#105 clip[\(k)] id=\(clip) subTicks=\(k < subTicks.count ? subTicks[k] : 0) predictedSec=\(String(format: "%.3f", predictedShift)) cumBeforeSec=\(String(format: "%.3f", cumBeforeSec)) byteStart=\(byteStart)", category: .demux) - allExtents += exts - } + let streamIndex = Dictionary(streamDir.map { ($0.name, $0) }, uniquingKeysWith: { first, _ in first }) + let assembled = assembleBluRayTitle( + clipIDs: selected.bdClipIDs ?? [], + subtractTicks: selected.bdClipSubtractTicks ?? [], + cumulativeBeforeTicks: selected.bdClipCumulativeBeforeTicks ?? [], + extentsOfClip: { clip in streamIndex["\(clip).m2ts"].flatMap { try? udf.extents(of: $0) } }) + let allExtents = assembled.extents + var clipTimeline = assembled.clipTimeline guard !allExtents.isEmpty else { EngineLog.emit("[disc] selected title \(selectedIndex) clips=\(selected.bdClipIDs ?? []) but resolved no m2ts extents in BDMV/STREAM (\(streamDir.count) entries); cannot build stream", category: .demux) return nil @@ -102,6 +88,108 @@ enum DiscReader { clipTimeline: clipTimeline) } + // MARK: - Bounds on what a disc image can make recognition do (audit NET-103) + + /// `.mpls` entries examined per PLAYLIST directory. Real discs carry dozens, anti-rip discs a few hundred. + static let maxPlaylistFiles = 4_000 + /// Bytes of playlist data read in total. A real `.mpls` is KB-scale, so this is orders above any disc. + static let maxPlaylistBytes: Int64 = 32 * 1024 * 1024 + /// PlayItems kept across every parsed playlist (about 32 bytes each), so a directory of maximal + /// playlists cannot be retained whole. + static let maxPlaylistItems = 500_000 + /// Extents in one assembled title. A real m2ts is a few dozen extents (1 GB each), a decoy loop + /// repeats a short clip a few hundred times. + static let maxTitleExtents = 65_536 + /// A run of this many unreadable playlists means the source is gone (cancelled or closed reader). + private static let maxConsecutiveShortReads = 16 + /// `readAll`'s ceiling. + private static let maxSmallFileBytes: Int64 = 8 * 1024 * 1024 + + /// Reads and parses the `.mpls` entries of a PLAYLIST directory under the bounds above. Stops at the + /// first bound that trips and keeps what was parsed before it. + static func scanPlaylists( + _ entries: [UDFEntry], + extents: (UDFEntry) -> [(offset: Int64, length: Int64)], + read: ([(offset: Int64, length: Int64)]) -> [UInt8] + ) -> [MPLSPlaylist] { + var parsed: [MPLSPlaylist] = [] + var examined = 0 + var bytesRead: Int64 = 0 + var items = 0 + var shortReads = 0 + for e in entries where e.name.hasSuffix(".mpls") { + examined += 1 + guard examined <= maxPlaylistFiles else { + EngineLog.emit("[disc] PLAYLIST scan stopped at \(maxPlaylistFiles) .mpls entries", category: .demux) + break + } + let exts = extents(e) + guard !exts.isEmpty else { continue } + let declared = exts.reduce(Int64(0)) { $0 + max(0, $1.length) } + guard declared > 0, declared <= maxSmallFileBytes else { continue } + bytesRead += declared + guard bytesRead <= maxPlaylistBytes else { + EngineLog.emit("[disc] PLAYLIST scan stopped after \(maxPlaylistBytes) bytes of playlists", category: .demux) + break + } + let bytes = read(exts) + if Int64(bytes.count) < declared { + shortReads += 1 + if shortReads >= maxConsecutiveShortReads { + EngineLog.emit("[disc] PLAYLIST scan stopped, \(shortReads) unreadable playlists in a row", category: .demux) + break + } + } else { + shortReads = 0 + } + guard let pl = MPLSParser.parse(bytes) else { continue } + items += pl.clipIDs.count + guard items <= maxPlaylistItems else { + EngineLog.emit("[disc] PLAYLIST scan stopped after \(maxPlaylistItems) PlayItems", category: .demux) + break + } + parsed.append(pl) + } + return parsed + } + + /// Resolves a title's clips into the concatenated extent list plus one `ClipSpan` per resolved clip + /// (byte start in the concat stream, and how far to pull its timestamps back so it continues + /// contiguously from clip 0, AE#105). Each distinct clip is looked up once, a repeated clip reuses its + /// extents, and the title ends at the clip that would pass `maxTitleExtents`. + static func assembleBluRayTitle( + clipIDs: [String], subtractTicks: [Int64], cumulativeBeforeTicks: [UInt64], + extentsOfClip: (String) -> [(offset: Int64, length: Int64)]? + ) -> (extents: [(offset: Int64, length: Int64)], clipTimeline: [ClipSpan]) { + var allExtents: [(offset: Int64, length: Int64)] = [] + var clipTimeline: [ClipSpan] = [] + var resolved: [String: [(offset: Int64, length: Int64)]?] = [:] + var byteStart: Int64 = 0 + for (k, clip) in clipIDs.enumerated() { + let lookup: [(offset: Int64, length: Int64)]? + if let known = resolved[clip] { + lookup = known + } else { + lookup = extentsOfClip(clip) + resolved[clip] = .some(lookup) + } + guard let exts = lookup else { continue } + guard allExtents.count + exts.count <= maxTitleExtents else { + EngineLog.emit("[disc] title truncated at clip[\(k)]: more than \(maxTitleExtents) extents", category: .demux) + break + } + let predictedShift = k < subtractTicks.count ? Double(subtractTicks[k]) / discTickRate : 0 + let cumBeforeSec = k < cumulativeBeforeTicks.count ? Double(cumulativeBeforeTicks[k]) / discTickRate : 0 + clipTimeline.append(ClipSpan(concatByteStart: byteStart, + cumulativeBeforeSec: cumBeforeSec, + predictedShiftSec: predictedShift)) + EngineLog.emit("[disc] AE#105 clip[\(k)] id=\(clip) subTicks=\(k < subtractTicks.count ? subtractTicks[k] : 0) predictedSec=\(String(format: "%.3f", predictedShift)) cumBeforeSec=\(String(format: "%.3f", cumBeforeSec)) byteStart=\(byteStart)", category: .demux, level: .verbose) + allExtents += exts + byteStart += exts.reduce(Int64(0)) { $0 + max(0, $1.length) } + } + return (allExtents, clipTimeline) + } + /// Memoize a successful recognition so a re-open of the same source (track switch on a remote ISO) /// reuses it. No-op when the caller passed no cache key (custom sources opt in explicitly). private static func storeRecognition( @@ -129,7 +217,7 @@ enum DiscReader { // Extent lengths are untrusted on-disc bytes (up to ~1 GB each). Cap the total before // allocating so a crafted .mpls cannot drive an arbitrary allocation (jetsam/DoS); 8 MB // matches UDFReader.readDirectory's guard and dwarfs any real playlist (KB-scale). - let maxBytes: Int64 = 8 * 1024 * 1024 + let maxBytes = maxSmallFileBytes let declared = exts.reduce(Int64(0)) { $0 + max(0, $1.length) } guard declared > 0, declared <= maxBytes else { return [] } let total = Int(declared) @@ -174,7 +262,8 @@ enum DiscReader { // incidental content VTS are excluded. Any parse failure (or a filter that would empty the list) // falls back to the full VOB-grouped set, so a disc with an unreadable VMGI still plays multi-title. var orderedGroups = groups - if let ifoFile = files.first(where: { $0.name.uppercased() == "VIDEO_TS.IFO" }) { + let filesByName = Dictionary(files.map { ($0.name.uppercased(), $0) }, uniquingKeysWith: { first, _ in first }) + if let ifoFile = filesByName["VIDEO_TS.IFO"] { let ifoBytes = readAll(reader, [(offset: Int64(ifoFile.startSector * iso.sectorSize), length: Int64(ifoFile.length))]) if let ifoTitles = DVDIFOParser.parseTitles(ifoBytes) { @@ -195,9 +284,10 @@ enum DiscReader { var chapters: [DiscChapter] = [] // The VOBs carry no track language, so the IFO's attribute tables are the only source (#527). var streamLanguages: [Int: String] = [:] + var subpictureStreamIDs: [Int]? let nn = g.vtsn < 10 ? "0\(g.vtsn)" : "\(g.vtsn)" let ifoName = "VTS_\(nn)_0.IFO" - if let vtsIFO = files.first(where: { $0.name.uppercased() == ifoName }) { + if let vtsIFO = filesByName[ifoName] { let bytes = readAll(reader, [(offset: Int64(vtsIFO.startSector * iso.sectorSize), length: Int64(vtsIFO.length))]) if let detail = DVDIFOParser.parseTitleDetail(bytes) { @@ -207,9 +297,11 @@ enum DiscReader { } } streamLanguages = DVDIFOParser.parseStreamLanguages(bytes) + subpictureStreamIDs = DVDIFOParser.parseSubpictureStreamIDs(bytes) } return DiscTitle(id: idx, durationTicks: durationTicks, chapters: chapters, dvdVTSN: g.vtsn, - streamLanguages: streamLanguages) + streamLanguages: streamLanguages, + dvdSubpictureStreamIDs: subpictureStreamIDs) } storeRecognition(cacheKey: cacheKey, selectTitleID: selectTitleID, formatHint: "mpeg", titles: titles, selectedIndex: selectedIndex, extents: extents) diff --git a/Sources/AetherEngine/Disc/ISO9660Reader.swift b/Sources/AetherEngine/Disc/ISO9660Reader.swift index a0fcc1b0e..2329d9ad2 100644 --- a/Sources/AetherEngine/Disc/ISO9660Reader.swift +++ b/Sources/AetherEngine/Disc/ISO9660Reader.swift @@ -102,8 +102,17 @@ final class ISO9660Reader { // MARK: - Raw IO + /// Matches `UDFReader.readDirectory`'s directory cap. `length` is an unchecked on-disc u32 + /// (root record or a subdirectory record); unclamped, a crafted `0xFFFFFFFF` zero-fills up to + /// 4 GB (jetsam on tvOS/iOS) and `Int32(count - got)` traps for anything past `Int32.max` + /// (audit NET-5). + private static let maxExtentBytes = 8 * 1024 * 1024 + private func readExtent(lba: Int, length: Int) throws -> [UInt8] { - try ISO9660Reader.readBytes(reader, at: lba * sectorSize, count: length) + guard length >= 0, length <= Self.maxExtentBytes else { + throw DiscError.malformed("extent length \(length) exceeds \(Self.maxExtentBytes) bytes") + } + return try ISO9660Reader.readBytes(reader, at: lba * sectorSize, count: length) } private static func readBytes(_ reader: IOReader, at offset: Int, count: Int) throws -> [UInt8] { @@ -114,7 +123,10 @@ final class ISO9660Reader { var got = 0 try buf.withUnsafeMutableBufferPointer { ptr in while got < count { - let n = reader.read(ptr.baseAddress!.advanced(by: got), size: Int32(count - got)) + // Clamped so `Int32(...)` is always a checked, in-range conversion, whatever `count` + // a future caller passes (audit NET-5: `Int32(count - got)` used to trap directly). + let chunk = min(count - got, 1 << 20) + let n = reader.read(ptr.baseAddress!.advanced(by: got), size: Int32(chunk)) if n == 0 { break } // EOF if n < 0 { throw DiscError.malformed("read error at \(offset + got)") } got += Int(n) diff --git a/Sources/AetherEngine/Disc/MPLSParser.swift b/Sources/AetherEngine/Disc/MPLSParser.swift index a08b39de7..cb4edb0c5 100644 --- a/Sources/AetherEngine/Disc/MPLSParser.swift +++ b/Sources/AetherEngine/Disc/MPLSParser.swift @@ -20,12 +20,16 @@ struct MPLSPlaylist: Equatable { } enum MPLSParser { + /// BD-ROM caps a playlist at 999 PlayItems, and the count field is 16 bits wide (audit NET-103). + static let maxPlayItems = 999 + static func parse(_ data: [UInt8]) -> MPLSPlaylist? { guard data.count >= 16, Array(data[0..<4]) == Array("MPLS".utf8) else { return nil } let plStart = be32(data, 8) guard plStart + 10 <= data.count else { return nil } let count = be16(data, plStart + 6) + guard count <= maxPlayItems else { return nil } var pos = plStart + 10 var clips: [String] = [] var ticks: UInt64 = 0 diff --git a/Sources/AetherEngine/Disc/UDFReader.swift b/Sources/AetherEngine/Disc/UDFReader.swift index a8f10dc3e..5caad1b7f 100644 --- a/Sources/AetherEngine/Disc/UDFReader.swift +++ b/Sources/AetherEngine/Disc/UDFReader.swift @@ -97,15 +97,18 @@ final class UDFReader { let avdp = try readSector(256) guard tagID(avdp) == 2 else { throw DiscError.notUDF } let vdsLen = u32(avdp, 16), vdsLoc = u32(avdp, 20) - let vdsSectors = max(1, vdsLen / ss) + // A real Volume Descriptor Sequence is 16 sectors (ECMA-167); vdsLen is an untrusted u32 + // from the anchor, and an unclamped one can drive millions of sector reads before parsing + // ever fails (audit NET-11). + let vdsSectors = min(max(1, vdsLen / ss), 256) var lvd: [UInt8]? = nil - for i in 0.. 0, off + len <= lvd.count else { break } - if type == 1 { + // `off + len <= lvd.count` only bounds the map's OWN declared length; type 1 and type 2 + // maps read fixed fields past that length (off+4, off+38..43), so a short len with a + // long-enough neighbour still let those reads run past the sector buffer and trap + // (audit NET-4). A too-short map of a known type falls through to the generic case below. + if type == 1, len >= 6 { let pn = u16(lvd, off+4) partMaps.append(PartMap(isMetadata: false, physicalPartNumber: pn, metadataFileBlock: 0)) - } else if type == 2 { + } else if type == 2, len >= 44 { let pn = u16(lvd, off+38) let metaFileBlock = u32(lvd, off+40) partMaps.append(PartMap(isMetadata: true, physicalPartNumber: pn, metadataFileBlock: metaFileBlock)) diff --git a/Sources/AetherEngine/Display/DisplayCriteriaController.swift b/Sources/AetherEngine/Display/DisplayCriteriaController.swift index 9a6f14f5e..8d1d181e6 100644 --- a/Sources/AetherEngine/Display/DisplayCriteriaController.swift +++ b/Sources/AetherEngine/Display/DisplayCriteriaController.swift @@ -44,6 +44,8 @@ final class DisplayCriteriaController { /// #339: armed at the criteria write, not when the play gate opens, so an engine-written switch is /// observable end to end instead of having to be guessed at from the in-progress flag. private let observation = SwitchObservation() + /// DEC-105: what is left of the observed-end wait for the armed switch (`observedEndCap`). + private var observedEndBudget: ObservedEndBudget? #endif /// The arm generation whose record has been spent. Only a gate that ends the load's wait spends it, so @@ -70,8 +72,14 @@ final class DisplayCriteriaController { /// #133 pure decision: skip only when we previously applied (`didApply`) exactly these criteria and have /// not reset since. Otherwise write, returning whether a dynamic-range switch is expected (HDR) or not (SDR). - nonisolated static func applyOutcome(didApply: Bool, last: AppliedCriteria?, target: AppliedCriteria) -> ApplyResult { - if didApply, last == target { return .unchanged } + /// + /// AE#678: `managerHoldsCriteria` is the display manager's own answer. A host that wrote + /// `preferredDisplayCriteria = nil` itself (leaving full screen for a preview) leaves `lastApplied` + /// describing criteria the panel no longer has, and the skip then kept the UI mode on a channel that + /// needed its frame rate. + nonisolated static func applyOutcome(didApply: Bool, last: AppliedCriteria?, target: AppliedCriteria, + managerHoldsCriteria: Bool = true) -> ApplyResult { + if didApply, managerHoldsCriteria, last == target { return .unchanged } return target.isHDR ? .willSwitch : .applied } @@ -156,6 +164,40 @@ final class DisplayCriteriaController { return observedEndCapMs } + /// The observed-end wait left for one armed switch, shared by every gate that waits on it. + struct ObservedEndBudget: Equatable { + let armGeneration: Int + let deadlineNanos: UInt64 + } + + /// DEC-105: `observedEndCapMs` is one budget per armed switch, not per gate. The #667 wait leaves the + /// record unspent for the play gate, so on a panel whose end is never announced (or whose in-progress + /// flag sticks) both read the same `.running`, and each used to spend the whole cap on one switch. + /// The first gate to wait sets the deadline; a later gate on the same arm gets what is left of it. + nonisolated static func observedEndCap(settleCap: SettleCap, startRecorded: Bool, armGeneration: Int, + nowNanos: UInt64, budget: ObservedEndBudget?) + -> (capMs: Int, budget: ObservedEndBudget?) { + let capMs = settleCapMs(cap: settleCap, startRecorded: startRecorded) + guard capMs == observedEndCapMs, armGeneration != 0 else { return (capMs, budget) } + if let budget, budget.armGeneration == armGeneration { + return (min(capMs, elapsedMs(fromNanos: nowNanos, toNanos: budget.deadlineNanos)), budget) + } + let deadline = nowNanos &+ UInt64(capMs) * 1_000_000 + return (capMs, ObservedEndBudget(armGeneration: armGeneration, deadlineNanos: deadline)) + } + + /// Audit LIF-106: one poll of a switch wait. False once the load waiting on it is gone: its task + /// was cancelled (a cancelled `Task.sleep` returns at once, so the `try?` these loops used turned + /// them into a hot spin on the main actor until the cap) or a newer load or stop superseded it. + static func gateTick(milliseconds: Int, isCurrent: () -> Bool) async -> Bool { + do { + try await Task.sleep(for: .milliseconds(milliseconds)) + } catch { + return false + } + return isCurrent() + } + /// Both stages spend a deadline, not a poll count. `n` sleeps of `m` ms is only `n * m` on an idle /// scheduler: the same 40 x 50 ms Stage 2 ran 2082 ms in one of the reporter's runs and 2862 ms in /// another, on a device reporting `thermal=serious` throughout. A budget that stretches 40 % under the @@ -758,7 +800,15 @@ final class DisplayCriteriaController { category: .engine ) } - if case .unchanged = Self.applyOutcome(didApply: didApply, last: lastApplied, target: target) { + let managerHoldsCriteria = displayManager.preferredDisplayCriteria != nil + if didApply, !managerHoldsCriteria { + EngineLog.emit( + "[DisplayCriteria] criteria were cleared outside the engine since its last SET; writing again", + category: .engine + ) + } + if case .unchanged = Self.applyOutcome(didApply: didApply, last: lastApplied, target: target, + managerHoldsCriteria: managerHoldsCriteria) { // Keep lastCriteriaWasHDR consistent with the still-active criteria for any waitForSwitch classification. lastCriteriaWasHDR = isHDR EngineLog.emit( @@ -826,6 +876,39 @@ final class DisplayCriteriaController { #endif } + /// AE#667: whether a switch this load was armed for has been seen to start and not yet to end. + /// + /// Read from the notifications, not from `isDisplayModeSwitchInProgress` alone, because the flag is + /// documented as unreliable around a write and sticks `true` on panels whose DV switch never reports. + /// A switch nobody saw start answers false, so a caller waiting on this never waits for an end that + /// cannot be announced. + func observedSwitchIsRunning() -> Bool { + #if os(tvOS) + guard let window = resolveWindow() else { return false } + let snapshot = observation.snapshot() + guard Self.recordIsFreshEvidence(recordGeneration: snapshot.generation, + lastSpentGeneration: spentArmGeneration) else { return false } + let observed = Self.observedSwitch( + startedAtNanos: snapshot.startedAt, + endedAtNanos: snapshot.endedAt, + gateEntryNanos: DispatchTime.now().uptimeNanoseconds, + switchInProgress: window.avDisplayManager.isDisplayModeSwitchInProgress) + if case .running = observed { return true } + return false + #else + return false + #endif + } + + /// AE#667: the system's in-progress flag, for a decision that may only be made more cautious by it. + var displayModeSwitchInProgress: Bool { + #if os(tvOS) + resolveWindow()?.avDisplayManager.isDisplayModeSwitchInProgress ?? false + #else + false + #endif + } + /// Block until the panel settles its HDR mode negotiation, bounded so an /// unobservable switch can't stall the first frame. /// @@ -848,9 +931,13 @@ final class DisplayCriteriaController { /// `consumesRecord: false` leaves the observation readable for the load's second gate. The pre-flight /// passes it: it waits an HDR switch out before the item is built, and the play gate that follows is /// entitled to the same start/end timestamps (#339). + /// + /// `isCurrent` answers whether the load this gate holds is still the engine's; the wait is abandoned + /// at the next tick once it is not, or once its task is cancelled (audit LIF-106). func waitForSwitch(startGrace: StartGrace = .full, consumesRecord: Bool = true, - settleCap: SettleCap = .standard) async { + settleCap: SettleCap = .standard, + isCurrent: () -> Bool = { true }) async { #if os(tvOS) guard startGrace != .skip else { return } guard let window = resolveWindow() else { return } @@ -954,7 +1041,10 @@ final class DisplayCriteriaController { break } isFirstPoll = false - try? await Task.sleep(for: .milliseconds(10)) + guard await Self.gateTick(milliseconds: 10, isCurrent: isCurrent) else { + EngineLog.emit("[DisplayCriteria] gate abandoned in the start phase after \(Self.elapsedMs(since: entry))ms: the load it held was cancelled or superseded", category: .engine) + return + } } // Time spent, not the budget: the polls carry scheduler overhead, and everything downstream is // reported relative to this (#49). @@ -973,10 +1063,15 @@ final class DisplayCriteriaController { // switch is unobservable to the app can't gate the first frame the way the // old fixed 5s poll did. // What this gate may spend waiting for the end, given who is waiting on it (`settleCapMs`). - let capMs = Self.settleCapMs( - cap: settleCap, - startRecorded: gateSnapshot.startedAt != nil || observation.hasNewStart(since: gateSnapshot)) let stage2Entry = DispatchTime.now() + let cap = Self.observedEndCap( + settleCap: settleCap, + startRecorded: gateSnapshot.startedAt != nil || observation.hasNewStart(since: gateSnapshot), + armGeneration: gateSnapshot.generation, + nowNanos: stage2Entry.uptimeNanoseconds, + budget: observedEndBudget) + observedEndBudget = cap.budget + let capMs = cap.capMs func timing() -> String { // The panel's own switch duration whenever both notifications were seen. This is the number the // `.brief` premise ("engine rate-only writes settle sub-second") has never been checked against @@ -994,7 +1089,10 @@ final class DisplayCriteriaController { // multiple after a dwell can be asked how long it has actually held it. var cadenceHistory = PanelCadenceHistory() while !Self.isBudgetSpent(elapsedMs: Self.elapsedMs(since: stage2Entry), budgetMs: capMs) { - try? await Task.sleep(for: .milliseconds(50)) + guard await Self.gateTick(milliseconds: 50, isCurrent: isCurrent) else { + EngineLog.emit("[DisplayCriteria] gate abandoned while waiting for the end (\(timing())): the load it held was cancelled or superseded", category: .engine) + return + } let stage2Ms = Self.elapsedMs(since: stage2Entry) let relation = panelRelation() cadenceHistory = Self.extendCadenceHistory(cadenceHistory, relation: relation, nowMs: stage2Ms) diff --git a/Sources/AetherEngine/FFmpegErrorConstants.swift b/Sources/AetherEngine/FFmpegErrorConstants.swift index ba5e01a1b..6f4c22da7 100644 --- a/Sources/AetherEngine/FFmpegErrorConstants.swift +++ b/Sources/AetherEngine/FFmpegErrorConstants.swift @@ -9,6 +9,9 @@ enum FFmpegErr { static let eagain: Int32 = -35 /// `AVERROR_INVALIDDATA` = FFERRTAG('I','N','D','A') = -0x41444E49. static let invalidData: Int32 = -0x41444E49 + /// `AVERROR_EXIT` = FFERRTAG('E','X','I','T') = -0x54495845: "immediate exit was requested", what + /// libavformat answers when its interrupt callback fires. + static let exit: Int32 = -0x54495845 /// `AVERROR(EINVAL)`; EINVAL is POSIX 22 on Apple platforms. Some decoders (notably `dca` on a /// DTS-HD MA XLL frame that residual-codes channels without a usable core) reject a single packet /// with this while staying usable for the next one (#64). diff --git a/Sources/AetherEngine/FrameExtractor/FrameCache.swift b/Sources/AetherEngine/FrameExtractor/FrameCache.swift index e8391587e..8a44738b7 100644 --- a/Sources/AetherEngine/FrameExtractor/FrameCache.swift +++ b/Sources/AetherEngine/FrameExtractor/FrameCache.swift @@ -34,7 +34,8 @@ final class FrameCache { // Thumbnails floor to grid; snapshots round to nearest so a value within // half a bucket of a stored position resolves to the same entry. let rounded = mode == .thumbnail ? scaled.rounded(.down) : scaled.rounded() - return Int(rounded) + // Audit BIT-105: `Int(_:)` traps past `Int.max`; every such position shares the last bucket. + return rounded < 0x1p62 ? Int(rounded) : 1 << 62 } func get(mode: FrameMode, seconds: Double) -> CGImage? { diff --git a/Sources/AetherEngine/FrameExtractor/FrameDecodeContext.swift b/Sources/AetherEngine/FrameExtractor/FrameDecodeContext.swift index 34385129a..ce40de84e 100644 --- a/Sources/AetherEngine/FrameExtractor/FrameDecodeContext.swift +++ b/Sources/AetherEngine/FrameExtractor/FrameDecodeContext.swift @@ -1,6 +1,7 @@ import Foundation import CoreGraphics import CoreMedia +import CoreVideo import AetherLibavformat import AetherLibavcodec import AetherLibavutil @@ -81,6 +82,14 @@ final class FrameDecodeContext: @unchecked Sendable { return false } + /// Whether this open may ask VideoToolbox. A no-base-layer Dolby Vision title stays on software + /// (audit BIT-102): VideoToolbox hands 10-bit 4:2:0 over as P010, `DolbyVisionStillConverter` accepts + /// only `yuv420p10le`, and the IPT-PQ-C2 planes then went through the BT.2020 tone map, which is the + /// green / magenta cast #103 fixed. + static func stillUsesHardware(allows: Bool, disabled: Bool, dvNoBaseLayer: Bool) -> Bool { + allows && !disabled && !dvNoBaseLayer + } + /// Thread budget for the disposable still/thumbnail decoder. Capped well below the /// core count so it cannot grab every core at playback's QoS and starve the /// real-time software decode (and, with subs on, the subtitle side-demuxer) on a @@ -214,7 +223,8 @@ final class FrameDecodeContext: @unchecked Sendable { throw FrameDecodeError.unsupportedCodec } var openedWithHardware = false - if allowsHardwareDecode, !hardwareDecodeDisabled { + if Self.stillUsesHardware(allows: allowsHardwareDecode, disabled: hardwareDecodeDisabled, + dvNoBaseLayer: isDolbyVisionNoBaseLayer) { let deviceResult = av_hwdevice_ctx_create( &hardwareDeviceContext, AV_HWDEVICE_TYPE_VIDEOTOOLBOX, nil, nil, 0) if deviceResult >= 0, hardwareDeviceContext != nil { @@ -404,8 +414,8 @@ final class FrameDecodeContext: @unchecked Sendable { demuxer.seek(to: seekSeconds) - guard timeBase.num > 0 else { return nil } - let targetPTS = Int64((seekSeconds * Double(timeBase.den)) / Double(timeBase.num)) + // A target the stream's own time base cannot hold is no position in it (audit BIT-105). + guard let targetPTS = Demuxer.ticks(forSeconds: seekSeconds, timeBase: timeBase) else { return nil } var frame: UnsafeMutablePointer? = av_frame_alloc() guard frame != nil else { return nil } @@ -615,6 +625,29 @@ final class FrameDecodeContext: @unchecked Sendable { return AVRational(num: 1, den: 1) } + /// The sws coefficient table for the matrix `ColorAttachments.presented` resolves, so a gap is + /// filled here exactly as it is on the buffer playback shows. + static func swsColorspace(for description: ColorDescription) -> Int32 { + switch ColorAttachments.presented(description).matrix { + case kCVImageBufferYCbCrMatrix_ITU_R_601_4: SWS_CS_ITU601 + case kCVImageBufferYCbCrMatrix_ITU_R_2020: SWS_CS_BT2020 + case kCVImageBufferYCbCrMatrix_SMPTE_240M_1995: SWS_CS_SMPTE240M + default: SWS_CS_ITU709 + } + } + + /// Full range when the frame says so, or when its pixel format is one of the JPEG variants that + /// carry the range in the format rather than in `color_range`. + static func isFullRange(_ frame: UnsafePointer) -> Bool { + if frame.pointee.color_range == AVCOL_RANGE_JPEG { return true } + switch AVPixelFormat(rawValue: frame.pointee.format) { + case AV_PIX_FMT_YUVJ420P, AV_PIX_FMT_YUVJ422P, AV_PIX_FMT_YUVJ444P, AV_PIX_FMT_YUVJ440P, AV_PIX_FMT_YUVJ411P: + return true + default: + return false + } + } + /// sws_scale the frame to RGBA at `targetWidth` (height from source aspect) into an /// owned-buffer CGImage. Mirrors the sws tuple-pointer dance in SoftwareVideoDecoder. private func convertToCGImage(frame: UnsafeMutablePointer, targetWidth: Int) -> CGImage? { @@ -637,6 +670,14 @@ final class FrameDecodeContext: @unchecked Sendable { Int32(SWS_BILINEAR.rawValue), nil, nil, nil ) guard swsContext != nil else { return nil } + // Without this sws converts every picture with its BT.601 default, which put a 1080p + // BT.709 colour bar up to 33 levels off in the still while playback showed it right. + // The matrix comes from the rule the displayed buffer is tagged by, so both agree. + let description = ColorDescription(frame: frame) + let coefficients = sws_getCoefficients(Self.swsColorspace(for: description)) + sws_setColorspaceDetails( + swsContext, coefficients, Self.isFullRange(frame) ? 1 : 0, + coefficients, 1, 0, 1 << 16, 1 << 16) let bytesPerRow = dstW * 4 var rgba = [UInt8](repeating: 0, count: bytesPerRow * dstH) @@ -674,7 +715,8 @@ final class FrameDecodeContext: @unchecked Sendable { let data = Data(rgba) guard let provider = CGDataProvider(data: data as CFData), - let colorSpace = CGColorSpace(name: CGColorSpace.sRGB) else { + let colorSpace = ColorAttachments.colorSpace(for: ColorAttachments.presented(description)) + ?? CGColorSpace(name: CGColorSpace.sRGB) else { return nil } // sws_scale RGBA yields opaque pixels (alpha 0xFF), so alpha is ignorable not premultiplied. diff --git a/Sources/AetherEngine/FrameExtractor/FrameExtractor.swift b/Sources/AetherEngine/FrameExtractor/FrameExtractor.swift index 01a53904d..83c8e3be1 100644 --- a/Sources/AetherEngine/FrameExtractor/FrameExtractor.swift +++ b/Sources/AetherEngine/FrameExtractor/FrameExtractor.swift @@ -119,7 +119,8 @@ public actor FrameExtractor { // MARK: - Core private func produce(at seconds: Double, mode: FrameMode, targetWidth: Int, maxSize: CGSize?) async -> CGImage? { - guard !isShutDown else { return nil } + // Audit BIT-105: a 0/0 scrub fraction or `CMTime.indefinite.seconds` is a host bug, not a position. + guard !isShutDown, seconds.isFinite, seconds >= 0 else { return nil } if let hit = cache.get(mode: mode, seconds: seconds) { scheduleIdleClose() return hit diff --git a/Sources/AetherEngine/IO/DataIOReader.swift b/Sources/AetherEngine/IO/DataIOReader.swift index abf473448..8b2db4078 100644 --- a/Sources/AetherEngine/IO/DataIOReader.swift +++ b/Sources/AetherEngine/IO/DataIOReader.swift @@ -1,43 +1,72 @@ import Foundation -/// In-memory `IOReader` over an immutable `Data` blob. Used by the scrub-thumbnail path: engine composes init.mp4 + cached segment into one buffer and demuxes via `FrameExtractor(reader:)`. NSLock makes read/seek safe off the demux thread. +/// In-memory `IOReader` over immutable `Data` parts read back to back as one stream. The scrub-thumbnail +/// path hands it init.mp4 plus a memory-mapped cached segment, so the segment is never copied into the heap +/// (audit SEG-2: concatenating the two made two full heap copies per still, one retained per LRU entry). +/// NSLock makes read/seek safe off the demux thread. final class DataIOReader: IOReader, @unchecked Sendable { - private let data: Data + private let parts: [Data] + private let partStarts: [Int] + private let totalCount: Int private var position = 0 private let lock = NSLock() - init(data: Data) { - self.data = data + convenience init(data: Data) { + self.init(parts: [data]) + } + + init(parts: [Data]) { + self.parts = parts + var starts: [Int] = [] + var offset = 0 + for part in parts { + starts.append(offset) + offset += part.count + } + self.partStarts = starts + self.totalCount = offset } func read(_ buffer: UnsafeMutablePointer?, size: Int32) -> Int32 { guard let buffer, size > 0 else { return -1 } lock.lock() defer { lock.unlock() } - guard position < data.count else { return 0 } // EOF - let n = min(Int(size), data.count - position) - data.copyBytes( - to: UnsafeMutableBufferPointer(start: buffer, count: n), - from: position..<(position + n) - ) - position += n - return Int32(n) + guard position < totalCount else { return 0 } // EOF + let n = min(Int(size), totalCount - position) + var copied = 0 + var partIndex = (partStarts.lastIndex(where: { $0 <= position }) ?? 0) + while copied < n, partIndex < parts.count { + let part = parts[partIndex] + let offsetInPart = position + copied - partStarts[partIndex] + let take = min(n - copied, part.count - offsetInPart) + if take > 0 { + let lower = part.startIndex + offsetInPart + part.copyBytes( + to: UnsafeMutableBufferPointer(start: buffer + copied, count: take), + from: lower..<(lower + take) + ) + copied += take + } + partIndex += 1 + } + position += copied + return Int32(copied) } func seek(offset: Int64, whence: Int32) -> Int64 { // AVSEEK_SIZE (65536): report total length, do not move. - if whence == 65536 { return Int64(data.count) } + if whence == 65536 { return Int64(totalCount) } lock.lock() defer { lock.unlock() } let target: Int switch whence { case SEEK_SET: target = Int(offset) case SEEK_CUR: target = position + Int(offset) - case SEEK_END: target = data.count + Int(offset) + case SEEK_END: target = totalCount + Int(offset) default: return -1 } guard target >= 0 else { return -1 } - position = min(target, data.count) + position = min(target, totalCount) return Int64(position) } diff --git a/Sources/AetherEngine/IO/HLSIngest/BoundedPlaylistFetch.swift b/Sources/AetherEngine/IO/HLSIngest/BoundedPlaylistFetch.swift new file mode 100644 index 000000000..cf9368ca2 --- /dev/null +++ b/Sources/AetherEngine/IO/HLSIngest/BoundedPlaylistFetch.swift @@ -0,0 +1,191 @@ +import Foundation + +/// Reads a playlist body with its size cap enforced while it arrives (audit NET-10). +/// +/// `session.data(for:)` buffers the whole body before a caller can look at its size, so a cap tested +/// afterwards bounded nothing: an origin answering with an endless body kept the fetch growing until +/// the resource timeout, or until jetsam. Here a declared length over the cap is refused before a +/// byte is read, and an undeclared one is cut off at the cap. +enum BoundedPlaylistFetch { + + /// The body and response of `request`. A non-2xx answer comes back with an empty body: every + /// caller throws on the status alone, so its body is not worth reading. + static func data(for request: URLRequest, session: URLSession, limit: Int) async throws + -> (Data, URLResponse) + { + do { + return try await BoundedFetch.data(for: request, session: session, limit: limit) + } catch let error as BoundedFetch.Exceeded { + throw HLSIngestError.playlistInvalid(reason: "playlist exceeds \(error.limit) bytes") + } + } +} + +/// One bounded HTTP body fetch for everything that holds a whole body in memory: playlists, key +/// files, transport-stream segments, WebVTT segments (audit NET-112, FEA-107, DEC-103, SUB-106). +/// +/// A delegate rather than `session.data(for:)`, which buffers the whole body before a caller sees its +/// size, and rather than iterating `AsyncBytes`, which delivered one byte per step into a per-byte +/// `Data.append`, about 16 ms per MB (audit NET-113). The response head is judged before any body is +/// read: a non-2xx answer or a declared length over `limit` ends the transfer there. The body then +/// arrives in the chunks the session delivers and the transfer is cancelled the moment it passes +/// `limit`, so the peak is the limit plus one delivery. +enum BoundedFetch { + + struct Exceeded: Error, Equatable { + let limit: Int + } + + /// An AES-128 key is 16 bytes; the callers keep their own `== 16` check. + static let keyLimit = 64 + + /// One WebVTT segment of a live subtitle rendition carries a few cues. + static let webVTTSegmentLimit = 1024 * 1024 + + /// What a transport-stream segment of `seconds` may weigh: 20 MB/s of its own duration, between + /// 32 MiB and 256 MiB. UHD remux HLS runs 6 to 10 s segments at about 100 Mbps (75 to 125 MB), so + /// a small fixed cap would refuse the content the path exists for. + static func segmentLimit(forDuration seconds: Double) -> Int { + let floor = 32.0 * 1024 * 1024 + let ceiling = 256.0 * 1024 * 1024 + guard seconds.isFinite, seconds > 0 else { return Int(floor) } + return Int(min(ceiling, max(floor, seconds * 20_000_000))) + } + + /// The body and response of `request`. A non-2xx answer comes back with an empty body, since every + /// caller decides on the status alone. Throws `Exceeded` for a body over `limit`. + static func data(for request: URLRequest, session: URLSession, limit: Int) async throws + -> (Data, URLResponse) + { + let delegate = BoundedBodyDelegate(limit: limit) + defer { withExtendedLifetime(delegate) {} } + let task = session.dataTask(with: request) + task.delegate = delegate + let outcome: BoundedBodyDelegate.Outcome = try await withTaskCancellationHandler { + try await withCheckedThrowingContinuation { continuation in + // Installed through the delegate's own lock, and it replays an outcome that has already + // landed: `onCancel` runs on the spot for a task that is already cancelled, so the + // transfer can end before this line. + delegate.installOutcomeHandler { continuation.resume(returning: $0) } + task.resume() + } + } onCancel: { + task.cancel() + } + switch outcome { + case .body(let data, let response): return (data, response) + case .refused(let response): return (Data(), response) + case .exceeded: throw Exceeded(limit: limit) + case .failed(let error): throw error + } + } +} + +private final class BoundedBodyDelegate: EngineTLS.RedirectScrubbingDelegate, URLSessionDataDelegate, + @unchecked Sendable +{ + enum Outcome { + case body(Data, URLResponse) + case refused(URLResponse) + case exceeded + case failed(Error) + } + + private enum Verdict { + case refused + case exceeded + } + + private let limit: Int + private var buffer = Data() + private var response: URLResponse? + private var verdict: Verdict? + + /// Guards the handoff between the caller, which installs the handler, and the session's delegate + /// queue, which produces the outcome. Either can be first. + private let handoff = NSLock() + private var onOutcome: ((Outcome) -> Void)? + private var landed: Outcome? + + init(limit: Int) { + self.limit = limit + super.init() + } + + func installOutcomeHandler(_ handler: @escaping (Outcome) -> Void) { + handoff.lock() + if let landed { + self.landed = nil + handoff.unlock() + handler(landed) + return + } + onOutcome = handler + handoff.unlock() + } + + private func deliver(_ outcome: Outcome) { + handoff.lock() + guard let handler = onOutcome else { + landed = outcome + handoff.unlock() + return + } + onOutcome = nil + handoff.unlock() + handler(outcome) + } + + func urlSession(_ session: URLSession, task: URLSessionTask, + didReceive challenge: URLAuthenticationChallenge, + completionHandler: @escaping (URLSession.AuthChallengeDisposition, URLCredential?) -> Void) { + EngineTLS.resolve(challenge, completionHandler: completionHandler) + } + + func urlSession(_ session: URLSession, dataTask: URLSessionDataTask, didReceive response: URLResponse, + completionHandler: @escaping (URLSession.ResponseDisposition) -> Void) { + self.response = response + let status = (response as? HTTPURLResponse)?.statusCode ?? -1 + guard (200..<300).contains(status) else { + verdict = .refused + completionHandler(.cancel) + return + } + let declared = response.expectedContentLength + guard declared <= Int64(limit) else { + verdict = .exceeded + completionHandler(.cancel) + return + } + if declared > 0 { buffer.reserveCapacity(Int(min(declared, 8 * 1024 * 1024))) } + completionHandler(.allow) + } + + func urlSession(_ session: URLSession, dataTask: URLSessionDataTask, didReceive data: Data) { + guard verdict == nil else { return } + guard buffer.count + data.count <= limit else { + verdict = .exceeded + buffer = Data() + dataTask.cancel() + return + } + buffer.append(data) + } + + func urlSession(_ session: URLSession, task: URLSessionTask, didCompleteWithError error: Error?) { + switch verdict { + case .refused?: + if let response { deliver(.refused(response)) } else { deliver(.failed(URLError(.badServerResponse))) } + case .exceeded?: + deliver(.exceeded) + case nil: + if let error { + deliver(.failed(error)) + } else if let response { + deliver(.body(buffer, response)) + } else { + deliver(.failed(URLError(.badServerResponse))) + } + } + } +} diff --git a/Sources/AetherEngine/IO/HLSIngest/ByteFIFO.swift b/Sources/AetherEngine/IO/HLSIngest/ByteFIFO.swift index 9d2f5d295..7ee1e936b 100644 --- a/Sources/AetherEngine/IO/HLSIngest/ByteFIFO.swift +++ b/Sources/AetherEngine/IO/HLSIngest/ByteFIFO.swift @@ -1,10 +1,19 @@ import Foundation -/// Bounded blocking byte queue between the HLS fetch loop (writer) and demux thread (reader). NSCondition-based; `finish()` signals EOF, `cancel()` signals error. Capacity is a soft bound: write blocks while at/above capacity then appends the whole chunk (overshoot = at most one chunk). Storage uses `subdata` re-base on consume, never `removeFirst` (Data.removeFirst slice-leak, AetherEngine 70430de). +/// Bounded blocking byte queue between the HLS fetch loop (writer) and demux thread (reader). NSCondition-based; `finish()` signals EOF, `cancel()` signals error. Capacity is a soft bound: write blocks while at/above capacity then appends the whole chunk (overshoot = at most one chunk). +/// +/// Storage is a queue of whole written chunks plus a head offset into the front one, the same shape +/// `HLSVODIngestReader` uses: a consumed chunk is dropped with `Array.removeFirst` (an Array op, not +/// `Data.removeFirst`'s slice-leak, AetherEngine 70430de). Earlier this was one `Data` re-based with +/// `subdata` on every read, which re-copies whatever is still queued: draining a full 16 MB FIFO in +/// 256 KB reads did about 64 reads averaging an 8 MB copy each, roughly 512 MB of memcpy for 16 MB +/// delivered (audit NET-8). final class ByteFIFO: @unchecked Sendable { private let capacity: Int private let condition = NSCondition() - private var storage = Data() + private var queue: [Data] = [] + private var headOffset = 0 + private var pendingCount = 0 private var finished = false private var cancelled = false /// Callers parked in `read` or `write`, guarded by the condition above. A reader from another @@ -18,6 +27,14 @@ final class ByteFIFO: @unchecked Sendable { return parked } + /// AE#684: nothing queued and a caller waiting for more. A writer only parks at capacity, so with + /// the queue empty the parked caller is the reader, and everything written so far is consumed. + var isEmptyWithReaderParked: Bool { + condition.lock() + defer { condition.unlock() } + return queue.isEmpty && parked > 0 && !finished && !cancelled + } + init(capacity: Int) { self.capacity = capacity } @@ -27,12 +44,13 @@ final class ByteFIFO: @unchecked Sendable { condition.lock() defer { condition.unlock() } parked += 1 - while storage.count >= capacity && !finished && !cancelled { + while pendingCount >= capacity && !finished && !cancelled { condition.wait() } parked -= 1 if finished || cancelled { return false } - storage.append(data) + queue.append(data) + pendingCount += data.count condition.broadcast() return true } @@ -42,17 +60,32 @@ final class ByteFIFO: @unchecked Sendable { condition.lock() defer { condition.unlock() } parked += 1 - while storage.isEmpty && !finished && !cancelled { + while queue.isEmpty && !finished && !cancelled { condition.wait() } parked -= 1 if cancelled { return -1 } - if storage.isEmpty { return 0 } // finished + drained - let n = min(maxLength, storage.count) - storage.copyBytes(to: UnsafeMutableBufferPointer(start: buffer, count: n), from: 0.. MPEGTransportStreamCodecProbe.Verdict { do { return try await classifySegmentHead( - url: url, httpHeaders: httpHeaders, session: session ?? sharedSession + url: url, + httpHeaders: RedirectHeaderPolicy.scoped(httpHeaders, grantedFor: credentialOrigin, sentTo: url), + session: session ?? sharedSession ) } catch { - EngineLog.emit("[HLSCarriageProbe] carriage probe inconclusive: \(error)", category: .engine) + EngineLog.emit( + "[HLSCarriageProbe] carriage probe inconclusive: \(EngineLog.summary(of: error))", + category: .engine) return .inconclusive } } @@ -104,7 +117,7 @@ enum HLSCarriageProbe { return verdict case .needsSegmentHead(let segmentURL): return await classifyDeferredSegmentHead( - url: segmentURL, httpHeaders: httpHeaders, session: session + url: segmentURL, httpHeaders: httpHeaders, credentialOrigin: playlistURL, session: session ) } } @@ -152,7 +165,8 @@ enum HLSCarriageProbe { advertisesFragmentedMP4OnlyVideo: Bool, session: URLSession ) async throws -> PlaylistEvidence { - let (root, rootURL) = try await fetchPlaylist(playlistURL, httpHeaders: httpHeaders, session: session) + let (root, rootURL) = try await fetchPlaylist( + playlistURL, httpHeaders: httpHeaders, credentialOrigin: playlistURL, session: session) let media: HLSMediaPlaylist let mediaURL: URL switch root { @@ -165,7 +179,7 @@ enum HLSCarriageProbe { return .settled(.inconclusive) } let (selected, selectedURL) = try await fetchPlaylist( - variantURL, httpHeaders: httpHeaders, session: session + variantURL, httpHeaders: httpHeaders, credentialOrigin: playlistURL, session: session ) guard case .media(let value) = selected else { return .settled(.inconclusive) } media = value @@ -193,14 +207,18 @@ enum HLSCarriageProbe { private static func fetchPlaylist( _ url: URL, httpHeaders: [String: String], + credentialOrigin: URL, session: URLSession ) async throws -> (HLSPlaylist, URL) { - let (data, response) = try await session.data(for: makeRequest(url, httpHeaders: httpHeaders)) + let request = makeRequest( + url, httpHeaders: RedirectHeaderPolicy.scoped(httpHeaders, grantedFor: credentialOrigin, sentTo: url)) + let (data, response) = try await BoundedPlaylistFetch.data( + for: request, session: session, limit: maximumPlaylistBytes) let status = (response as? HTTPURLResponse)?.statusCode ?? -1 guard (200..<300).contains(status) else { throw HLSIngestError.playlistUnreachable(status: status) } - guard data.count <= maximumPlaylistBytes, let text = String(data: data, encoding: .utf8) else { + guard let text = String(data: data, encoding: .utf8) else { throw HLSIngestError.playlistInvalid(reason: "playlist is not bounded UTF-8") } return (try HLSPlaylistParser.parse(text), response.url ?? url) diff --git a/Sources/AetherEngine/IO/HLSIngest/HLSLiveIngestReader.swift b/Sources/AetherEngine/IO/HLSIngest/HLSLiveIngestReader.swift index 4f0a216f1..bf595ee5f 100644 --- a/Sources/AetherEngine/IO/HLSIngest/HLSLiveIngestReader.swift +++ b/Sources/AetherEngine/IO/HLSIngest/HLSLiveIngestReader.swift @@ -18,8 +18,14 @@ public final class HLSLiveIngestReader: IOReader, LiveIngestSourceInfo, @uncheck private let playlistURL: URL private let httpHeaders: [String: String] + /// The URL the host gave `httpHeaders` for. A companion inherits its parent's, since its own + /// playlist URL is one the master named (audit NET-7). + let credentialOrigin: URL private let role: Role private let fifo = ByteFIFO(capacity: 16 * 1024 * 1024) + /// Wider than the VOD reader's 2 MB: a live window with hours of DVR at short segments is a + /// legitimately long playlist, and this one is refetched every few seconds rather than once. + static let maximumPlaylistBytes = 8 * 1024 * 1024 private let session: URLSession private var ingestTask: Task? private let startLock = NSLock() @@ -35,6 +41,11 @@ public final class HLSLiveIngestReader: IOReader, LiveIngestSourceInfo, @uncheck /// AE#447: longest EXTINF the upstream has actually served, the measured counterpart to /// `_upstreamTargetDuration`. Monotonic; read via `upstreamSegmentDurationSeconds`. private var _upstreamSegmentDurationSeconds: Double? + /// AE#684: summed EXTINF of the join batch. Written with the join line, before its bytes flow. + private var _joinBacklogSeconds: Double? + /// AE#684: the join batch has been committed to the FIFO in full / has been seen consumed. + private var _joinBatchCommitted = false + private var _joinSpent = false /// Installed by the resolver before the first FIFO byte; nil = muxed audio. private var _companionAudioReader: HLSLiveIngestReader? /// AE#359: SUBTITLES renditions of the picked variant, resolved to absolute URLs. Metadata only. @@ -59,6 +70,17 @@ public final class HLSLiveIngestReader: IOReader, LiveIngestSourceInfo, @uncheck /// fetches saturate the link while bounding in-memory segment bytes to the window size. static let maxConcurrentSegmentFetches = 4 + /// AE#678: the prefetch window narrowed to the origin's request budget. A host that declares + /// `maxConcurrentSourceRequests = 1` for a single-connection provider meant every request this + /// reader makes, and four parallel segment fetches were four connections to it. + static func segmentFetchConcurrency(budgetLimit: Int?) -> Int { + guard let budgetLimit else { return maxConcurrentSegmentFetches } + return max(1, min(maxConcurrentSegmentFetches, budgetLimit)) + } + + /// The URL the host loaded, which is what `OriginRequestBudget` keys the declared ceiling on. + var budgetOriginURL: URL { playlistURL } + /// First-segment classification latch; touched only from the ingest task's ordered commit path. private var sniffedFirstSegment = false @@ -79,6 +101,47 @@ public final class HLSLiveIngestReader: IOReader, LiveIngestSourceInfo, @uncheck startLock.withLock { _upstreamSegmentDurationSeconds } } + var joinBacklogSeconds: Double? { + startLock.withLock { _joinBacklogSeconds } + } + + var joinIsSpent: Bool { + let (spent, committed, companion) = startLock.withLock { + (_joinSpent, _joinBatchCommitted, _companionAudioReader) + } + if spent { return true } + guard Self.pumpJoinIsSpent( + main: (committed, fifo.isEmptyWithReaderParked), + companion: companion?.pumpJoinState + ) else { return false } + startLock.withLock { _joinSpent = true } + return true + } + + /// One reader's half of `joinIsSpent`: whether its ingest ever started, whether its join batch + /// is committed, and whether it is empty with its consumer parked on it. + var pumpJoinState: (started: Bool, committed: Bool, parked: Bool) { + let (isStarted, committed) = startLock.withLock { (started, _joinBatchCommitted) } + return (isStarted, committed, fifo.isEmptyWithReaderParked) + } + + /// AE#684: the fact is the PUMP's, not one reader's. With a demuxed audio rendition the cutter + /// merges two readers on one thread and parks on whichever runs dry first, and a rendition whose + /// segments end a little before the video's runs dry first every time: the video reader is then + /// never parked, and a fact read off it alone never becomes true (measured: the full seal over a + /// window that cannot hold it, 2.2 s to first picture under `.fastZap` and 10.4 s under + /// `.standard`, where 7.25.1 took 0.18 s). So both join batches have to be committed, and the + /// cutter has to be parked on EITHER empty reader: it holds a packet of the other one it cannot + /// place until the dry one delivers, so nothing more is cut either way. A companion that was + /// never started is not being read at all and does not count. + static func pumpJoinIsSpent(main: (committed: Bool, parked: Bool), + companion: (started: Bool, committed: Bool, parked: Bool)?) -> Bool { + guard main.committed else { return false } + guard let companion, companion.started else { return main.parked } + guard companion.committed else { return false } + return main.parked || companion.parked + } + public var closedLiveCadenceSeconds: Double? { startLock.withLock { _cadenceMeter.closedCadence } } @@ -134,7 +197,8 @@ public final class HLSLiveIngestReader: IOReader, LiveIngestSourceInfo, @uncheck /// `httpHeaders` ride on every fetch (playlist, segment, AES key) and inherit to the companion audio /// reader, so header-enforcing IPTV origins (Referer / User-Agent / Authorization, #119) accept the - /// ingest the same way they accept the AVPlayer bypass (AetherEngine#168). + /// ingest the same way they accept the AVPlayer bypass (AetherEngine#168). Credential headers go + /// only to `playlistURL`'s origin with no TLS downgrade (audit NET-7). public convenience init(playlistURL: URL, httpHeaders: [String: String]) { self.init(playlistURL: playlistURL, httpHeaders: httpHeaders, role: .mainVideo) } @@ -148,9 +212,11 @@ public final class HLSLiveIngestReader: IOReader, LiveIngestSourceInfo, @uncheck return HLSLiveIngestReader(playlistURL: playlistURL, httpHeaders: httpHeaders, role: .mainVideo) } - init(playlistURL: URL, httpHeaders: [String: String] = [:], role: Role) { + init(playlistURL: URL, httpHeaders: [String: String] = [:], role: Role, + credentialOrigin: URL? = nil) { self.playlistURL = playlistURL self.httpHeaders = httpHeaders + self.credentialOrigin = credentialOrigin ?? playlistURL self.role = role let config = URLSessionConfiguration.ephemeral config.timeoutIntervalForRequest = 10 @@ -281,8 +347,9 @@ public final class HLSLiveIngestReader: IOReader, LiveIngestSourceInfo, @uncheck startLock.withLock { _joinWallClock = joinDate } } let backlog = fresh.reduce(0.0) { $0 + $1.duration } + startLock.withLock { _joinBacklogSeconds = backlog } EngineLog.emit( - "[HLSIngest] joined \(fresh.count) segment(s), ~\(Int(backlog))s behind the live edge" + "[HLSIngest] joined \(fresh.count) segment(s), ~\(String(format: "%.0f", backlog))s behind the live edge" + " pdt=\(fresh.first?.programDateTime.map { "\($0)" } ?? "nil")", category: .engine ) @@ -292,6 +359,7 @@ public final class HLSLiveIngestReader: IOReader, LiveIngestSourceInfo, @uncheck guard try await ingestSegmentBatch(fresh, mediaURL: mediaURL) else { return // FIFO closed underneath us } + if isJoin { startLock.withLock { _joinBatchCommitted = true } } } if media.hasEndList { @@ -333,13 +401,15 @@ public final class HLSLiveIngestReader: IOReader, LiveIngestSourceInfo, @uncheck } return (segment, url) } + let window = Self.segmentFetchConcurrency( + budgetLimit: OriginRequestBudget.shared.limit(for: mediaURL)) return try await withThrowingTaskGroup(of: (Int, Data).self) { group -> Bool in var nextToSpawn = 0 var nextToCommit = 0 var ready: [Int: Data] = [:] while nextToSpawn < resolved.count, - nextToSpawn < nextToCommit + Self.maxConcurrentSegmentFetches { + nextToSpawn < nextToCommit + window { spawnFetch(into: &group, index: nextToSpawn, item: resolved[nextToSpawn], mediaURL: mediaURL) nextToSpawn += 1 } @@ -366,7 +436,7 @@ public final class HLSLiveIngestReader: IOReader, LiveIngestSourceInfo, @uncheck } } while nextToSpawn < resolved.count, - nextToSpawn < nextToCommit + Self.maxConcurrentSegmentFetches { + nextToSpawn < nextToCommit + window { spawnFetch(into: &group, index: nextToSpawn, item: resolved[nextToSpawn], mediaURL: mediaURL) nextToSpawn += 1 } @@ -385,7 +455,7 @@ public final class HLSLiveIngestReader: IOReader, LiveIngestSourceInfo, @uncheck mediaURL: URL ) { group.addTask { - let fetched = try await self.fetchSegment(item.url) + let fetched = try await self.fetchSegment(item.url, duration: item.segment.duration) guard !fetched.isEmpty, let crypt = item.segment.crypt else { return (index, fetched) } return (index, try await self.decryptSegment(fetched, crypt: crypt, against: mediaURL)) } @@ -479,7 +549,9 @@ public final class HLSLiveIngestReader: IOReader, LiveIngestSourceInfo, @uncheck + "starting companion reader on \(audioURL.lastPathComponent)", category: .engine ) - installCompanion(HLSLiveIngestReader(playlistURL: audioURL, httpHeaders: httpHeaders, role: .companionAudio)) + installCompanion(HLSLiveIngestReader( + playlistURL: audioURL, httpHeaders: httpHeaders, role: .companionAudio, + credentialOrigin: credentialOrigin)) } // AE#359: the variant's SUBTITLES group, resolved to absolute playlist URLs and published as // metadata. Nothing is fetched here; the host decides whether a subtitle track is ever wanted. @@ -545,10 +617,46 @@ public final class HLSLiveIngestReader: IOReader, LiveIngestSourceInfo, @uncheck try await Task.sleep(nanoseconds: UInt64(delay * 1_000_000_000)) } - /// Applies the configured origin headers to every ingest fetch. Internal for the header-contract tests. + /// AE#678: every ingest request is charged to `OriginRequestBudget` like the reader's and the relay's, + /// so the host's declared ceiling and a ceiling learned from a refusal bind this path too. The slot + /// wait blocks, so it runs off the cooperative pool; no ticket is ever held across another acquire. + private func budgeted( + _ url: URL, label: String, _ fetch: () async throws -> (Data, URLResponse) + ) async throws -> (Data, URLResponse) { + let ticket: OriginRequestBudget.Ticket? = await withCheckedContinuation { continuation in + DispatchQueue.global(qos: .userInitiated).async { [weak self] in + continuation.resume(returning: OriginRequestBudget.shared.acquire( + for: url, label: label, timeout: Self.budgetSlotWaitSeconds, + shouldAbort: { self?.isClosed ?? true })) + } + } + defer { OriginRequestBudget.shared.release(ticket) } + try Task.checkCancellation() + let (data, response) = try await fetch() + if let http = response as? HTTPURLResponse { + if let finalURL = http.url, finalURL != url { + OriginRequestBudget.shared.noteRedirect(from: url, to: finalURL) + } + if Self.refusalStatuses.contains(http.statusCode) { + OriginRequestBudget.shared.noteRefusal( + for: http.url ?? url, status: http.statusCode, + retryAfter: http.value(forHTTPHeaderField: "Retry-After").flatMap(TimeInterval.init)) + } + } + return (data, response) + } + + static let budgetSlotWaitSeconds: TimeInterval = 10 + static let refusalStatuses: Set = [429, 503, 509] + + private var isClosed: Bool { startLock.withLock { closed } } + + /// Applies the configured origin headers to every ingest fetch, credentials only where the host's + /// origin is (audit NET-7). Internal for the header-contract tests. func makeRequest(_ url: URL) -> URLRequest { var request = URLRequest(url: url) - for (field, value) in httpHeaders { + for (field, value) in RedirectHeaderPolicy.scoped( + httpHeaders, grantedFor: credentialOrigin, sentTo: url) { request.setValue(value, forHTTPHeaderField: field) } return request @@ -556,7 +664,10 @@ public final class HLSLiveIngestReader: IOReader, LiveIngestSourceInfo, @uncheck /// Fetch + parse a playlist. Returns parsed playlist and final URL after redirects (relative segment URIs resolve against it). private func fetchPlaylist(_ url: URL) async throws -> (HLSPlaylist, URL) { - let (data, response) = try await session.data(for: makeRequest(url)) + let (data, response) = try await budgeted(url, label: "ingest-playlist") { + try await BoundedPlaylistFetch.data( + for: self.makeRequest(url), session: self.session, limit: Self.maximumPlaylistBytes) + } let status = (response as? HTTPURLResponse)?.statusCode ?? -1 guard (200..<300).contains(status) else { throw HLSIngestError.playlistUnreachable(status: status) @@ -567,12 +678,21 @@ public final class HLSLiveIngestReader: IOReader, LiveIngestSourceInfo, @uncheck return (try HLSPlaylistParser.parse(text), response.url ?? url) } - private func fetchSegment(_ url: URL) async throws -> Data { + private func fetchSegment(_ url: URL, duration: Double) async throws -> Data { var lastStatus = -1 + let limit = BoundedFetch.segmentLimit(forDuration: duration) for attempt in 0..<3 { if Task.isCancelled { throw CancellationError() } do { - let (data, response) = try await session.data(for: makeRequest(url)) + let (data, response): (Data, URLResponse) + do { + (data, response) = try await budgeted(url, label: "ingest-segment") { + try await BoundedFetch.data(for: self.makeRequest(url), session: self.session, limit: limit) + } + } catch is BoundedFetch.Exceeded { + // Audit NET-112: an endless body is the origin's answer, not a blip to retry. + throw HLSIngestError.playlistInvalid(reason: "segment exceeds \(limit) bytes") + } lastStatus = (response as? HTTPURLResponse)?.statusCode ?? -1 if (200..<300).contains(lastStatus) { return data } if lastStatus == 404 { return Data() } // slid out of provider window; tracker advances regardless @@ -605,7 +725,16 @@ public final class HLSLiveIngestReader: IOReader, LiveIngestSourceInfo, @uncheck let cacheKey = url.absoluteString if let cached = keyCacheLock.withLock({ keyCache[cacheKey] }) { return cached } - let (data, response) = try await session.data(for: makeRequest(url)) + let data: Data + let response: URLResponse + do { + (data, response) = try await budgeted(url, label: "ingest-key") { + try await BoundedFetch.data( + for: self.makeRequest(url), session: self.session, limit: BoundedFetch.keyLimit) + } + } catch is BoundedFetch.Exceeded { + throw HLSIngestError.segmentDecryptFailed(reason: "key exceeds \(BoundedFetch.keyLimit) bytes") + } let status = (response as? HTTPURLResponse)?.statusCode ?? -1 guard (200..<300).contains(status) else { throw HLSIngestError.segmentDecryptFailed(reason: "key fetch HTTP \(status)") diff --git a/Sources/AetherEngine/IO/HLSIngest/HLSPlaylist.swift b/Sources/AetherEngine/IO/HLSIngest/HLSPlaylist.swift index a27082336..f5847bb43 100644 --- a/Sources/AetherEngine/IO/HLSIngest/HLSPlaylist.swift +++ b/Sources/AetherEngine/IO/HLSIngest/HLSPlaylist.swift @@ -81,10 +81,23 @@ enum HLSPlaylist: Equatable { /// Line-oriented RFC 8216 parser for the subset the live ingest needs. Pure (no I/O). enum HLSPlaylistParser { + /// Audit NET-105: the longest line any tag or URI in a real playlist comes to is a DATERANGE with + /// a SCTE-35 payload, a few KB. A line past this is not a playlist line, and refusing it keeps every + /// per-line helper, present and future, off a megabyte of one attribute soup. + static let maximumLineBytes = 64 * 1024 + static func parse(_ text: String) throws -> HLSPlaylist { - let lines = text - .split(separator: "\n", omittingEmptySubsequences: true) - .map { $0.trimmingCharacters(in: .whitespacesAndNewlines) } + // `split(separator: "\n")` compares whole Characters, and "\r\n" is one grapheme cluster in + // Swift, so a CRLF playlist came back as a single line (audit NET-2, NAT-3). `isNewline` is + // true for "\r\n", "\n" and "\r" alike. + let lines = try text + .split(whereSeparator: \.isNewline) + .map { line -> String in + guard line.utf8.count <= maximumLineBytes else { + throw HLSIngestError.playlistInvalid(reason: "playlist line exceeds \(maximumLineBytes) bytes") + } + return line.trimmingCharacters(in: .whitespacesAndNewlines) + } .filter { !$0.isEmpty } guard lines.first?.hasPrefix("#EXTM3U") == true else { throw HLSIngestError.playlistInvalid(reason: "missing #EXTM3U") @@ -175,12 +188,24 @@ enum HLSPlaylistParser { for line in lines { if line.hasPrefix("#EXT-X-TARGETDURATION:") { - targetDuration = Double(line.dropFirst("#EXT-X-TARGETDURATION:".count)) + targetDuration = try boundedDuration(line.dropFirst("#EXT-X-TARGETDURATION:".count), + tag: "TARGETDURATION") } else if line.hasPrefix("#EXT-X-MEDIA-SEQUENCE:") { - mediaSequence = Int(line.dropFirst("#EXT-X-MEDIA-SEQUENCE:".count)) ?? 0 + // A hostile or MITM value near Int.max makes `mediaSequence + segments.count` + // overflow and trap downstream (audit NET-3); reject it here instead. A merely + // unparseable tag keeps the pre-existing default of 0. + if let parsed = Int(line.dropFirst("#EXT-X-MEDIA-SEQUENCE:".count)) { + guard parsed >= 0, parsed <= Int.max / 2 else { + throw HLSIngestError.playlistInvalid(reason: "media sequence out of range") + } + mediaSequence = parsed + } } else if line.hasPrefix("#EXTINF:") { - let payload = line.dropFirst("#EXTINF:".count) - pendingDuration = Double(payload.split(separator: ",").first.map(String.init) ?? "") + // The duration is the first non-empty comma field; `split` built a Substring for every + // field of the title to find it (audit NET-105). + let payload = line.dropFirst("#EXTINF:".count).drop(while: { $0 == "," }) + let field = payload[..<(payload.firstIndex(of: ",") ?? payload.endIndex)] + pendingDuration = try boundedDuration(field, tag: "EXTINF") } else if line.hasPrefix("#EXT-X-DISCONTINUITY") && !line.hasPrefix("#EXT-X-DISCONTINUITY-SEQUENCE") { pendingDiscontinuity = true } else if line.hasPrefix("#EXT-X-KEY:") { @@ -250,19 +275,32 @@ enum HLSPlaylistParser { ) } + /// A duration tag's value; nil when it does not parse, which keeps the pre-existing fallbacks. + /// `Double(_:)` accepts `inf`, `nan` and any magnitude, and every consumer converts to an integer + /// somewhere downstream (audit NET-101), so a non-finite value or one past the program ceiling is + /// refused here. Negatives are left to the consumers: they trap nothing, and a sloppy live origin + /// should not lose its channel over one. + private static func boundedDuration(_ text: Substring, tag: String) throws -> Double? { + guard let value = Double(text) else { return nil } + guard value.isFinite, value <= MediaDurationCeiling.seconds else { + throw HLSIngestError.playlistInvalid(reason: "\(tag) out of range") + } + return value + } + /// Parse a `0x`-prefixed hex EXT-X-KEY IV into 16-byte big-endian Data. Returns nil on malformed length (caller falls back to sequence-number IV). /// ISO 8601 with fractional seconds is what every broadcaster emits; the plain form is accepted /// because the spec allows it. private static func parseProgramDateTime(_ raw: String) -> Date? { let text = raw.trimmingCharacters(in: .whitespaces) - let withFraction = ISO8601DateFormatter() - withFraction.formatOptions = [.withInternetDateTime, .withFractionalSeconds] - if let date = withFraction.date(from: text) { return date } - let plain = ISO8601DateFormatter() - plain.formatOptions = [.withInternetDateTime] - return plain.date(from: text) + return (try? fractionalDateStyle.parse(text)) ?? (try? plainDateStyle.parse(text)) } + /// Audit NET-106: two `ISO8601DateFormatter`s were built per PROGRAM-DATE-TIME line, 1.1 to 1.6 s + /// for a 3600-segment window that carries one on every segment. Value-type styles are built once. + private static let fractionalDateStyle = Date.ISO8601FormatStyle(includingFractionalSeconds: true) + private static let plainDateStyle = Date.ISO8601FormatStyle() + private static func parseHexIV(_ raw: String) -> Data? { var hex = raw.trimmingCharacters(in: .whitespaces) if hex.hasPrefix("0x") || hex.hasPrefix("0X") { hex = String(hex.dropFirst(2)) } @@ -292,28 +330,43 @@ enum HLSPlaylistParser { /// Extract a KEY=VALUE attribute from a tag line, tolerating quoted values. Match is anchored to `:` or `,` before the key: bare substring search matched `BANDWIDTH=` inside `AVERAGE-BANDWIDTH=` and caused wrong variant selection. /// Internal rather than private since #316: the master rewriter reads the same attributes off the /// same tag lines, and a second parser would be a second place for the anchoring trap to reappear. + /// + /// One left-to-right pass that carries the quote parity along. It recounted the quotes from the + /// line start for every `KEY=` it met, so a line of unclosed quote plus thousands of `,KEY=` took + /// time quadratic in its length, 9.6 s at 64 KiB (audit NET-105, NAT-102). Scanned as UTF-8: the + /// delimiters are ASCII. static func attribute(_ key: String, in line: String) -> String? { - let needle = "\(key)=" - var searchStart = line.startIndex - while let range = line.range(of: needle, range: searchStart.. String? in + let count = bytes.count + var insideQuotes = false + var index = 0 + while index < count { + let byte = bytes[index] + if byte == UInt8(ascii: "\"") { + insideQuotes.toggle() + index += 1 + continue + } + guard !insideQuotes, byte == needle[0], index + needle.count <= count, + index == 0 || bytes[index - 1] == UInt8(ascii: ":") || bytes[index - 1] == UInt8(ascii: ","), + needle.indices.allSatisfy({ bytes[index + $0] == needle[$0] }) + else { + index += 1 + continue + } + let valueStart = index + needle.count + if valueStart < count, bytes[valueStart] == UInt8(ascii: "\"") { + let quoted = UnsafeBufferPointer(rebasing: bytes[(valueStart + 1)...]) + guard let end = quoted.firstIndex(of: UInt8(ascii: "\"")) else { return nil } + return String(decoding: UnsafeBufferPointer(rebasing: quoted[.. Double { + guard let longest = segments.map(\.duration).max(), longest > 0 else { return 0 } + let targetDuration = LiveEdgePolicy.targetDurationForCadence(longest) + return LiveEdgePolicy.holdBackSeconds(targetDuration: targetDuration) + + min(longest, openGOPMarginSeconds) + } + + /// The 7.24.0 coverage rule on its own: how many segments a tune takes when `durations` (oldest + /// first) is what the upstream lists. nil when the listing runs out before the coverage is met and + /// before the count limit, so the depth of that tune cannot be read off this listing. + static func coverageJoinDepth(durations: ArraySlice, coverage: Double, limit: Int) -> Int? { + var taken = 0 + var seconds = 0.0 + for duration in durations.reversed() { + if taken >= limit { return taken } + if taken > 0, seconds >= coverage { return taken } + taken += 1 + seconds += duration + } + return (taken >= limit || seconds >= coverage) ? taken : nil + } + + /// AE#684: the join, equalised over the phases of one upstream. + /// + /// The coverage loop counts seconds back from the newest segment, so on an upstream whose segment + /// lengths alternate its depth depends on which one happens to be newest. The channel this issue + /// was reported on alternates 6 s and 4 s: `4 + 6 + 4` is under the 16 s target and a fourth is + /// taken (20 s listed, 18 s cut, seal 6), while `6 + 4 + 6` meets it at three (16 s listed, 14 s + /// cut, seal 4, which is 7.25.1's value and its stalls). About four tunes in ten landed in the + /// second phase, and the same channel behaved differently from one zap to the next. + /// + /// The rule: a tune never loads more than the deepest tune of the same channel already did. The + /// depth is the deepest the coverage rule takes over the actual tune and the tunes that would + /// have happened one, two, ... segments earlier, as far back as the newest segment of such a tune + /// is still one this join takes itself (a rhythm shorter than the join shows all its phases in + /// that span; looking further back would let one stray segment anywhere in the listing deepen + /// every tune). So 6 s / 4 s joins four in both phases, and the three-segment phase pays one + /// additional short segment that the other phase always loaded. + /// + /// It deliberately does nothing where no phase is deeper. 10 s / 8 s segments join three in both + /// phases and 6 s / 6 s / 4 s three in all three, exactly as before, and their seal is what that + /// join pays, as for a uniform upstream: a deeper join on EVERY tune is the cost AE#678 declined. + /// A lone short segment on an otherwise uniform upstream deepens the join only while it sits + /// within that span: for the tunes that needed the extra segment anyway, and the two after them. + /// + /// Bounded at one segment above the actual tune's own depth. Phases of one listing differ by more + /// than that only when lengths swing wildly (a run of 2 s segments behind a 10 s one), and there + /// the listing is describing a change of source, not a rhythm to equalise. + static func phaseEqualisedJoinDepth(durations: [Double], coverage: Double, limit: Int) -> Int { + guard let own = coverageJoinDepth(durations: durations[...], coverage: coverage, limit: limit) else { + return min(durations.count, limit) + } + var deepest = own + for earlier in 1.. [HLSMediaSegment] { let windowStart = playlist.mediaSequence - let windowEnd = playlist.mediaSequence + playlist.segments.count // exclusive + // The parser rejects a MEDIA-SEQUENCE outside `0...Int.max/2` (audit NET-3), but the struct + // itself can be built directly (tests, or a future caller), so this combines with `&+`/`&-` + // rather than trapping on a value that slipped past that guard. + let windowEnd = playlist.mediaSequence &+ playlist.segments.count // exclusive func segments(from sequence: Int, markFirstDiscontinuity: Bool) -> [HLSMediaSegment] { - let startIndex = sequence - windowStart + let startIndex = sequence &- windowStart guard startIndex < playlist.segments.count else { return [] } var result = Array(playlist.segments[max(0, startIndex)...]) if markFirstDiscontinuity, !result.isEmpty { @@ -59,18 +142,13 @@ struct HLSPlaylistTracker { } func joinStart() -> Int { - let coverage = max(minJoinCoverageSeconds, 1.5 * playlist.targetDuration) + let coverage = max(minJoinCoverageSeconds, 1.5 * playlist.targetDuration, + Self.loopbackCushionCoverageSeconds(segments: playlist.segments)) let limit = Self.joinSegmentLimit(edgeOffset: edgeOffset, windowSegmentCount: playlist.segments.count) - var taken = 0 - var seconds = 0.0 - for segment in playlist.segments.reversed() { - if taken >= limit { break } - if taken > 0, seconds >= coverage { break } - taken += 1 - seconds += segment.duration - } - return windowEnd - taken + let taken = Self.phaseEqualisedJoinDepth(durations: playlist.segments.map(\.duration), + coverage: coverage, limit: limit) + return windowEnd &- taken } guard let cursor = nextSequence else { diff --git a/Sources/AetherEngine/IO/HLSIngest/HLSVODIngestReader.swift b/Sources/AetherEngine/IO/HLSIngest/HLSVODIngestReader.swift index aa07b9b64..f2d0b5853 100644 --- a/Sources/AetherEngine/IO/HLSIngest/HLSVODIngestReader.swift +++ b/Sources/AetherEngine/IO/HLSIngest/HLSVODIngestReader.swift @@ -111,7 +111,7 @@ final class HLSVODIngestReader: TimeSeekableIOReader, @unchecked Sendable { } catch { if Task.isCancelled { throw CancellationError() } EngineLog.emit( - "[HLSVODIngest] carriage probe inconclusive: \(error)", + "[HLSVODIngest] carriage probe inconclusive: \(EngineLog.summary(of: error))", category: .engine ) return nil @@ -384,7 +384,8 @@ final class HLSVODIngestReader: TimeSeekableIOReader, @unchecked Sendable { func spawn(_ index: Int) { let item = resolvedSegments[index] group.addTask { - let bytes = try await self.fetch(item.1) + let bytes = try await self.fetch( + item.1, limit: BoundedFetch.segmentLimit(forDuration: item.0.duration)) guard let crypt = item.0.crypt else { return (index, bytes) } return ( index, @@ -449,7 +450,7 @@ final class HLSVODIngestReader: TimeSeekableIOReader, @unchecked Sendable { producer = nil condition.broadcast() condition.unlock() - EngineLog.emit("[HLSVODIngest] terminal: \(error)", category: .engine) + EngineLog.emit("[HLSVODIngest] terminal: \(EngineLog.summary(of: error))", category: .engine) } private func resolveMedia() async throws -> ResolvedMedia { @@ -481,47 +482,67 @@ final class HLSVODIngestReader: TimeSeekableIOReader, @unchecked Sendable { guard !media.hasUnsupportedEncryption else { throw HLSIngestError.encryptedNotSupported } + let timeline = try Self.segmentTimeline(media.segments) + return ResolvedMedia( + url: mediaURL, + segments: media.segments, + starts: timeline.starts, + duration: timeline.duration + ) + } + + static func segmentTimeline(_ segments: [HLSMediaSegment]) throws -> (starts: [Double], duration: Double) { var starts: [Double] = [] - starts.reserveCapacity(media.segments.count) + starts.reserveCapacity(segments.count) var duration = 0.0 - for segment in media.segments { + for segment in segments { guard segment.duration.isFinite, segment.duration > 0 else { throw HLSIngestError.playlistInvalid(reason: "segment duration must be positive") } starts.append(duration) duration += segment.duration } - return ResolvedMedia( - url: mediaURL, - segments: media.segments, - starts: starts, - duration: duration - ) + // Audit HLS-102: each entry is inside the parser's ceiling, the sum need not be. A tiny entry + // after a large sum also leaves the total unmoved, the starts stop being monotonic, and the + // uniform fallback plans the whole sum. + guard duration <= MediaDurationCeiling.seconds else { + throw HLSIngestError.playlistInvalid(reason: "program duration exceeds \(MediaDurationCeiling.seconds)s") + } + return (starts, duration) } - private func makeRequest(_ url: URL) -> URLRequest { + /// Credentials only where the host's playlist is (audit NET-7). + func makeRequest(_ url: URL) -> URLRequest { var request = URLRequest(url: url) - for (field, value) in httpHeaders { + for (field, value) in RedirectHeaderPolicy.scoped( + httpHeaders, grantedFor: playlistURL, sentTo: url) { request.setValue(value, forHTTPHeaderField: field) } return request } private func fetchPlaylist(_ url: URL) async throws -> (HLSPlaylist, URL) { - let (data, response) = try await session.data(for: makeRequest(url)) + let (data, response) = try await BoundedPlaylistFetch.data( + for: makeRequest(url), session: session, limit: Self.maximumPlaylistBytes) let status = (response as? HTTPURLResponse)?.statusCode ?? -1 guard (200..<300).contains(status) else { throw HLSIngestError.playlistUnreachable(status: status) } - guard data.count <= Self.maximumPlaylistBytes, - let text = String(data: data, encoding: .utf8) else { + guard let text = String(data: data, encoding: .utf8) else { throw HLSIngestError.playlistInvalid(reason: "playlist is not bounded UTF-8") } return (try HLSPlaylistParser.parse(text), response.url ?? url) } - private func fetch(_ url: URL) async throws -> Data { - let (data, response) = try await session.data(for: makeRequest(url)) + /// A segment or a key body, cut off at `limit` while it arrives (audit NET-112). + private func fetch(_ url: URL, limit: Int) async throws -> Data { + let data: Data + let response: URLResponse + do { + (data, response) = try await BoundedFetch.data(for: makeRequest(url), session: session, limit: limit) + } catch is BoundedFetch.Exceeded { + throw HLSIngestError.playlistInvalid(reason: "body exceeds \(limit) bytes") + } let status = (response as? HTTPURLResponse)?.statusCode ?? -1 guard (200..<300).contains(status), !data.isEmpty else { throw HLSIngestError.playlistUnreachable(status: status) @@ -534,7 +555,7 @@ final class HLSVODIngestReader: TimeSeekableIOReader, @unchecked Sendable { private func probeCarriage(_ url: URL) async throws -> MPEGTransportStreamCodecProbe.Verdict { try await HLSCarriageProbe.classifySegmentHead( url: url, - httpHeaders: httpHeaders, + httpHeaders: RedirectHeaderPolicy.scoped(httpHeaders, grantedFor: playlistURL, sentTo: url), session: session ) } @@ -551,7 +572,7 @@ final class HLSVODIngestReader: TimeSeekableIOReader, @unchecked Sendable { if let cached = keyCacheLock.withLock({ keyCache[keyURL.absoluteString] }) { key = cached } else { - let fetched = try await fetch(keyURL) + let fetched = try await fetch(keyURL, limit: BoundedFetch.keyLimit) guard fetched.count == kCCKeySizeAES128 else { throw HLSIngestError.segmentDecryptFailed(reason: "key length is not 16 bytes") } diff --git a/Sources/AetherEngine/IO/HLSIngest/LiveIngestSourceInfo.swift b/Sources/AetherEngine/IO/HLSIngest/LiveIngestSourceInfo.swift index c6118d234..35fa2be48 100644 --- a/Sources/AetherEngine/IO/HLSIngest/LiveIngestSourceInfo.swift +++ b/Sources/AetherEngine/IO/HLSIngest/LiveIngestSourceInfo.swift @@ -27,11 +27,24 @@ protocol LiveIngestSourceInfo: AnyObject, Sendable { var closedLiveCadenceSeconds: Double? { get } /// Longest segment duration (EXTINF) the upstream has actually SERVED, nil until the first arrival. - /// The measured counterpart to `upstreamTargetDuration`, and the term that keeps the served - /// TARGETDURATION honest when a join burst makes the arrival intervals look shorter than the steady - /// state ever will be: an upstream cutting 4 s segments cannot sustain a 0.5 s cadence (AE#447). + /// The measured counterpart to `upstreamTargetDuration`. Two things are read off it: a bound on + /// the steady-state cadence when a join burst makes the arrival intervals look shorter than they + /// will ever be (an upstream cutting 4 s segments cannot sustain 0.5 s, AE#447), and the unit the + /// source delivers in, which the served TARGETDURATION covers whole (AE#684). var upstreamSegmentDurationSeconds: Double? { get } + /// Summed EXTINF of the segments this reader joined with, nil until it has joined. Reported in + /// the seal line; nothing is decided from it, because an upstream's EXTINF and its media need + /// not agree (AE#684). + var joinBacklogSeconds: Double? { get } + + /// Whether the join has been handed over in full AND consumed: every byte of the join batch has + /// been committed to the reader (and to its companion audio reader, where there is one), and the + /// consumer is parked on an empty reader, either of the two, waiting for the next upstream + /// delivery. From then on nothing more can be cut until that delivery, which is + /// what the first-serve gate needs to know before it waits for a deeper cushion (AE#684). + var joinIsSpent: Bool { get } + /// OBSERVED upstream segment-arrival cadence in seconds (recent max inter-arrival interval, widened by /// the currently-open gap), nil until the first arrival. Unlike `upstreamTargetDuration` this reflects /// how the origin actually delivers, so the engine can detect bursty relays that advertise a normal @@ -46,6 +59,10 @@ protocol LiveIngestSourceInfo: AnyObject, Sendable { /// flows, like `companionAudioReader`, so it is final once the load reads it. var subtitleRenditions: [LiveSubtitleRenditionInfo] { get } + /// The URL the host gave the reader's headers for. A sibling rendition fetch sends the credential + /// headers only to this origin, like the reader's own fetches (audit Vcred-101). + var credentialOrigin: URL { get } + /// EXT-X-PROGRAM-DATE-TIME of the segment this reader joined at, nil when the upstream carries no /// PDT. Together with the engine's clock at session start this is the wall-to-player mapping a /// sibling rendition needs (AE#359). diff --git a/Sources/AetherEngine/IO/HTTPDiscIOReader.swift b/Sources/AetherEngine/IO/HTTPDiscIOReader.swift index 3c40980df..1e805efa4 100644 --- a/Sources/AetherEngine/IO/HTTPDiscIOReader.swift +++ b/Sources/AetherEngine/IO/HTTPDiscIOReader.swift @@ -13,7 +13,12 @@ import Foundation /// playback of the title's extents costs few requests); any non-contiguous read resets it. Each /// range request retries with backoff so a transient network blip does not end playback. The server /// MUST honor range requests (any static file host does); if it does not, `init` returns nil after a -/// clear log and the caller falls back to the plain streaming path. +/// clear log and `open` throws `AVIOReaderError.originIgnoresRange`, and the response is hung up on at +/// its head, so a 40 GB image answered with a 200 is never downloaded to be rejected (audit NET-102). +/// +/// A source the host warmed with `AetherEngine.prewarm` (#551) is read out of those bytes first +/// (#647): the warm has already stated the size and proven range support, so the reader neither +/// probes nor refetches the head, and its first request starts at the warm frontier. final class HTTPDiscIOReader: IOReader, @unchecked Sendable { private let url: URL @@ -24,6 +29,8 @@ final class HTTPDiscIOReader: IOReader, @unchecked Sendable { private let maxChunkSize: Int private let maxRetries: Int private let totalSize: Int64 + /// Warmed bytes (#647), read before the network. Immutable, so a fork shares them without a copy. + private let residentSpans: [ResidentSpan] private let lock = NSLock() private var position: Int64 = 0 @@ -39,15 +46,70 @@ final class HTTPDiscIOReader: IOReader, @unchecked Sendable { private let cancelLock = NSLock() private var cancelled = false - /// Probes total size and range support with one (retried) `bytes=0-0` request. Returns nil if - /// the source is unreachable or answers `200` (full body, no range support); logs which. - init?(url: URL, - extraHeaders: [String: String] = [:], - baseChunkSize: Int = 256 * 1024, - maxChunkSize: Int = 8 * 1024 * 1024, - maxRetries: Int = 3, - requestTimeout: TimeInterval = 30, - sessionConfiguration: URLSessionConfiguration? = nil) { + /// Probes total size and range support with one (retried) `bytes=0-0` request, unless + /// `prewarmed` has already stated both. Returns nil if the source is unreachable or answers `200` + /// (full body, no range support); logs which. `open` tells the two apart. + convenience init?(url: URL, + extraHeaders: [String: String] = [:], + baseChunkSize: Int = 256 * 1024, + maxChunkSize: Int = 8 * 1024 * 1024, + maxRetries: Int = 3, + requestTimeout: TimeInterval = 30, + sessionConfiguration: URLSessionConfiguration? = nil, + prewarmed: PrewarmedSource? = nil) { + let session = Self.makeSession(sessionConfiguration) + guard case .size(let size) = Self.resolveSize( + url: url, extraHeaders: extraHeaders, session: session, requestTimeout: requestTimeout, + maxRetries: maxRetries, knownSize: prewarmed?.contentLength) else { + session.invalidateAndCancel() + return nil + } + self.init(url: url, extraHeaders: extraHeaders, baseChunkSize: baseChunkSize, + maxChunkSize: maxChunkSize, maxRetries: maxRetries, requestTimeout: requestTimeout, + session: session, totalSize: size, + residentSpans: prewarmed.map { [$0.head] + ($0.tail.map { [$0] } ?? []) } ?? []) + } + + /// `init?` for a caller that has to say why a disc image cannot be read: an origin that answers the + /// range probe with a 200 cannot address bytes, which is a different failure from an unreachable + /// one, and a disc image is a filesystem that no reader can walk without ranges (audit NET-102). + /// nil still means unreachable, and the caller falls back to the plain streaming path. + static func open(url: URL, + extraHeaders: [String: String] = [:], + baseChunkSize: Int = 256 * 1024, + maxChunkSize: Int = 8 * 1024 * 1024, + maxRetries: Int = 3, + requestTimeout: TimeInterval = 30, + sessionConfiguration: URLSessionConfiguration? = nil, + prewarmed: PrewarmedSource? = nil) throws -> HTTPDiscIOReader? { + let session = makeSession(sessionConfiguration) + switch resolveSize(url: url, extraHeaders: extraHeaders, session: session, + requestTimeout: requestTimeout, maxRetries: maxRetries, + knownSize: prewarmed?.contentLength) { + case .size(let size): + return HTTPDiscIOReader( + url: url, extraHeaders: extraHeaders, baseChunkSize: baseChunkSize, + maxChunkSize: maxChunkSize, maxRetries: maxRetries, requestTimeout: requestTimeout, + session: session, totalSize: size, + residentSpans: prewarmed.map { [$0.head] + ($0.tail.map { [$0] } ?? []) } ?? []) + case .ignoresRange: + session.invalidateAndCancel() + throw AVIOReaderError.originIgnoresRange + case .unreachable: + session.invalidateAndCancel() + return nil + } + } + + private init(url: URL, + extraHeaders: [String: String], + baseChunkSize: Int, + maxChunkSize: Int, + maxRetries: Int, + requestTimeout: TimeInterval, + session: URLSession, + totalSize: Int64, + residentSpans: [ResidentSpan]) { self.url = url self.extraHeaders = extraHeaders self.baseChunkSize = max(64 * 1024, baseChunkSize) @@ -55,22 +117,50 @@ final class HTTPDiscIOReader: IOReader, @unchecked Sendable { self.currentChunkSize = max(64 * 1024, baseChunkSize) self.maxRetries = max(0, maxRetries) self.requestTimeout = requestTimeout + self.session = session + self.residentSpans = residentSpans.filter { !$0.isEmpty } + self.totalSize = totalSize + } + + private static func makeSession(_ sessionConfiguration: URLSessionConfiguration?) -> URLSession { let config = sessionConfiguration ?? { let c = URLSessionConfiguration.ephemeral c.requestCachePolicy = .reloadIgnoringLocalCacheData return c }() - self.session = URLSession( - configuration: config, delegate: EngineTLS.sessionDelegate, delegateQueue: nil) + return URLSession(configuration: config, delegate: EngineTLS.sessionDelegate, delegateQueue: nil) + } - guard let size = Self.probeSize( - url: url, extraHeaders: extraHeaders, session: session, - timeout: requestTimeout, maxRetries: max(0, maxRetries) - ), size > 0 else { - session.invalidateAndCancel() + /// `knownSize` skips the range probe. Only a source that has already answered a range request + /// with its total may pass it: the warm, or the reader a fork is made from. + private static func resolveSize(url: URL, extraHeaders: [String: String], session: URLSession, + requestTimeout: TimeInterval, maxRetries: Int, + knownSize: Int64?) -> SizeProbe { + if let knownSize, knownSize > 0 { return .size(knownSize) } + return probeSize(url: url, extraHeaders: extraHeaders, session: session, + timeout: requestTimeout, maxRetries: max(0, maxRetries)) + } + + /// #647: take what a host warmed for this URL, under the rule `AVIOReader` adopts by (#551). + /// + /// A take, like every adoption: the reader holds the bytes from here on. A caller whose source + /// turns out not to be a disc puts them back (`SourcePrewarmStore.store`) so the streaming + /// reader it falls back to adopts them instead. + static func takePrewarm(for url: URL, extraHeaders: [String: String], + store: SourcePrewarmStore = .shared) -> PrewarmedSource? { + guard let warm = store.take(for: url) else { return nil } + guard warm.head.start == 0, !warm.head.isEmpty, warm.contentLength > 0 else { return nil } + guard warm.requestHeaders == extraHeaders else { + EngineLog.emit( + "[HTTPDiscIOReader] \(url.lastPathComponent): a warm exists for this URL but was " + + "fetched with different headers; opening cold (#647)", category: .demux) return nil } - self.totalSize = size + EngineLog.emit( + "[HTTPDiscIOReader] \(url.lastPathComponent): adopted a prewarmed source: " + + "head=\(warm.head.data.count)B tail=\(warm.tail?.data.count ?? 0)B of " + + "\(warm.contentLength)B (#647)", category: .demux) + return warm } // MARK: - Pure helpers @@ -88,6 +178,15 @@ final class HTTPDiscIOReader: IOReader, @unchecked Sendable { return Int64(total) } + /// Start offset from a `Content-Range` value (`bytes 100-200/12345` -> 100). Nil for junk. + static func parseContentRangeStart(_ value: String) -> Int64? { + let trimmed = value.trimmingCharacters(in: .whitespaces) + guard trimmed.hasPrefix("bytes ") else { return nil } + let rest = trimmed.dropFirst("bytes ".count) + guard let dash = rest.firstIndex(of: "-") else { return nil } + return Int64(rest[rest.startIndex..= totalSize { return 0 } + if let span = residentSpans.first(where: { $0.covers(position) }) { + let spanOffset = Int(position - span.start) + let toCopy = min(Int(n), span.data.count - spanOffset, Int(totalSize - position)) + span.data.withUnsafeBytes { src in + out.update(from: src.baseAddress!.assumingMemoryBound(to: UInt8.self) + .advanced(by: spanOffset), count: toCopy) + } + position += Int64(toCopy) + // Reading on past the span is the sequential continuation, so the window grows there. + lastFetchEnd = position + return Int32(toCopy) + } + if position < bufferStart || position >= bufferStart + Int64(buffer.count) { currentChunkSize = Self.nextChunkSize( position: position, lastFetchEnd: lastFetchEnd, current: currentChunkSize, base: baseChunkSize, maxChunk: maxChunkSize) - let want = Int(min(Int64(currentChunkSize), totalSize - position)) + // Stop at the next warmed span: its bytes are already here. + let limit = residentSpans.lazy.map(\.start).filter { $0 > self.position }.min() ?? totalSize + let want = Int(min(Int64(currentChunkSize), limit - position)) guard want > 0, let data = fetchWithRetry(offset: position, length: want), !data.isEmpty else { return -1 } @@ -152,12 +266,20 @@ final class HTTPDiscIOReader: IOReader, @unchecked Sendable { func makeIndependentReader() -> IOReader? { HTTPDiscIOReader(url: url, extraHeaders: extraHeaders, baseChunkSize: baseChunkSize, maxChunkSize: maxChunkSize, - maxRetries: maxRetries, requestTimeout: requestTimeout) + maxRetries: maxRetries, requestTimeout: requestTimeout, + session: Self.makeSession(nil), + totalSize: totalSize, residentSpans: residentSpans) } // MARK: - HTTP /// One range GET retried up to `maxRetries` times with linear backoff; aborts early on cancel. + /// + /// Requires a 206 whose `Content-Range` start matches `offset` exactly: a proxy that answers a + /// range request with a full 200 (cache miss, range coalescing, a Range-stripping origin) or a + /// 206 that starts somewhere else places its body at `position` regardless, corrupting the disc + /// structure or media stream read through it (audit NET-9). The body is also trimmed to `length` + /// in case the server sent more than asked. private func fetchWithRetry(offset: Int64, length: Int) -> Data? { var attempt = 0 while true { @@ -165,8 +287,10 @@ final class HTTPDiscIOReader: IOReader, @unchecked Sendable { if stop { return nil } if let r = Self.rangeGet(url: url, extraHeaders: extraHeaders, session: session, timeout: requestTimeout, offset: offset, length: length), - r.status == 206 || r.status == 200, !r.body.isEmpty { - return r.body + r.status == 206, !r.body.isEmpty, + let contentRange = r.contentRange, + Self.parseContentRangeStart(contentRange) == offset { + return r.body.count > length ? r.body.prefix(length) : r.body } attempt += 1 if attempt > maxRetries { return nil } @@ -174,23 +298,29 @@ final class HTTPDiscIOReader: IOReader, @unchecked Sendable { } } + private enum SizeProbe { + case size(Int64) + /// The origin answered the range probe with a 200 that is not the byte asked for. + case ignoresRange + case unreachable + } + private static func probeSize(url: URL, extraHeaders: [String: String], session: URLSession, - timeout: TimeInterval, maxRetries: Int) -> Int64? { + timeout: TimeInterval, maxRetries: Int) -> SizeProbe { var attempt = 0 while true { let r = rangeGet(url: url, extraHeaders: extraHeaders, session: session, timeout: timeout, offset: 0, length: 1) if let r = r, r.status == 206, let cr = r.contentRange, let total = parseContentRangeTotal(cr) { - return total + return .size(total) } if let r = r, r.status == 200 { EngineLog.emit( - "[HTTPDiscIOReader] \(url.lastPathComponent): server answered 200 without a " - + "Content-Range; remote disc images need HTTP byte-range support. " - + "Falling back to the streaming path.", + "[HTTPDiscIOReader] \(url.lastPathComponent): server answered 200 to a range " + + "probe (hung up at the head); remote disc images need HTTP byte-range support.", category: .demux) - return nil + return r.answer == .ignored ? .ignoresRange : .unreachable } attempt += 1 if attempt > maxRetries { @@ -199,14 +329,25 @@ final class HTTPDiscIOReader: IOReader, @unchecked Sendable { + "\(attempt) attempt(s) (status=\(r.map { String($0.status) } ?? "no response")). " + "Falling back to the streaming path.", category: .demux) - return nil + return .unreachable } Thread.sleep(forTimeInterval: min(0.25 * Double(attempt), 1.0)) } } - private struct RangeResponse { let status: Int; let contentRange: String?; let body: Data } + private struct RangeResponse { + let status: Int + let contentRange: String? + let body: Data + let answer: AVIOReader.RangeAnswer + } + /// Takes one range of the source. Only a 206 that starts where asked is taken at all, and what is + /// taken is capped at `length`: a 200 or a misplaced 206 is hung up on at its head, and a 206 that + /// runs past the range is cut where the range ends. The completion-handler form buffered the whole + /// body first, so an origin that ignored Range (or answered `bytes N-EOF`) put a disc image into + /// memory before the check ran (audit NET-102). + /// /// #243: the pull path runs this synchronously on FFmpeg's read callback, i.e. on a demux pump /// thread that lives for the whole session inside ONE dispatch block, so nothing ever drains /// that thread's autorelease pool. Every response bridged out of the completion handler is then @@ -228,27 +369,22 @@ final class HTTPDiscIOReader: IOReader, @unchecked Sendable { req.setValue(rangeHeader(offset: offset, length: length), forHTTPHeaderField: "Range") for (k, v) in extraHeaders { req.setValue(v, forHTTPHeaderField: k) } + let delegate = RangeFetchDelegate(offset: offset, length: length) let sem = DispatchSemaphore(value: 0) - nonisolated(unsafe) var result: RangeResponse? - let task = session.dataTask(with: req) { data, response, _ in - if let http = response as? HTTPURLResponse { - result = RangeResponse( - status: http.statusCode, - contentRange: http.value(forHTTPHeaderField: "Content-Range"), - body: data ?? Data() - ) - } - sem.signal() - } + delegate.onCompletion = { sem.signal() } + let task = session.dataTask(with: req) + task.delegate = delegate task.resume() if sem.wait(timeout: .now() + timeout + 5) == .timedOut { task.cancel() return nil } - // The body escapes the pool inside a retained `Data`; the pool only balances the - // bridging autorelease, so the buffer still lives until `read` has copied it out. - Self.recordFetched(bytes: result?.body.count ?? 0) - return result + guard let http = delegate.response else { return nil } + let body = delegate.body + Self.recordFetched(bytes: body.count) + return RangeResponse(status: http.statusCode, + contentRange: http.value(forHTTPHeaderField: "Content-Range"), + body: body, answer: delegate.answer) } } @@ -261,6 +397,7 @@ final class HTTPDiscIOReader: IOReader, @unchecked Sendable { /// counter. Reset per session by the memory probe. private static let fetchedLock = NSLock() nonisolated(unsafe) private static var fetchedBytesTotal: Int64 = 0 + nonisolated(unsafe) private static var fetchedResetCount: Int = 0 private static func recordFetched(bytes: Int) { guard bytes > 0 else { return } @@ -274,9 +411,76 @@ final class HTTPDiscIOReader: IOReader, @unchecked Sendable { return fetchedBytesTotal } + /// Bumped by every reset, so a reader of `lifetimeFetchedBytes` deltas can tell that a session + /// start in between (`startMemoryProbe`) zeroed the tally under it. + static var lifetimeFetchedResetCount: Int { + fetchedLock.lock(); defer { fetchedLock.unlock() } + return fetchedResetCount + } + static func resetLifetimeFetchedBytes() { fetchedLock.lock() fetchedBytesTotal = 0 + fetchedResetCount &+= 1 fetchedLock.unlock() } } + +/// One range request's delivery. The decision is made at the response head, where nothing of the body +/// has been taken yet, and the body is capped at the range that was asked for. +private final class RangeFetchDelegate: NSObject, URLSessionDataDelegate, @unchecked Sendable { + private let offset: Int64 + private let length: Int + private(set) var response: HTTPURLResponse? + private(set) var answer: AVIOReader.RangeAnswer = .unjudged + private(set) var body = Data() + var onCompletion: (() -> Void)? + + init(offset: Int64, length: Int) { + self.offset = offset + self.length = length + } + + func urlSession( + _ session: URLSession, + task: URLSessionTask, + didReceive challenge: URLAuthenticationChallenge, + completionHandler: @escaping (URLSession.AuthChallengeDisposition, URLCredential?) -> Void + ) { + EngineTLS.resolve(challenge, completionHandler: completionHandler) + } + + func urlSession( + _ session: URLSession, + dataTask: URLSessionDataTask, + didReceive response: URLResponse, + completionHandler: @escaping (URLSession.ResponseDisposition) -> Void + ) { + guard let http = response as? HTTPURLResponse else { + completionHandler(.cancel) + return + } + self.response = http + answer = AVIOReader.rangeAnswer( + http, requestedStart: offset, requestedEnd: offset + Int64(length) - 1) + switch answer { + case .honoured, .overWide: + body.reserveCapacity(Int(max(0, min(http.expectedContentLength, Int64(length))))) + completionHandler(.allow) + case .ignored, .wholeFile, .misplaced, .unjudged: + completionHandler(.cancel) + } + } + + func urlSession(_ session: URLSession, dataTask: URLSessionDataTask, didReceive data: Data) { + let room = length - body.count + if room > 0 { body.append(data.prefix(room)) } + // A body that ends on the range end is left to finish, which keeps the connection for the + // next range. Hanging up is for what runs past it. + if data.count > room || (answer == .overWide && body.count >= length) { dataTask.cancel() } + } + + func urlSession(_ session: URLSession, task: URLSessionTask, didCompleteWithError error: Error?) { + onCompletion?() + } +} diff --git a/Sources/AetherEngine/MediaDurationCeiling.swift b/Sources/AetherEngine/MediaDurationCeiling.swift new file mode 100644 index 000000000..854ebe7b5 --- /dev/null +++ b/Sources/AetherEngine/MediaDurationCeiling.swift @@ -0,0 +1,8 @@ +import Foundation + +/// Longest program the engine parses, plans or serves (audit NET-101, HLS-102). One week: a whole-program +/// subtitle playlist puts a whole film in one EXTINF, so the per-entry ceiling has to equal the program +/// ceiling, and a 30 h marathon must parse under the same rule the subtitle proxy serves it by. +enum MediaDurationCeiling { + static let seconds: Double = 7 * 24 * 3600 +} diff --git a/Sources/AetherEngine/Native/MasterFallbackDecision.swift b/Sources/AetherEngine/Native/MasterFallbackDecision.swift index 96a0246cd..0e9d57817 100644 --- a/Sources/AetherEngine/Native/MasterFallbackDecision.swift +++ b/Sources/AetherEngine/Native/MasterFallbackDecision.swift @@ -29,6 +29,24 @@ enum MasterFallbackDecision { code == -11868 || code == -11848 } + /// AE#535: whether a refusal teaches the process something about the panel. + /// + /// A display rejection answers "does this output configuration accept an HDR master" only while + /// the display is eligible for HDR at all. Measured on an Apple TV 4K on tvOS 26.6: an audio route + /// death right after a display mode switch opens a window in which the criteria readout reads + /// `matching=off hdrEligible=no`, a session-preserving reload lands in it, and AVPlayer fails the + /// master with -11868 because nothing HDR is compatible WITH AN INELIGIBLE DISPLAY. That refusal is + /// fully explained by the window, so it earns this item its media fallback and nothing more. + /// Latching it cost the rest of the viewing session its master, and in a process that never + /// backgrounds the AE#588 clear never came. + /// + /// AE#667: nor while a display mode switch is in progress. AVPlayer answers for the mode the panel is + /// in at that moment, which mid-switch is the one it is leaving, so the refusal is about the moment. + static func shouldLatchPanelRefusal(code: Int, displayEligibleForHDRNow: Bool, + displaySwitchInProgress: Bool) -> Bool { + isDisplayRejectionCode(code) && displayEligibleForHDRNow && !displaySwitchInProgress + } + /// Any code that means "AVPlayer rejected the served master itself": the display-rejection pair /// plus NSURLErrorDomain -1002 "unsupported URL", which AVPlayer surfaces when it filters EVERY /// EXT-X-STREAM-INF out of the master at parse time and is left with no variant URL to load diff --git a/Sources/AetherEngine/Native/NativeAVPlayerHost.swift b/Sources/AetherEngine/Native/NativeAVPlayerHost.swift index ec3ebaca7..75cc958bc 100644 --- a/Sources/AetherEngine/Native/NativeAVPlayerHost.swift +++ b/Sources/AetherEngine/Native/NativeAVPlayerHost.swift @@ -91,6 +91,10 @@ final class NativeAVPlayerHost { private var liveJoinThinBufferLogged: Bool = false /// AE#440 round 5: keeps the "the hold was over before the reading came back" line to one per load. private var liveJoinNoDecisionLogged: Bool = false + /// AE#684: one line per load for a hold the depth would have cut and the item's status refused. + private var liveJoinNotReadyLogged: Bool = false + /// AE#684: the one repeat a not-ready refusal gets when the item turned ready under it. + private var liveJoinNotReadyAskedAgain: Bool = false /// AE#440 round 3: one witness per load for a hold that was refused, so the bound is read while the /// hold stands and not only at its edges. Observes, never acts (see `startLiveJoinHoldWitness`). private var liveJoinHoldWitnessStarted: Bool = false @@ -121,13 +125,6 @@ final class NativeAVPlayerHost { /// duration, and publishing that transient would bounce the engine through `.paused` and back for /// what the viewer must not even notice; the real status is republished when the recovery settles. private var prematureEndRecoveryInFlight = false - /// Uptimes bounding the last premature-end recovery. A rate change AVPlayer reported inside that - /// interval belongs to the recovery, even when its main-actor hop runs after the recovery ended. - private var prematureEndRecoveryStartedUptime: UInt64 = 0 - private var prematureEndRecoveryEndedUptime: UInt64 = 0 - /// Uptime of the newest transport event applied to `pausedSinceUptime`. Rate reports reach the - /// main actor after engine commands issued later, so an older report must not overwrite them. - private var transportStampEventUptime: UInt64 = 0 /// Mirrors avPlayer.timeControlStatus so the engine can reconcile when AVKit's transport bar, Control Center, or hardware buttons toggle the player externally (without this, engine state goes stale and play/pause presses are swallowed). @Published private(set) var timeControlStatus: AVPlayer.TimeControlStatus = .paused /// Monotonic count of AVPlayerItem playbackStalled notifications (#93 residual): the engine @@ -138,30 +135,6 @@ final class NativeAVPlayerHost { /// at .paused, which every pause-guarded recovery layer misreads as user intent; the engine /// subscribes and escalates into the stage-2 item reload with the pause guard bypassed. @Published private(set) var endFailureCount: Int = 0 - /// Whether the transport had already stopped before the latest counted end failure: the viewer - /// paused (through the engine, AVKit, Control Center or PiP) and the item died under that pause. - /// Set before `endFailureCount` publishes, so its subscribers read the value for their failure. - private(set) var endFailureFollowedPause = false - /// The latest engine-routed transport command since the latest counted end failure: true for - /// play, false for pause, nil for none. A viewer can press either while the engine confirms the - /// death, and that press outranks the transport state the item died in. - private(set) var transportCommandSinceEndFailure: Bool? - /// Uptime at which the commanded transport stopped: stamped when AVPlayer's `rate` drops to 0 or an - /// engine-routed pause lands, cleared by any non-zero rate. The rate is what play and pause set, - /// from any source, even on an item that cannot roll; `timeControlStatus` only reports the outcome. - private var pausedSinceUptime: UInt64? - /// Whether AVPlayer's rate went non-zero since the engine last stopped the transport (a pause, a - /// zero rate or a fresh load), judged by when AVPlayer reported it. A Play from AVKit, Control - /// Center or PiP moves the rate but not `playIntent`, so this is the only record of it. An in-place - /// swap carries the outgoing item's rate in. - private var rolledSinceEngineStop = false - /// Uptime of the engine's last transport stop. A rate report older than it is not a roll. - private var engineStopUptime: UInt64 = 0 - /// Uptime of the latest master refusal handed to the engine (#98), and the engine-routed transport - /// command since it: true for play, false for pause, nil for none. The media fallback runs a task - /// hop after the refusal and reads both then. - private var displayRejectionUptime: UInt64 = 0 - private var transportCommandSinceRejection: Bool? /// End of the last seekable time range (seconds); tracks the live edge for EVENT playlists. /// KVO mirror of `seekableTimeRanges`, NOT a live read: the getter is a sync XPC round-trip /// to mediaserverd, and clock-tick sinks plus the 1 Hz paused-live timer read this at a @@ -194,6 +167,12 @@ final class NativeAVPlayerHost { /// spelling the engine publishes elsewhere. Set beside `detectedVideoFormat`, which the engine's sink /// reads it with; nil while no video track resolves. @Published private(set) var detectedVideoCodecName: String? + /// Dimensions and colour description of the delivered video from the same read, for the bypass's + /// `sourceVideoWidth` / `sourceVideoHeight` / `sourceVideoStreamFormat`. Set before `detectedVideoFormat`. + @Published private(set) var detectedVideoDescription: RemoteHLSStreamDescription.Video? + /// The audio tracks AVPlayer built for the item, read at the same two points as the video format. + /// The engine publishes them as `audioTracks` on the bypass, where no probe lists them. + @Published private(set) var detectedAudioTracks: [RemoteHLSStreamDescription.AudioReading] = [] /// AetherEngine#168 follow-up: fires once when the armed carriage watchdog concludes the master /// advertises a video rendition but AVPlayer never built a video track past the grace window @@ -249,6 +228,16 @@ final class NativeAVPlayerHost { /// uses this as authoritative presented-frame evidence when that publication wins the MainActor /// queue race against the resumed deadline continuation. private(set) var latestSeekRenderedTimePublished = false + /// AE#629: a mount whose item has not landed its mount seek yet. Until the landing AVPlayer reads + /// first the target, then the start of the segment it decodes up from (12.00 s under a 15.97 s + /// revive on the harness). Published, that second reading rewinds the playhead, and a + /// software-path rebuild raised in the window resumed there and replayed the gap. So neither + /// reading is a landing: only the seek's own completion is, or the item playing, which it cannot + /// do short of the target. A seek of the host's own takes the clock over from here as well. + /// Round 5: not only an in-place swap. A VOD mount a host `load()` placed past the head has the + /// same window, and there the rebuild resumed at 12.00 s once that load had returned. See + /// `mountHoldsClock`. + private var mountSeekPending = false // MARK: - Output @@ -260,6 +249,8 @@ final class NativeAVPlayerHost { // MARK: - Private state private var playerItem: AVPlayerItem? + /// AE#616: the item the latest `load` attached, for an output the engine hangs on it. + var currentPlayerItem: AVPlayerItem? { playerItem } /// Applied immediately and replayed onto fresh items across internal reloads so Now Playing title/artwork survives audio-switch/background-reopen seams. private var pendingExternalMetadata: [AVMetadataItem] = [] private var timeObserver: Any? @@ -434,6 +425,9 @@ final class NativeAVPlayerHost { /// then tunnels it through a 2-channel MAT carrier, so the route's channel count is not a /// statement about the audio and the surround-downmix warning below must not read it as one. var audioIsAtmosStreamCopy: Bool = false + /// Read the item's audio tracks back into `detectedAudioTracks`. Only the remote-HLS bypass needs + /// them; the loopback's probe already listed its audio, and the reads are XPC round trips. + var readsBackAudioTracks: Bool = false } /// AE#446 round 5: a fresh item is about to attach, invoked before anything can fetch a playlist @@ -470,6 +464,14 @@ final class NativeAVPlayerHost { self.sessionContract = contract mountedStartPosition = skipInitialSeek ? nil : (startPosition ?? 0) + mountSeekPending = Self.mountHoldsClock(inPlaceSwap: inPlaceSwap, skipInitialSeek: skipInitialSeek, + startPosition: startPosition, isLive: contract.isLive) + // A swap keeps the outgoing item's clock, which already reads the place it holds. A fresh + // mount has nothing there yet, so it states the place it is mounted at until the landing. + if mountSeekPending, !inPlaceSwap, let startPosition { + currentTime = startPosition + renderedTime = startPosition + } let forwardBufferDuration = contract.forwardBufferDuration let httpHeaders = contract.httpHeaders let armIngestFallback = contract.armIngestFallback @@ -484,6 +486,8 @@ final class NativeAVPlayerHost { self.liveJoinImmediateStartProbeInFlight = false self.liveJoinThinBufferLogged = false self.liveJoinNoDecisionLogged = false + self.liveJoinNotReadyLogged = false + self.liveJoinNotReadyAskedAgain = false // AE#440 round 4: both of these are per LOAD, and the host is reused across loads on the // keepNativeHost path. Left standing, the second live join of a reused host would arm no witness // at all and read the previous join's spend reason. @@ -654,10 +658,20 @@ final class NativeAVPlayerHost { self.avPlayer.play() } if self.timeControlStatus == .playing { + // Audit NAT-103: a carried `.playing` that never changes status on its way to + // motion is this item's roll from here, as for the AE#440 one-shot. + self.hasEverPlayed = true + self.mountSeekPending = false self.startLiveJoinImmediatelyIfHolding(waitingReason: "-") + } else if self.timeControlStatus == .waitingToPlayAtSpecifiedRate { + // AE#684: a hold refused for an item that could not play yet is asked again + // now that it can, on the playhead the item actually starts from. + self.startLiveJoinImmediatelyIfHolding( + waitingReason: self.avPlayer.reasonForWaitingToPlay?.rawValue ?? "-") } // #168: publish the item's real dynamic range for the probe-free remote-HLS badge. await self.publishDetectedVideoFormat(from: item) + await self.publishDetectedAudioTracks(from: item) guard self.sessionID == sid else { return } // #168 follow-up: watch for an advertised video rendition that never builds a track // (HEVC-in-MPEG-TS carriage); anchored at readyToPlay so dead origins never arm it. @@ -675,20 +689,10 @@ final class NativeAVPlayerHost { rateObservation = avPlayer.observe(\.rate, options: [.new]) { [weak self] player, _ in let rate = player.rate - let observedAt = DispatchTime.now().uptimeNanoseconds EngineLog.emit("[NativeAVPlayerHost] #\(sid) rate=\(rate)", category: .engine) Task { @MainActor in guard let self, self.sessionID == sid else { return } self.rate = rate - if rate != 0, observedAt >= self.engineStopUptime { self.rolledSinceEngineStop = true } - // AE#287: a stop AVPlayer reported during the premature-end re-seek is the recovery's, - // not the viewer's. Judged by when AVPlayer reported it: this hop can run after the - // recovery has ended. - let recoveryOwned = observedAt >= self.prematureEndRecoveryStartedUptime - && (self.prematureEndRecoveryInFlight || observedAt <= self.prematureEndRecoveryEndedUptime) - if rate != 0 || !recoveryOwned { - self.stampTransport(rolling: rate != 0, at: observedAt) - } } } @@ -721,7 +725,14 @@ final class NativeAVPlayerHost { self.timeControlStatus = status self.startLiveJoinImmediatelyIfHolding(waitingReason: reason) // First .playing: re-sample route after 2.5s settle -- AVKit only negotiates HDMI format on playback start (issue #24). - if status == .playing { self.hasEverPlayed = true } + // Audit NAT-103: an in-place swap reuses a player that is still `.playing`, and that status + // reaches the fresh item before it is ready. Taken as a roll it released the AE#629 hold + // before the mount seek landed and latched #50's "has played" for an item that had not. + if status == .playing, + Self.playingIsThisItemsRoll(itemIsReadyToPlay: self.playerItem?.status == .readyToPlay) { + self.hasEverPlayed = true + self.mountSeekPending = false + } if status == .playing, !self.didSampleSettledRoute { self.didSampleSettledRoute = true Task { @MainActor [weak self] in @@ -734,6 +745,7 @@ final class NativeAVPlayerHost { // #168: the video track can be absent from item.tracks at readyToPlay for HLS; // re-read once playing so the remote-HLS badge settles on the real dynamic range. await self.publishDetectedVideoFormat(from: item) + await self.publishDetectedAudioTracks(from: item) } } } @@ -794,10 +806,6 @@ final class NativeAVPlayerHost { surfaceEndFailures: false, hasEverPlayed: self.hasEverPlayed) { // #93 round 3: loopback path. Count the death for the engine's revive // escalation; a startup death (never played) stays with the startup watchdogs. - self.endFailureFollowedPause = Self.transportPausedBeforeFailure( - pausedSinceUptime: self.pausedSinceUptime, - failureUptime: DispatchTime.now().uptimeNanoseconds) - self.transportCommandSinceEndFailure = nil self.endFailureCount += 1 } } @@ -843,6 +851,7 @@ final class NativeAVPlayerHost { let value = time.seconds.isFinite ? time.seconds : 0 Task { @MainActor in guard let self, self.sessionID == sid else { return } + if self.mountSeekPending { return } // renderedTime tracks the parked on-screen frame mid-seek (issue #49). self.renderedTime = value // seekInFlight suppresses currentTime: AVPlayer still reports pre-seek clock until physical landing (issue #37). @@ -902,7 +911,13 @@ final class NativeAVPlayerHost { category: .engine) // Load-time seek (not a user scrub): no seekInFlight needed; the async seek(to:) carries #37/#38 semantics for user seeks. avPlayer.seek(to: CMTime(seconds: startPosition ?? 0, preferredTimescale: 600), - toleranceBefore: .zero, toleranceAfter: .zero) + toleranceBefore: .zero, toleranceAfter: .zero) { [weak self] finished in + guard finished else { return } + Task { @MainActor in + guard let self, self.sessionID == sid else { return } + self.mountSeekPending = false + } + } } } @@ -949,77 +964,6 @@ final class NativeAVPlayerHost { !surfaceEndFailures && hasEverPlayed } - /// Keeps `pausedSinceUptime` on the commanded transport: cleared when it rolls, stamped once when - /// it stops and left alone while it stays stopped. - private func stampTransport(rolling: Bool, at uptime: UInt64 = DispatchTime.now().uptimeNanoseconds) { - guard uptime >= transportStampEventUptime else { return } - transportStampEventUptime = uptime - if rolling { - pausedSinceUptime = nil - } else if pausedSinceUptime == nil { - pausedSinceUptime = uptime - } - } - - /// The dead item's own drop to rate 0 and its `failedToPlayToEndTime` land within a runloop turn - /// of each other, in either order (the two are unsynchronized, see #50). A stop older than this - /// was the viewer's. - nonisolated static let pausedBeforeFailureMarginSeconds: Double = 1.0 - - /// Pure decision: had the transport already stopped when the item died? Read from AVPlayer's - /// own `rate` rather than the #122 intent latch, because AVKit's transport bar, Control Center - /// and PiP pause and resume the player without passing through the engine. - nonisolated static func transportPausedBeforeFailure( - pausedSinceUptime: UInt64?, failureUptime: UInt64 - ) -> Bool { - guard let pausedSinceUptime, failureUptime > pausedSinceUptime else { return false } - let pausedSeconds = Double(failureUptime - pausedSinceUptime) / 1_000_000_000 - return pausedSeconds >= pausedBeforeFailureMarginSeconds - } - - /// Pure decision: does the reload of a dead item restart transport? A Play or Pause pressed - /// through the engine after the failure decides. Otherwise a transport rolling again (a Play from - /// AVKit, Control Center or PiP) resumes, and one that had stopped before the failure stays paused. - nonisolated static func itemDeathReloadResumesPlaying( - diedUnderPause: Bool, commandSinceFailure: Bool?, transportRolling: Bool - ) -> Bool { - if let commandSinceFailure { return commandSinceFailure } - return transportRolling || !diedUnderPause - } - - /// Whether the media fallback replacing the latest refused master (#98) should play. The engine - /// asks when the fallback runs, a task hop after the refusal, so a rate report or a transport - /// command raced with the refusal has landed, and before it swaps, which resets the records read. - func mediaFallbackResumesPlaying() -> Bool { - Self.mediaFallbackResumesPlaying( - commandSinceRejection: transportCommandSinceRejection, - intentIsPlaying: playIntent, - rolledSinceEngineStop: rolledSinceEngineStop, - pausedBeforeRejection: Self.transportPausedBeforeFailure( - pausedSinceUptime: pausedSinceUptime, failureUptime: displayRejectionUptime)) - } - - /// Pure decision: does the media fallback play the item that replaces a refused master? A Play or - /// Pause through the engine after the refusal decides. Otherwise a refused item was told to play, - /// since `item.status` does not advance before that, so it plays on unless the viewer paused it - /// before the refusal (the same one-second margin as an item death). Who told it to play is read - /// from both records: `playIntent` for the engine, and a roll since the engine's last stop for - /// AVKit, Control Center or PiP, which never touch the intent. An engine pause inside the margin - /// still counts, because it clears both. - nonisolated static func mediaFallbackResumesPlaying( - commandSinceRejection: Bool?, intentIsPlaying: Bool, rolledSinceEngineStop: Bool, - pausedBeforeRejection: Bool - ) -> Bool { - if let commandSinceRejection { return commandSinceRejection } - return !pausedBeforeRejection && (intentIsPlaying || rolledSinceEngineStop) - } - - /// The engine stopped the transport: a roll reported before now no longer says anyone wants it. - private func noteEngineStop() { - engineStopUptime = DispatchTime.now().uptimeNanoseconds - rolledSinceEngineStop = false - } - /// #50: AVPlayer fires .failed for self-healing transients (loopback 404, AVIOReader reconnect) while playback advances uninterrupted (rrgomes: tcs=playing at .failed). /// Discriminates on hasEverPlayed, not instantaneous timeControlStatus: .failed and timeControlStatus KVOs are unsynchronized (426b45c: still published terminal failure at 27.3s while AVPlayer played smoothly). /// Before first .playing: surface promptly (genuine startup failure). After: defer 5s and confirm -- clear if .playing or clock advanced, surface if both stopped. @@ -1091,8 +1035,6 @@ final class NativeAVPlayerHost { "[NativeAVPlayerHost] #\(sessionID) startup .failed is a master rejection " + "(code=\(code)); signalling engine for media fallback instead of surfacing", category: .engine) - displayRejectionUptime = DispatchTime.now().uptimeNanoseconds - transportCommandSinceRejection = nil pendingDisplayRejection = DisplayRejection(code: code, message: desc, domain: (item.error as NSError?)?.domain) @@ -1344,16 +1286,26 @@ final class NativeAVPlayerHost { /// behind the same `false` a genuinely starved join holds a fraction of a second, and starting /// there trades a still picture for an immediate stall. The depth is the axis that separates the /// two mechanisms, so it is the one the guard reads. + /// - `itemIsReadyToPlay` (AE#684): the depth is measured from `currentTime()`, and until the item + /// is ready that is where the loader started fetching, not where playback will begin. Under an + /// `EXT-X-START` placement AVPlayer fetches from about 6 s below the target first, so the reading + /// is the lookback itself: captured on a rejoin placed at 53.908 s as `buffer ahead 4.00s` from a + /// playhead of 48.00 s, every second of it behind the start point, on an item whose status had + /// not left `unknown`, 36 ms before it did. That is the starved start the depth guard exists to + /// refuse, read as a four-second cushion, and it was the one item of five in that capture to + /// be forced, and the one its viewer reported out of sync. The hold is asked again at readiness. nonisolated static func shouldStartLiveJoinImmediately( armed: Bool, alreadySpent: Bool, hostWantsToPlay: Bool, isWaitingToMinimizeStalls: Bool, playbackBufferEmpty: Bool, - bufferedAheadSeconds: Double + bufferedAheadSeconds: Double, + itemIsReadyToPlay: Bool ) -> Bool { guard armed, !alreadySpent, hostWantsToPlay else { return false } guard isWaitingToMinimizeStalls else { return false } + guard itemIsReadyToPlay else { return false } guard !playbackBufferEmpty else { return false } // NaN fails every comparison silently, and an unresolved item's currentTime() is NaN, so an // absent reading has to be rejected rather than fall through the bound below. @@ -1378,7 +1330,7 @@ final class NativeAVPlayerHost { private func startLiveJoinImmediatelyIfHolding(waitingReason: String) { guard liveJoinStartsImmediately, !liveJoinImmediateStartSpent else { return } if timeControlStatus == .playing { - if Self.playingSpendsLiveJoinOneShot(itemIsReadyToPlay: playerItem?.status == .readyToPlay) { + if Self.playingIsThisItemsRoll(itemIsReadyToPlay: playerItem?.status == .readyToPlay) { liveJoinImmediateStartSpent = true } return @@ -1394,10 +1346,16 @@ final class NativeAVPlayerHost { else { return } liveJoinImmediateStartProbeInFlight = true let probeStart = DispatchTime.now() + let sid = sessionID Task { @MainActor [weak self] in guard let self else { return } - defer { self.liveJoinImmediateStartProbeInFlight = false } let reading = await Self.liveJoinBufferReading(item) + // Audit NAT-4: a load during the read reset the one-shot and the in-flight flag for its own + // item, and a swap replaced the item this reading describes. Neither is this probe's to + // spend or clear. + guard self.sessionID == sid else { return } + self.liveJoinImmediateStartProbeInFlight = false + guard self.playerItem === item else { return } // AE#422: the reading is async, so the hold it described may be over. Decide on the state // that exists now, not on the edge that asked. guard !self.liveJoinImmediateStartSpent, @@ -1426,11 +1384,20 @@ final class NativeAVPlayerHost { hostWantsToPlay: self.playIntent, isWaitingToMinimizeStalls: true, playbackBufferEmpty: reading.bufferEmpty, - bufferedAheadSeconds: reading.aheadSeconds + bufferedAheadSeconds: reading.aheadSeconds, + itemIsReadyToPlay: reading.itemStatus == .readyToPlay ) else { - // The cushion, not the decision, is what a later report needs: it separates a join - // waiting on AVPlayer's rate estimate from one genuinely starved at the edge. - if !self.liveJoinThinBufferLogged { + // AE#684: a refusal the depth alone would have granted is the item's, not the + // cushion's. It has its own line, and the thin-buffer line would contradict it (it + // prints the same depth as the reason for leaving the wait alone). + if let line = Self.liveJoinNotReadyRefusal(reading: reading) { + if !self.liveJoinNotReadyLogged { + self.liveJoinNotReadyLogged = true + EngineLog.emit("[NativeAVPlayerHost] #\(self.sessionID) " + line, category: .engine) + } + } else if !self.liveJoinThinBufferLogged { + // The cushion, not the decision, is what a later report needs: it separates a join + // waiting on AVPlayer's rate estimate from one genuinely starved at the edge. self.liveJoinThinBufferLogged = true EngineLog.emit( "[NativeAVPlayerHost] #\(self.sessionID) AE#440 live join: leaving the " @@ -1443,6 +1410,18 @@ final class NativeAVPlayerHost { ) } self.startLiveJoinHoldWitness(item: item) + // The item can turn ready while this reading is in flight. The readiness sink asked + // at that moment and was turned away by the in-flight flag, so a reading taken on an + // item that could not play yet, thin or deep, is the last word unless the question + // is put again here. + if Self.liveJoinAsksAgainAfterNotReadyRefusal( + readingWasNotReady: reading.itemStatus != .readyToPlay, + itemIsReadyNow: item.status == .readyToPlay, + alreadyAskedAgain: self.liveJoinNotReadyAskedAgain) { + self.liveJoinNotReadyAskedAgain = true + self.startLiveJoinImmediatelyIfHolding( + waitingReason: self.avPlayer.reasonForWaitingToPlay?.rawValue ?? "-") + } return } self.liveJoinImmediateStartSpent = true @@ -1460,18 +1439,30 @@ final class NativeAVPlayerHost { } } - /// Whether a `.playing` transport status is this item's rate rolling, the event that spends the - /// AE#440 one-shot. + /// Whether a `.playing` transport status is this item's rate rolling: the event that spends the + /// AE#440 one-shot, latches #50's `hasEverPlayed` and releases the AE#629 swap hold (audit NAT-103). /// /// An in-place swap reuses a player that is still `.playing`, and that status reaches the fresh /// item as its first edge, before the item can play anything. Spent there, the one-shot was gone /// before the join's own `ToMinimizeStalls` hold began, so a #446 rejoin produced no decision and no /// line. An item cannot roll before it is ready, so readiness is what separates the carry from the /// roll. The readyToPlay sink asks again, for a carry that never changes status on its way to motion. - nonisolated static func playingSpendsLiveJoinOneShot(itemIsReadyToPlay: Bool) -> Bool { + nonisolated static func playingIsThisItemsRoll(itemIsReadyToPlay: Bool) -> Bool { itemIsReadyToPlay } + /// AE#629 round 5: whether a mount holds the clock until its mount seek lands. Every in-place swap + /// that seeks does (#646). A fresh mount does when it is VOD placed past the head: at the head + /// there is no earlier segment start to read, and a live mount's anchor sits on the item axis + /// while its clock is folded through the session shift. + nonisolated static func mountHoldsClock( + inPlaceSwap: Bool, skipInitialSeek: Bool, startPosition: Double?, isLive: Bool + ) -> Bool { + if skipInitialSeek { return false } + if inPlaceSwap { return true } + return !isLive && (startPosition ?? 0) > 0 + } + /// AE#440 round 3: what a refused hold did next, reported and never acted on. /// /// The guard above answers one instant, and the bound it read is read only at the hold's edges: a @@ -1533,7 +1524,8 @@ final class NativeAVPlayerHost { guard Self.shouldStartLiveJoinImmediately( armed: true, alreadySpent: false, hostWantsToPlay: true, isWaitingToMinimizeStalls: true, - playbackBufferEmpty: reading.bufferEmpty, bufferedAheadSeconds: reading.aheadSeconds + playbackBufferEmpty: reading.bufferEmpty, bufferedAheadSeconds: reading.aheadSeconds, + itemIsReadyToPlay: reading.itemStatus == .readyToPlay ) else { continue } account(.crossed) return @@ -1736,6 +1728,32 @@ final class NativeAVPlayerHost { } } + /// AE#684: whether a refusal taken on a not-ready item puts the question again, whatever the + /// depth it read (a thin reading on such an item is as stale as a deep one). Once per load, and + /// only when the item has become ready since the reading was taken: the reading is asynchronous, the readiness + /// sink's own question is dropped while one is in flight, and without this a hold that outlives + /// readiness on that race is never judged on the playhead it starts from. + nonisolated static func liveJoinAsksAgainAfterNotReadyRefusal(readingWasNotReady: Bool, + itemIsReadyNow: Bool, + alreadyAskedAgain: Bool) -> Bool { + readingWasNotReady && itemIsReadyNow && !alreadyAskedAgain + } + + /// AE#684: the refusal line for a hold that had the depth and not the item. nil for every other + /// refusal, which the thin-buffer line already accounts for. + nonisolated static func liveJoinNotReadyRefusal(reading: LiveJoinBufferReading) -> String? { + guard reading.itemStatus != .readyToPlay, !reading.bufferEmpty, + reading.aheadSeconds.isFinite, reading.aheadSeconds >= minimumLiveJoinBufferAhead + else { return nil } + let head = reading.playheadSeconds.isFinite + ? String(format: "%.2f", reading.playheadSeconds) + "s" + : "an unreadable position" + return "AE#440 live join: leaving the stall-avoidance wait alone on an item that cannot play " + + "yet (buffer ahead " + String(format: "%.2f", reading.aheadSeconds) + "s of playhead " + + head + ", which until readiness is where the fetch began and not where playback will); " + + "asked again at readiness" + } + /// The placement clause the two accounts below carry when the cushion reads zero. nil when a range /// contains the playhead: there the depth is the whole story and this would only add noise. nonisolated static func liveJoinPlacementClause(reading: LiveJoinBufferReading) -> String? { @@ -1763,21 +1781,12 @@ final class NativeAVPlayerHost { func play() { // Set intent before play() so readyToPlay observer can re-assert if the replaceCurrentItem swap swallowed it. playIntent = true - transportCommandSinceEndFailure = true - transportCommandSinceRejection = true - stampTransport(rolling: true) // Call play() immediately (no defer-until-ready): item.status never advances past .unknown until AVPlayer is told to play. avPlayer.play() } func pause() { playIntent = false - transportCommandSinceEndFailure = false - transportCommandSinceRejection = false - noteEngineStop() - // Stamped here as well as from the rate KVO: pausing a player whose rate is already 0 (a dead - // or parked item) changes nothing AVPlayer reports, and the viewer's pause must still count. - stampTransport(rolling: false) avPlayer.pause() } @@ -1829,7 +1838,6 @@ final class NativeAVPlayerHost { prematureEndRecoveryAttempts += 1 lastPrematureEndRecoveryPlayhead = playhead prematureEndRecoveryInFlight = true - prematureEndRecoveryStartedUptime = DispatchTime.now().uptimeNanoseconds EngineLog.emit( "[NativeAVPlayerHost] #\(sessionID) AE#287 premature end: playhead=" + "\(String(format: "%.3f", playhead))s duration=\(String(format: "%.3f", duration))s " @@ -1846,7 +1854,6 @@ final class NativeAVPlayerHost { // A viewer who paused through the engine while the re-seek was in flight keeps the pause. guard playIntent else { prematureEndRecoveryInFlight = false - prematureEndRecoveryEndedUptime = DispatchTime.now().uptimeNanoseconds timeControlStatus = avPlayer.timeControlStatus EngineLog.emit( "[NativeAVPlayerHost] #\(sessionID) AE#287 re-seeked; staying paused for the viewer", @@ -1855,10 +1862,6 @@ final class NativeAVPlayerHost { } avPlayer.play() prematureEndRecoveryInFlight = false - prematureEndRecoveryEndedUptime = DispatchTime.now().uptimeNanoseconds - // The premature end stopped the rate before the recovery began, and nobody paused: the - // recovery has now commanded play, so drop that stamp even if AVPlayer's rate has not moved. - stampTransport(rolling: true) timeControlStatus = avPlayer.timeControlStatus let resumedAt = await prematureEndReading().playhead EngineLog.emit( @@ -1909,6 +1912,8 @@ final class NativeAVPlayerHost { seekGeneration &+= 1 let gen = seekGeneration seekInFlight = true + // AE#629: this seek's own landing publishes the clock from here on. + mountSeekPending = false latestSeekRenderedTimePublished = false let resumeGuard = SeekResumeGuard() return await withCheckedContinuation { (cont: CheckedContinuation) in @@ -2038,10 +2043,6 @@ final class NativeAVPlayerHost { func setRate(_ value: Float) { // Non-zero rate counts as play intent (must survive replaceCurrentItem swap like play() does). playIntent = (value != 0) - transportCommandSinceEndFailure = (value != 0) - transportCommandSinceRejection = (value != 0) - if value == 0 { noteEngineStop() } - stampTransport(rolling: value != 0) // #436: `play()` is rate 1.0 by definition, and it is re-issued from paths no client can see: // the readyToPlay re-assert after an item swap, interruption and background resume, the #287 // premature-end recovery, plus AVKit's own transport and the remote command centre calling @@ -2188,6 +2189,9 @@ final class NativeAVPlayerHost { notificationObservers.removeAll() // Clear terminal flags: keepNativeHost reload reuses the host and @Published replays on subscribe; stale failure/didReachEnd corrupt the new session (issue #15). failure = nil + // Audit Vcore-102: the same replay would hand the successor's sinks this item's refusal. + pendingDisplayRejection = nil + pendingSoftwarePathEscalation = nil didReachEnd = false // #315: same reason. The layer itself still reads true for a few tens of ms past this point // (AVFoundation clears it after the swap), so the published value leads the layer here on @@ -2202,6 +2206,8 @@ final class NativeAVPlayerHost { detectedVideoFormat = nil detectedVideoFrameRate = nil detectedVideoCodecName = nil + detectedVideoDescription = nil + detectedAudioTracks = [] // #168 follow-up: the carriage verdict belongs to the outgoing item. carriageWatchdogTask?.cancel() carriageWatchdogTask = nil @@ -2217,11 +2223,6 @@ final class NativeAVPlayerHost { readinessDeadlineSeconds = nil // Re-arm #50 hasEverPlayed: reused host must not inherit prior session's established state. hasEverPlayed = false - if inPlaceSwap { - rolledSinceEngineStop = avPlayer.rate != 0 - } else { - noteEngineStop() - } // #93 recovery reload: same content, same position, playback must continue. Skip the // pause + nil-item gap below (PiP content-source invalidation + transport bounce); the // old item keeps playing until replaceCurrentItem swaps in the fresh one, and playIntent @@ -2241,6 +2242,10 @@ final class NativeAVPlayerHost { renderedTime = 0 duration = 0 rate = 0 + // The observation was invalidated above, so the pause just issued is never published. Left at the + // outgoing item's `.playing`, the next session's sink reads a roll on subscribe and then publishes + // its own pre-roll `.paused` as a real pause (a host raises its transport on it). + timeControlStatus = .paused // The AVAudioSession is NOT released here. Teardown ordering is the engine's call, not the host's: // AetherEngine.stopInternal deactivates once every render path is quiesced (#215). } @@ -2365,7 +2370,7 @@ final class NativeAVPlayerHost { break } let verdict = await HLSCarriageProbe.classifyDeferredSegmentHead( - url: segmentURL, httpHeaders: httpHeaders) + url: segmentURL, httpHeaders: httpHeaders, credentialOrigin: url) guard !Task.isCancelled, self.sessionID == sid else { return } self.publishCarriageProbeVerdict(verdict, sid: sid, from: "segment PMT") } @@ -2557,9 +2562,11 @@ final class NativeAVPlayerHost { let ext = CMFormatDescriptionGetExtensions(cm) as? [String: Any] ?? [:] let transfer = ext[kCMFormatDescriptionExtension_TransferFunction as String] as? String let fmt = RemoteHLSFormatDetection.videoFormat(transferFunction: transfer, videoSubType: subType) - // Rate and codec before format: the engine's format sink reads both when it fires. + // Rate, codec and description before format: the engine's format sink reads them when it fires. if let rate, rate > 0 { detectedVideoFrameRate = rate } detectedVideoCodecName = RemoteHLSFormatDetection.codecName(videoSubType: subType) + let description = RemoteHLSStreamDescription.video(from: cm) + if detectedVideoDescription != description { detectedVideoDescription = description } if detectedVideoFormat != fmt { detectedVideoFormat = fmt EngineLog.emit( @@ -2573,6 +2580,34 @@ final class NativeAVPlayerHost { } } + /// The audio half of the read above, at the same two points. Language comes from the track, and where + /// the track has none (muxed HLS audio rarely does) from the audible option AVPlayer selected. + @MainActor + private func publishDetectedAudioTracks(from item: AVPlayerItem) async { + let sid = sessionID + guard sid != 0, playerItem === item, sessionContract.readsBackAudioTracks else { return } + var selectedOptionLanguage: String? + if let group = try? await item.asset.loadMediaSelectionGroup(for: .audible), + let option = item.currentMediaSelection.selectedMediaOption(in: group) { + selectedOptionLanguage = option.extendedLanguageTag + } + var readings: [RemoteHLSStreamDescription.AudioReading] = [] + for itemTrack in item.tracks { + guard let assetTrack = itemTrack.assetTrack, assetTrack.mediaType == .audio else { continue } + guard let cm = try? await assetTrack.load(.formatDescriptions).first else { continue } + let extendedTag = try? await assetTrack.load(.extendedLanguageTag) + let languageCode = try? await assetTrack.load(.languageCode) + let language = [extendedTag ?? nil, languageCode ?? nil, selectedOptionLanguage] + .compactMap { $0 }.first { !$0.isEmpty && $0 != "und" } + if let reading = RemoteHLSStreamDescription.audioReading( + from: cm, isEnabled: itemTrack.isEnabled, language: language) { + readings.append(reading) + } + } + guard sessionID == sid, playerItem === item else { return } + if detectedAudioTracks != readings { detectedAudioTracks = readings } + } + /// Compact video track summary: dimensions + color attachments (primaries/transfer/matrix). Mismatch vs source-side codecpar signals DV/HDR signaling didn't survive the muxer. /// Dump item.tracks on .failed (FourCC per track). Async: AVAssetTrack.formatDescriptions is /// load-based; assetTrack access is main-actor. @@ -2716,24 +2751,9 @@ final class NativeAVPlayerHost { /// Dump audio route channel capability post-load (route renegotiates on asset load; pre-load poll is stale). outputNumberOfChannels is the actual LPCM limit; EAC3/Atmos bypasses it via bitstream tunnel. nonisolated private static func dumpAudioRoute(sid: Int, phase: String) { - #if os(iOS) || os(tvOS) - let session = AVAudioSession.sharedInstance() - let out = session.outputNumberOfChannels - let pref = session.preferredOutputNumberOfChannels - let maxCh = session.maximumOutputNumberOfChannels - let route = session.currentRoute - let outputDescs = route.outputs.map { port in - let portName = port.portName - let portType = port.portType.rawValue - let nChannels = port.channels?.count ?? -1 - return "\(portName)[\(portType), ch=\(nChannels)]" - }.joined(separator: ", ") - EngineLog.emit( - "[NativeAVPlayerHost] #\(sid) audioRoute output=\(out) preferred=\(pref) max=\(maxCh) " - + "ports=[\(outputDescs)] (\(phase))", - category: .engine - ) - #endif + // AE#684: the route's own latency rides on the line, per item, beside the item's start. + guard let route = AudioRouteDescription.current() else { return } + EngineLog.emit("[NativeAVPlayerHost] #\(sid) audioRoute \(route) (\(phase))", category: .engine) } /// Read sr/ch/bits/layoutTag from CMAudioFormatDescription. Layout tag diagnoses where downmix occurs: unknown/stereo tag = AVPlayer parse layer; correct 7.1 tag = route/soundbar layer. @@ -2741,7 +2761,7 @@ final class NativeAVPlayerHost { var parts: [String] = [] if let asbdPtr = CMAudioFormatDescriptionGetStreamBasicDescription(fmt) { let asbd = asbdPtr.pointee - parts.append("sr=\(Int(asbd.mSampleRate))") + parts.append("sr=\(RemoteHLSStreamDescription.wholeSampleRate(asbd.mSampleRate))") parts.append("ch=\(asbd.mChannelsPerFrame)") parts.append(String(format: "bits=%d", asbd.mBitsPerChannel)) parts.append("fmt=\(fourccString(asbd.mFormatID))") diff --git a/Sources/AetherEngine/Native/RemoteHLSCueClock.swift b/Sources/AetherEngine/Native/RemoteHLSCueClock.swift new file mode 100644 index 000000000..beae55f62 --- /dev/null +++ b/Sources/AetherEngine/Native/RemoteHLSCueClock.swift @@ -0,0 +1,108 @@ +import Foundation + +/// AE#616: the offset between AVPlayer's item time and the media timestamps of what it presents, on the +/// `nativeRemoteHLS` bypass, measured off the #316 renditions the engine injected itself. +/// +/// A server that restarts a transcode at the keyframe before a segment's slot (Jellyfin: `-ss +/// -noaccurate_seek -copyts`) serves a first segment whose `tfdt` lies before the slot its playlist +/// promises. AVPlayer anchors its item timeline to the first segment it loads after a seek, so item +/// time then leads the displayed frame by that gap, 1 to 8 s measured, different on every seek, and +/// the anchor survives later job restarts. Nothing on the bypass sees a segment, so the gap cannot be +/// read where it arises. +/// +/// The injected WebVTT is placed by media timestamp, so each cue reaches the legible output at +/// `item = cue.start + offset`. Matching the presented text back to the cue the engine wrote gives the +/// offset, and it re-measures on every line. Pure so the matching is testable without an item. +struct RemoteHLSCueClock { + + /// Offsets outside this band are a different line with the same text, not a measurement. The gap + /// is at most one keyframe interval in the seek direction; the band is wide on purpose. + static let plausibleOffsets: ClosedRange = -2.0...30.0 + + /// Item seconds minus media seconds of the presented frame. Nil until the first line is matched. + private(set) var offset: Double? + + /// Whether `offset` was measured since the last time jump. A jump keeps the previous offset as the + /// best guess, but the anchor may have moved, so until the next matched line it is only a guess. + private(set) var isMeasuredSinceJump = false + + private var startsByText: [String: [Double]] = [:] + private var presented: Set = [] + /// The first delivery after a time jump reports whatever is active at the landing, stamped with + /// the landing time rather than the cue's start, so it teaches nothing. + private var skipNextDelivery = true + + var hasCues: Bool { !startsByText.isEmpty } + + mutating func setCues(_ cues: [(start: Double, text: String)]) { + var index: [String: [Double]] = [:] + for cue in cues { + let key = Self.normalize(cue.text) + guard !key.isEmpty else { continue } + index[key, default: []].append(cue.start) + } + startsByText = index + } + + /// A seek or an item change: the anchor may move, and the next delivery is a landing. + mutating func noteTimeJump() { + presented = [] + skipNextDelivery = true + isMeasuredSinceJump = false + } + + /// Feed one legible-output delivery. Returns the new offset when this delivery measured one. + mutating func observe(strings: [String], itemTime: Double) -> Double? { + let now = Set(strings.map(Self.normalize).filter { !$0.isEmpty }) + let appeared = now.subtracting(presented) + presented = now + if skipNextDelivery { + skipNextDelivery = false + return nil + } + // A cue ending changes the set without starting anything; only an appearing line is stamped + // with its own start. + guard !appeared.isEmpty else { return nil } + + var unique: Double? + var ambiguous: [Double] = [] + for text in appeared { + let candidates = (startsByText[text] ?? []) + .map { itemTime - $0 } + .filter { Self.plausibleOffsets.contains($0) } + if candidates.count == 1 { + unique = candidates[0] + break + } + ambiguous.append(contentsOf: candidates) + } + let reference = offset ?? 0 + guard let measured = unique ?? ambiguous.min(by: { abs($0 - reference) < abs($1 - reference) }) + else { return nil } + offset = measured + isMeasuredSinceJump = true + return measured + } + + /// What the legible output hands back, reduced to what the served `.vtt` said: the rendition's + /// text went through `MovTextSampleBuilder.sanitize`, and AVPlayer drops the WebVTT markup and may + /// re-break lines. + static func normalize(_ text: String) -> String { + let sanitized = MovTextSampleBuilder.sanitize(text) + // One forward pass: removing a tag at a time and searching again from the start cost tags x + // length, seconds for a crafted 64 KiB cue. Once a `<` finds no `>`, no later one can either. + // Scanned by Character, as before, so a `<` carrying a combining mark is still text. + var s = "" + var rest = sanitized[...] + while let open = rest.firstIndex(of: "<"), let close = rest[open...].firstIndex(of: ">") { + s.append(contentsOf: rest[..") + .replacingOccurrences(of: " ", with: " ") + .replacingOccurrences(of: "&", with: "&") + return s.split(whereSeparator: { $0.isWhitespace || $0.isNewline }).joined(separator: " ") + } +} diff --git a/Sources/AetherEngine/Native/RemoteHLSCueClockObserver.swift b/Sources/AetherEngine/Native/RemoteHLSCueClockObserver.swift new file mode 100644 index 000000000..e2cbacf84 --- /dev/null +++ b/Sources/AetherEngine/Native/RemoteHLSCueClockObserver.swift @@ -0,0 +1,80 @@ +import AVFoundation + +/// AE#616: feeds `RemoteHLSCueClock` from a legible output on the bypass item. +/// +/// The output never suppresses player rendering. A host that draws AVPlayer's renditions keeps them, a +/// host with its own suppressing output keeps that, and the frozen-line trap (an output that starts +/// suppressing while a line is on screen leaves it there for good) cannot be triggered from here. +/// AVPlayer only delivers the selected legible option, so this measures exactly while one of the +/// injected renditions is selected and is silent otherwise. +@MainActor +final class RemoteHLSCueClockObserver: NSObject, AVPlayerItemLegibleOutputPushDelegate { + + private let output = AVPlayerItemLegibleOutput() + private weak var item: AVPlayerItem? + private let provider: RemoteHLSSubtitleProvider + private var clock = RemoteHLSCueClock() + private var jumpObserver: NSObjectProtocol? + private let onOffset: @MainActor (Double) -> Void + private let onTimeJump: @MainActor () -> Void + + init(item: AVPlayerItem, provider: RemoteHLSSubtitleProvider, + onOffset: @escaping @MainActor (Double) -> Void, + onTimeJump: @escaping @MainActor () -> Void) { + self.item = item + self.provider = provider + self.onOffset = onOffset + self.onTimeJump = onTimeJump + super.init() + output.suppressesPlayerRendering = false + output.advanceIntervalForDelegateInvocation = 0 + output.setDelegate(self, queue: .main) + item.add(output) + jumpObserver = NotificationCenter.default.addObserver( + forName: AVPlayerItem.timeJumpedNotification, object: item, queue: .main + ) { [weak self] _ in + MainActor.assumeIsolated { + guard let self else { return } + self.clock.noteTimeJump() + self.onTimeJump() + } + } + } + + func detach() { + if let jumpObserver { NotificationCenter.default.removeObserver(jumpObserver) } + jumpObserver = nil + output.setDelegate(nil, queue: nil) + item?.remove(output) + item = nil + } + + nonisolated func legibleOutput(_ output: AVPlayerItemLegibleOutput, + didOutputAttributedStrings strings: [NSAttributedString], + nativeSampleBuffers nativeSamples: [Any], + forItemTime itemTime: CMTime) { + let texts = strings.map(\.string) + let seconds = itemTime.seconds + guard seconds.isFinite else { return } + MainActor.assumeIsolated { self.deliver(texts: texts, itemTime: seconds) } + } + + private func deliver(texts: [String], itemTime: Double) { + // The fill runs from mount; a line can be presented before it finished. Index once it has. + if !clock.hasCues, provider.isFillFinished { + clock.setCues(provider.allCueStarts()) + } + guard clock.hasCues else { return } + let previous = clock.offset + guard let offset = clock.observe(strings: texts, itemTime: itemTime) else { return } + if previous == nil || abs(offset - (previous ?? 0)) >= 0.05 { + EngineLog.emit( + "[AetherEngine] AE#616: item time leads the presented frame by " + + "\(String(format: "%.3f", offset))s (was " + + (previous.map { String(format: "%.3f", $0) } ?? "unmeasured") + + "), measured at item \(String(format: "%.3f", itemTime))s", + category: .engine) + } + onOffset(offset) + } +} diff --git a/Sources/AetherEngine/Native/RemoteHLSMasterRewrite.swift b/Sources/AetherEngine/Native/RemoteHLSMasterRewrite.swift index 1530bbc31..15d14118c 100644 --- a/Sources/AetherEngine/Native/RemoteHLSMasterRewrite.swift +++ b/Sources/AetherEngine/Native/RemoteHLSMasterRewrite.swift @@ -59,14 +59,36 @@ enum RemoteHLSMasterRewrite { originURL: URL, renditions: [Rendition], mediaPlaylistBandwidth: Int = 5_000_000) throws -> String { + try rewriteDeclaringNames(originPlaylist: originPlaylist, originURL: originURL, + renditions: renditions, + mediaPlaylistBandwidth: mediaPlaylistBandwidth).master + } + + /// The rewritten master, and the NAME it declares for each rendition, in `renditions` order. + struct Rewritten: Equatable, Sendable { + var master: String + var renditionNames: [String] + } + + /// `rewrite`, also answering with the exact NAME each rendition was written under. The engine + /// selects an injected rendition by that NAME and hides it from the legible list by it, so it has + /// to key on what the master says, not on the name it asked for (audit NAT-2): a sidecar that + /// repeats another's name, or the origin's, is written with a disambiguator. + static func rewriteDeclaringNames(originPlaylist: String, + originURL: URL, + renditions: [Rendition], + mediaPlaylistBandwidth: Int = 5_000_000) throws -> Rewritten { guard !renditions.isEmpty else { throw Refusal.noRenditions } let lines = originPlaylist.components(separatedBy: .newlines) guard lines.contains(where: { $0.trimmingCharacters(in: .whitespaces).hasPrefix("#EXTM3U") }) else { throw Refusal.notAPlaylist } guard lines.contains(where: { $0.hasPrefix("#EXT-X-STREAM-INF:") }) else { - return wrapMediaPlaylist(originURL: originURL, renditions: renditions, - bandwidth: mediaPlaylistBandwidth) + let names = declaredNames(for: renditions, taken: []) + return Rewritten( + master: wrapMediaPlaylist(originURL: originURL, renditions: renditions, names: names, + bandwidth: mediaPlaylistBandwidth), + renditionNames: names) } return try rewriteMaster(lines: lines, originURL: originURL, renditions: renditions) } @@ -75,20 +97,19 @@ enum RemoteHLSMasterRewrite { private static func rewriteMaster(lines: [String], originURL: URL, - renditions: [Rendition]) throws -> String { + renditions: [Rendition]) throws -> Rewritten { // Which groups the injected renditions have to join: every SUBTITLES group the origin declares // (a variant references exactly one, and different variants may reference different ones), plus // our own group if any variant references none. var originGroups: [String] = [] - var namesPerGroup: [String: Set] = [:] + var originNames: Set = [] for line in lines where line.hasPrefix("#EXT-X-MEDIA:") { guard HLSPlaylistParser.attribute("TYPE", in: line) == "SUBTITLES", let group = HLSPlaylistParser.attribute("GROUP-ID", in: line) else { continue } if !originGroups.contains(group) { originGroups.append(group) } - if let name = HLSPlaylistParser.attribute("NAME", in: line) { - namesPerGroup[group, default: []].insert(name) - } + if let name = HLSPlaylistParser.attribute("NAME", in: line) { originNames.insert(name) } } + let names = declaredNames(for: renditions, taken: originNames) let variantsWithoutGroup = lines.contains { $0.hasPrefix("#EXT-X-STREAM-INF:") && HLSPlaylistParser.attribute("SUBTITLES", in: $0) == nil } @@ -113,8 +134,8 @@ enum RemoteHLSMasterRewrite { // The renditions have to be declared before the first variant references their group. if !injected { out.append(contentsOf: mediaTags(renditions: renditions, - groups: targetGroups, - takenNames: namesPerGroup)) + names: names, + groups: targetGroups)) injected = true } out.append(withSubtitlesGroup(trimmed)) @@ -124,7 +145,7 @@ enum RemoteHLSMasterRewrite { out.append(rewriteURIAttribute(in: line, against: originURL)) } guard sawVariantURI else { throw Refusal.masterWithoutVariants } - return out.joined(separator: "\n") + "\n" + return Rewritten(master: out.joined(separator: "\n") + "\n", renditionNames: names) } /// A variant that already names a SUBTITLES group keeps it (the injected renditions joined that @@ -138,11 +159,12 @@ enum RemoteHLSMasterRewrite { private static func wrapMediaPlaylist(originURL: URL, renditions: [Rendition], + names: [String], bandwidth: Int) -> String { var lines = ["#EXTM3U", "#EXT-X-INDEPENDENT-SEGMENTS"] lines.append(contentsOf: mediaTags(renditions: renditions, - groups: [injectedGroupID], - takenNames: [:])) + names: names, + groups: [injectedGroupID])) lines.append("#EXT-X-STREAM-INF:BANDWIDTH=\(bandwidth),SUBTITLES=\"\(injectedGroupID)\"") lines.append(originURL.absoluteString) return lines.joined(separator: "\n") + "\n" @@ -156,14 +178,12 @@ enum RemoteHLSMasterRewrite { /// rendition whose language matches the selected audio regardless of the user's caption preference /// (Sodalite#38). `isForced` still rides the published `TrackInfo` so a host can label and pick it. private static func mediaTags(renditions: [Rendition], - groups: [String], - takenNames: [String: Set]) -> [String] { + names: [String], + groups: [String]) -> [String] { var tags: [String] = [] for group in groups { - var used = takenNames[group] ?? [] - for rendition in renditions { - let name = uniqueName(rendition.name, in: &used) - var attrs = ["TYPE=SUBTITLES", "GROUP-ID=\"\(group)\"", "NAME=\"\(escaped(name))\""] + for (rendition, name) in zip(renditions, names) { + var attrs = ["TYPE=SUBTITLES", "GROUP-ID=\"\(group)\"", "NAME=\"\(name)\""] if let language = rendition.language, !language.isEmpty { attrs.append("LANGUAGE=\"\(escaped(language))\"") } @@ -181,7 +201,14 @@ enum RemoteHLSMasterRewrite { /// NAMEs must be unique within a group or AVFoundation collapses the legible options into one. The /// origin's own names are already taken, so a sidecar that repeats one gets a numeric disambiguator - /// rather than silently disappearing from the picker. + /// rather than silently disappearing from the picker. One name per rendition across every group, + /// taken against the origin names of all of them, so the engine can select it by that one string; + /// uniqueness is decided on the escaped form, which is the one the master carries. + private static func declaredNames(for renditions: [Rendition], taken: Set) -> [String] { + var used = taken + return renditions.map { uniqueName(escaped($0.name), in: &used) } + } + private static func uniqueName(_ preferred: String, in used: inout Set) -> String { let base = preferred.isEmpty ? "Subtitle" : preferred if !used.contains(base) { @@ -196,8 +223,21 @@ enum RemoteHLSMasterRewrite { } /// Quoted attribute values cannot carry a `"`; a name that contains one would truncate the tag. + /// Nor a line break or any other control character (audit NAT-6): a sidecar name is host input, + /// often a file name or server metadata, and a CR/LF in it would end the tag and let the rest of + /// the name write its own master lines, a variant included. Mapped per scalar, because CRLF is a + /// single Character and a Character-level test would let it through as one. private static func escaped(_ value: String) -> String { - value.replacingOccurrences(of: "\"", with: "'") + var out = String.UnicodeScalarView() + for scalar in value.unicodeScalars { + switch scalar.properties.generalCategory { + case .control, .lineSeparator, .paragraphSeparator: + out.append(" ") + default: + out.append(scalar == "\"" ? "'" : scalar) + } + } + return String(out) } // MARK: - URI absolutisation diff --git a/Sources/AetherEngine/Native/RemoteHLSMediaSelection.swift b/Sources/AetherEngine/Native/RemoteHLSMediaSelection.swift index 7e58a4d3e..addb12117 100644 --- a/Sources/AetherEngine/Native/RemoteHLSMediaSelection.swift +++ b/Sources/AetherEngine/Native/RemoteHLSMediaSelection.swift @@ -62,6 +62,14 @@ enum RemoteHLSMediaSelection { option.playlistName ?? option.displayName } + /// The option that carries an injected rendition's served NAME (audit NAT-104). Matched on + /// `injectionKey`, the identity the dedupe and the auto-select mirror already use: a first-hit test + /// of `playlistName == name || displayName == name` picked the ORIGIN's option whenever its localized + /// display name equalled the sidecar's NAME, and the origin's renditions sit ahead of the injected ones. + static func injectedRenditionIndex(named name: String, in options: [LegibleOption]) -> Int? { + options.firstIndex { injectionKey($0) == name } + } + /// Reroute only the typed VOD-path misroute, and only for URL sources: custom readers have no /// URL for AVPlayer to open, and the live raw-path misroute has its own destination /// (`shouldRouteLiveOntoIngest`, AE#363). @@ -88,6 +96,34 @@ enum RemoteHLSMediaSelection { /// Same two guards as the VOD side: only the typed misroute, and only for URL sources, because a /// custom reader has no playlist URL to ingest from and keeps the typed rejection. The reader only /// classifies as `hlsPlaylistOnRawLivePath` on the live path, so `isLive` needs no guard here. + /// AE#678: a live URL already known to answer with a playlist goes straight onto the ingest. The + /// raw path would open it, read `#EXTM3U`, throw it away and send the load around through AE#363, + /// which is one discarded request per zap, and against a single-connection origin one more request + /// the ingest's own playlist fetch has to queue behind. Known means a `.m3u8` / `.m3u` path, or a + /// URL that has taken the AE#363 route in this process before (IPTV channel URLs often carry no + /// extension at all). Keyed on the whole URL: one Xtream path serves TS or HLS depending on its query. + static func isKnownLivePlaylist(_ url: URL) -> Bool { + if ["m3u8", "m3u"].contains(url.pathExtension.lowercased()) { return true } + return knownPlaylistLock.withLock { knownPlaylistURLs.contains(url.absoluteString) } + } + + static func noteLivePlaylist(_ url: URL) { + knownPlaylistLock.withLock { + let key = url.absoluteString + knownPlaylistURLs.removeAll { $0 == key } + knownPlaylistURLs.append(key) + if knownPlaylistURLs.count > knownPlaylistCapacity { knownPlaylistURLs.removeFirst() } + } + } + + static func forgetKnownLivePlaylistsForTesting() { + knownPlaylistLock.withLock { knownPlaylistURLs.removeAll() } + } + + private static let knownPlaylistLock = NSLock() + nonisolated(unsafe) private static var knownPlaylistURLs: [String] = [] + private static let knownPlaylistCapacity = 512 + static func shouldRouteLiveOntoIngest(failure: Error?, isCustomSource: Bool) -> Bool { guard !isCustomSource, let readerError = failure as? AVIOReaderError, diff --git a/Sources/AetherEngine/Native/RemoteHLSStreamDescription.swift b/Sources/AetherEngine/Native/RemoteHLSStreamDescription.swift new file mode 100644 index 000000000..7cb05cc1a --- /dev/null +++ b/Sources/AetherEngine/Native/RemoteHLSStreamDescription.swift @@ -0,0 +1,303 @@ +import Foundation +import CoreMedia +import AudioToolbox +import AetherLibavutil +import AetherLibavcodec + +/// The `nativeRemoteHLS` bypass runs no libav probe, so the fields a stats panel reads (dimensions, colour +/// description, audio tracks) had no source on it, and a host fell back to metadata of the ORIGINAL file: +/// a capped 1280x720 H.264 transcode read "3840x2160, Main 10". AVPlayer has already parsed the delivered +/// stream into its item tracks' format descriptions, so this maps those onto the probe path's vocabulary +/// (libav names), without a second connection to the origin. Pure so it is unit-testable; the host does +/// the async track reads. +enum RemoteHLSStreamDescription { + + /// First id of the audio tracks published on the bypass. Synthetic, like the legible range at + /// `RemoteHLSMediaSelection.subtitleTrackIDBase`: there is no AVStream index on this route, and a + /// small ordinal would collide with a host's own source stream numbering. + static let audioTrackIDBase = 400_000 + + // MARK: - Video + + struct Video: Equatable, Sendable { + let width: Int32 + let height: Int32 + let codecName: String? + let format: VideoStreamFormat + } + + static func video(from description: CMFormatDescription) -> Video? { + guard CMFormatDescriptionGetMediaType(description) == kCMMediaType_Video else { return nil } + let subType = CMFormatDescriptionGetMediaSubType(description) + let dims = CMVideoFormatDescriptionGetDimensions(description) + let ext = CMFormatDescriptionGetExtensions(description) as? [String: Any] ?? [:] + return Video( + width: dims.width, height: dims.height, + codecName: RemoteHLSFormatDetection.codecName(videoSubType: subType), + format: videoStreamFormat(extensions: ext)) + } + + static func videoStreamFormat(extensions ext: [String: Any]) -> VideoStreamFormat { + let atoms = ext[kCMFormatDescriptionExtension_SampleDescriptionExtensionAtoms as String] as? [String: Any] ?? [:] + let config: CodecConfiguration? = + (atoms["avcC"] as? Data).flatMap(avcConfiguration) + ?? (atoms["hvcC"] as? Data).flatMap(hevcConfiguration) + let declaredDepth = (ext[kCMFormatDescriptionExtension_BitsPerComponent as String] as? NSNumber)?.intValue + let range: String? + if let full = ext[kCMFormatDescriptionExtension_FullRangeVideo as String] as? Bool { + range = full ? "pc" : "tv" + } else { + range = nil + } + return VideoStreamFormat( + pixelFormat: config?.pixelFormat, + bitDepth: config?.bitDepth ?? declaredDepth, + colorPrimaries: (ext[kCMFormatDescriptionExtension_ColorPrimaries as String] as? String).map(primariesName), + transfer: (ext[kCMFormatDescriptionExtension_TransferFunction as String] as? String).map(transferName), + matrix: (ext[kCMFormatDescriptionExtension_YCbCrMatrix as String] as? String).map(matrixName), + range: range, + profile: config?.profileName) + } + + /// CoreMedia's colour names onto libav's. An unknown name passes through: a value CoreMedia adds later + /// is still the stream's own declaration, and dropping it would read as "untagged". + static func primariesName(_ name: String) -> String { primariesNames[name] ?? name } + static func transferName(_ name: String) -> String { transferNames[name] ?? name } + static func matrixName(_ name: String) -> String { matrixNames[name] ?? name } + + private static let primariesNames: [String: String] = [ + kCMFormatDescriptionColorPrimaries_ITU_R_709_2 as String: "bt709", + kCMFormatDescriptionColorPrimaries_ITU_R_2020 as String: "bt2020", + kCMFormatDescriptionColorPrimaries_EBU_3213 as String: "bt470bg", + kCMFormatDescriptionColorPrimaries_SMPTE_C as String: "smpte170m", + kCMFormatDescriptionColorPrimaries_P3_D65 as String: "smpte432", + kCMFormatDescriptionColorPrimaries_DCI_P3 as String: "smpte431", + ] + + private static let transferNames: [String: String] = [ + kCMFormatDescriptionTransferFunction_ITU_R_709_2 as String: "bt709", + kCMFormatDescriptionTransferFunction_SMPTE_ST_2084_PQ as String: "smpte2084", + kCMFormatDescriptionTransferFunction_ITU_R_2100_HLG as String: "arib-std-b67", + kCMFormatDescriptionTransferFunction_ITU_R_2020 as String: "bt2020-10", + kCMFormatDescriptionTransferFunction_SMPTE_240M_1995 as String: "smpte240m", + kCMFormatDescriptionTransferFunction_Linear as String: "linear", + kCMFormatDescriptionTransferFunction_sRGB as String: "iec61966-2-1", + ] + + private static let matrixNames: [String: String] = [ + kCMFormatDescriptionYCbCrMatrix_ITU_R_709_2 as String: "bt709", + kCMFormatDescriptionYCbCrMatrix_ITU_R_601_4 as String: "smpte170m", + kCMFormatDescriptionYCbCrMatrix_ITU_R_2020 as String: "bt2020nc", + kCMFormatDescriptionYCbCrMatrix_SMPTE_240M_1995 as String: "smpte240m", + ] + + /// What a decoder configuration record declares: libav's profile name, and where the record pins + /// them, chroma format and luma depth as a libav pixel format. + struct CodecConfiguration: Equatable { + let profileName: String? + let pixelFormat: String? + let bitDepth: Int? + } + + /// ISO/IEC 14496-15 avcC. The chroma/depth extension exists only for the High family, and there only + /// when the muxer wrote it; the lower profiles are 8-bit 4:2:0 by definition, and so is plain High. + static func avcConfiguration(_ record: Data) -> CodecConfiguration? { + let b = [UInt8](record) + guard b.count >= 7, b[0] == 1 else { return nil } + let idc = Int32(b[1]) + let compatibility = b[2] + var profile = idc + if idc == 66, compatibility & 0x40 != 0 { profile |= 1 << 9 } // AV_PROFILE_H264_CONSTRAINED + if [110, 122, 244].contains(idc), compatibility & 0x10 != 0 { profile |= 1 << 11 } // _INTRA + let highFamily: Set = [100, 110, 122, 244, 44, 83, 86, 118, 128, 138, 139, 134, 135] + + var chroma: Int? + var depth: Int? + if highFamily.contains(idc) { + // Skip the SPS and PPS arrays to reach the extension. + var i = 5 + let spsCount = Int(b[i] & 0x1F); i += 1 + var ok = true + for _ in 0.. CodecConfiguration? { + let b = [UInt8](record) + guard b.count >= 23, b[0] == 1 else { return nil } + let depth = Int(b[17] & 0x07) + 8 + return CodecConfiguration( + profileName: VideoStreamFormat.profileName(codecID: AV_CODEC_ID_HEVC, profile: Int32(b[1] & 0x1F)), + pixelFormat: pixelFormatName(chroma: Int(b[16] & 0x03), depth: depth), + bitDepth: depth) + } + + /// The pixel format libav's decoder reports for a chroma format and depth, checked against libav's own + /// table so a combination it has no name for stays nil instead of becoming a made-up one. + static func pixelFormatName(chroma: Int?, depth: Int?) -> String? { + guard let chroma, let depth else { return nil } + let base: String + switch chroma { + case 0: base = "gray" + case 1: base = "yuv420p" + case 2: base = "yuv422p" + case 3: base = "yuv444p" + default: return nil + } + let name = depth == 8 ? base : "\(base)\(depth)le" + return av_get_pix_fmt(name) == AV_PIX_FMT_NONE ? nil : name + } + + // MARK: - Audio + + /// One audio track as AVPlayer built it, reduced to values so the mapping below stays pure. + struct AudioReading: Equatable, Sendable { + let formatID: AudioFormatID + let sampleRate: Double + let channels: Int + let bitsPerChannel: Int + let carriesJOC: Bool + let isEnabled: Bool + let language: String? + } + + static func audioReading(from description: CMFormatDescription, isEnabled: Bool, language: String?) -> AudioReading? { + guard CMFormatDescriptionGetMediaType(description) == kCMMediaType_Audio, + let asbd = CMAudioFormatDescriptionGetStreamBasicDescription(description)?.pointee else { return nil } + var joc = false + if asbd.mFormatID == kAudioFormatEnhancedAC3 { + var size = 0 + if let cookie = CMAudioFormatDescriptionGetMagicCookie(description, sizeOut: &size), size > 0 { + joc = eac3CarriesJOC(dec3: Data(bytes: cookie, count: size)) + } + } + return AudioReading( + formatID: asbd.mFormatID, + sampleRate: asbd.mSampleRate, + channels: Int(asbd.mChannelsPerFrame), + bitsPerChannel: Int(asbd.mBitsPerChannel), + carriesJOC: joc, + isEnabled: isEnabled, + language: language) + } + + /// ETSI TS 102 366 Annex F dec3: `flag_ec3_extension_type_a` after the substream loop is the JOC + /// (Atmos) signal, the same one libav reads into E-AC-3 profile 30. Accepts the bare payload or the + /// whole box, since the magic cookie AVFoundation hands back has been seen both ways. + static func eac3CarriesJOC(dec3 record: Data) -> Bool { + var b = [UInt8](record) + if b.count >= 8, Array(b[4..<8]) == Array("dec3".utf8) { b.removeFirst(8) } + var reader = BitReader(bytes: b) + guard reader.skip(13), let indSubs = reader.read(3) else { return false } + for _ in 0...indSubs { + // fscod bsid reserved asvc bsmod acmod lfeon reserved + guard reader.skip(2 + 5 + 1 + 1 + 3 + 3 + 1 + 3), let depSubs = reader.read(4) else { return false } + guard reader.skip(depSubs > 0 ? 9 : 1) else { return false } + } + guard reader.skip(7), let flag = reader.read(1) else { return false } + return flag == 1 + } + + /// Audio format IDs onto the libavcodec name the probe path publishes, AAC's object type onto its profile. + static func audioCodec(formatID: AudioFormatID) -> (name: String, profile: String?) { + switch formatID { + case kAudioFormatMPEG4AAC: + return ("aac", VideoStreamFormat.profileName(codecID: AV_CODEC_ID_AAC, profile: 1)) + case kAudioFormatMPEG4AAC_HE: + return ("aac", VideoStreamFormat.profileName(codecID: AV_CODEC_ID_AAC, profile: 4)) + case kAudioFormatMPEG4AAC_HE_V2: + return ("aac", VideoStreamFormat.profileName(codecID: AV_CODEC_ID_AAC, profile: 28)) + case kAudioFormatMPEG4AAC_LD: return ("aac", "LD") + case kAudioFormatMPEG4AAC_ELD: return ("aac", "ELD") + case kAudioFormatAC3: return ("ac3", nil) + case kAudioFormatEnhancedAC3: return ("eac3", nil) + case kAudioFormatMPEGLayer1: return ("mp1", nil) + case kAudioFormatMPEGLayer2: return ("mp2", nil) + case kAudioFormatMPEGLayer3: return ("mp3", nil) + case kAudioFormatFLAC: return ("flac", nil) + case kAudioFormatOpus: return ("opus", nil) + case kAudioFormatAppleLossless: return ("alac", nil) + case kAudioFormatLinearPCM: return ("pcm", nil) + case 0x61632D34: return ("ac4", nil) // 'ac-4' + default: + let bytes = [UInt8((formatID >> 24) & 0xFF), UInt8((formatID >> 16) & 0xFF), + UInt8((formatID >> 8) & 0xFF), UInt8(formatID & 0xFF)] + return (String(bytes: bytes, encoding: .macOSRoman) ?? "unknown", nil) + } + } + + /// The published list: one entry per audio track AVPlayer built, ids from `audioTrackIDBase` in item + /// order, the enabled one as the active index. Named the way the probe path names an untitled track. + /// A parsed `mSampleRate` as a whole number, 0 when it cannot be one (audit NAT-106). A QuickTime + /// SoundDescriptionV2 carries the rate as a Float64 verbatim, so on the bypass the origin controls + /// it, and `Int(_:)` traps on NaN, infinity and anything past `Int`. + static func wholeSampleRate(_ rate: Double) -> Int { + rate.isFinite && rate >= 0 && rate < 1e9 ? Int(rate) : 0 + } + + static func audioTracks(_ readings: [AudioReading]) -> (tracks: [TrackInfo], activeID: Int?) { + var tracks: [TrackInfo] = [] + var activeID: Int? + for (i, reading) in readings.enumerated() { + let id = audioTrackIDBase + i + let codec = audioCodec(formatID: reading.formatID) + let language = reading.language.flatMap { $0.isEmpty || $0 == "und" ? nil : $0 } + let name = language.map { "\($0.uppercased()) (\(codec.name))" } ?? "Track \(i + 1) (\(codec.name))" + let profile = reading.carriesJOC + ? VideoStreamFormat.profileName(codecID: AV_CODEC_ID_EAC3, profile: 30) + : codec.profile + tracks.append(TrackInfo( + id: id, name: name, codec: codec.name, language: language, + channels: reading.channels, isDefault: reading.isEnabled, isAtmos: reading.carriesJOC, + sampleRate: wholeSampleRate(reading.sampleRate), + bitsPerSample: reading.bitsPerChannel, profile: profile)) + if reading.isEnabled, activeID == nil { activeID = id } + } + return (tracks, activeID) + } + + private struct BitReader { + let bytes: [UInt8] + var position = 0 + + init(bytes: [UInt8]) { self.bytes = bytes } + + mutating func read(_ count: Int) -> Int? { + guard position + count <= bytes.count * 8 else { return nil } + var value = 0 + for _ in 0..> (7 - UInt8(position % 8))) & 1 + value = value << 1 | Int(bit) + position += 1 + } + return value + } + + mutating func skip(_ count: Int) -> Bool { read(count) != nil || count == 0 } + } +} diff --git a/Sources/AetherEngine/Native/RemoteHLSSubtitleProvider.swift b/Sources/AetherEngine/Native/RemoteHLSSubtitleProvider.swift index fc54a1f13..8d2027028 100644 --- a/Sources/AetherEngine/Native/RemoteHLSSubtitleProvider.swift +++ b/Sources/AetherEngine/Native/RemoteHLSSubtitleProvider.swift @@ -44,7 +44,13 @@ final class RemoteHLSSubtitleProvider: HLSSegmentProvider, @unchecked Sendable { vttFillWaitSeconds: TimeInterval = defaultVTTFillWaitSeconds) { self.tracks = tracks self.staticMasterPlaylistBody = masterBody - self.programDuration = max(1, programDuration) + // `RemoteHLSSubtitleProxy.sumSegmentDurations` already refuses a non-finite, negative or + // absurd EXTINF sum before it reaches here; this is the last stop before the value leaves + // the provider (`segmentDuration(at:)`) for `wholeSecondsCovering`'s `Int(Double)`, which + // traps on `+inf` (audit NAT-1). Bounded again here so a caller that builds this provider + // directly, bypassing the proxy, cannot reintroduce the trap. + let finiteDuration = programDuration.isFinite ? programDuration : 1 + self.programDuration = min(max(1, finiteDuration), RemoteHLSSubtitleProxy.maxProgramDurationSeconds) self.defaultHeaders = defaultHeaders self.vttFillWaitSeconds = vttFillWaitSeconds self.stores = tracks.map { _ in NativeSubtitleCueStore() } @@ -102,6 +108,14 @@ final class RemoteHLSSubtitleProvider: HLSSegmentProvider, @unchecked Sendable { await currentFillTask()?.value } + /// AE#616: every sidecar's decode has finished, so `allCueStarts()` is the whole program. + var isFillFinished: Bool { stores.allSatisfy(\.isFinished) } + + /// AE#616: cue starts and texts across every injected rendition, on the axis the `.vtt` serves. + func allCueStarts() -> [(start: Double, text: String)] { + stores.flatMap { $0.allCues().map { (start: $0.start, text: $0.text) } } + } + /// Reading the handle stays synchronous: `NSLock` is unavailable from an async context. private func currentFillTask() -> Task? { fillLock.lock() @@ -164,11 +178,14 @@ final class RemoteHLSSubtitleProvider: HLSSegmentProvider, @unchecked Sendable { /// serves a finished master instead. The EXT-X-MEDIA tags were written by `RemoteHLSMasterRewrite`. var nativeSubtitleRenditions: [(ordinal: Int, language: String?, name: String, isForced: Bool)] { [] } - /// Whole-program WebVTT on the origin item's timeline. The host can declare - /// an upstream reanchor offset; unfinished stores must not be served because - /// AVPlayer caches this response for the rest of the session. + /// Whole-program WebVTT for one sidecar. Cue times are the sidecar's own, moved by the track's + /// declared `nativeTimelineOffsetSeconds`: this bypass has no loopback producer and therefore no + /// playlist shift, and AVPlayer places the cues by media timestamp, so the sidecar's absolute + /// seconds line up with the frames. They do NOT line up with item time where an origin's segments + /// start before their playlist slot; AE#616 measures that gap off them. An unfinished store is + /// answered `.pending`, never served: AVPlayer caches this response for the rest of the session. func nativeSubtitleVTT(ordinal: Int, segmentIndex: Int) -> NativeSubtitleVTTResponse { - guard tracks.indices.contains(ordinal), segmentIndex == 0 else { return .missing } + guard ordinal >= 0, ordinal < stores.count, segmentIndex == 0 else { return .missing } let store = stores[ordinal] let deadline = Date().addingTimeInterval(vttFillWaitSeconds) while !store.isFinished, Date() < deadline { diff --git a/Sources/AetherEngine/Native/RemoteHLSSubtitleProxy.swift b/Sources/AetherEngine/Native/RemoteHLSSubtitleProxy.swift index 16e649de4..26610f34b 100644 --- a/Sources/AetherEngine/Native/RemoteHLSSubtitleProxy.swift +++ b/Sources/AetherEngine/Native/RemoteHLSSubtitleProxy.swift @@ -28,6 +28,8 @@ enum RemoteHLSSubtitleProxy { let server: HLSLocalServer let provider: RemoteHLSSubtitleProvider? let masterURL: URL + /// The NAME the served master declares for each injected track, in track order (audit NAT-2). + var renditionNames: [String] = [] /// False when the relay stands alone and nothing was injected. var servesSubtitleRenditions: Bool { provider != nil } @@ -88,7 +90,7 @@ enum RemoteHLSSubtitleProxy { private static func relayOnly(originURL: URL, httpHeaders: [String: String], httpRequestAuthorization: HTTPRequestAuthorization?) -> Prepared? { let relay = HLSOriginRelay(authorization: httpRequestAuthorization) - relay.admit(originURL, httpHeaders: httpHeaders) + relay.grantCredentials(to: originURL, httpHeaders: httpHeaders) let server = HLSLocalServer(relay: relay) do { try server.start() @@ -135,23 +137,32 @@ enum RemoteHLSSubtitleProxy { deadline: Date().addingTimeInterval(budgetSeconds)) } defer { preflightRelay?.stop() } + // Audit NAT-105: the variant is whatever the master names, on any host or scheme, so the + // host's credentials go only where the host sent them. + let credentials = CredentialScope(headers: httpHeaders, anchor: originURL) let (body, finalURL) = try await fetchPlaylist(originURL, session: session, headers: httpHeaders, relay: preflightRelay) let parsed = try parse(body, at: finalURL) let duration = try await programDuration(of: parsed, at: finalURL, - session: session, headers: httpHeaders, relay: preflightRelay) + session: session, credentials: credentials, + relay: preflightRelay) try Task.checkCancellation() - let master = try RemoteHLSMasterRewrite.rewrite( + let rewritten = try RemoteHLSMasterRewrite.rewriteDeclaringNames( originPlaylist: body, originURL: finalURL, renditions: RemoteHLSSubtitleProvider.renditions(for: tracks)) + let master = rewritten.master let provider = RemoteHLSSubtitleProvider(tracks: tracks, masterBody: master, programDuration: duration, defaultHeaders: httpHeaders) let relay: HLSOriginRelay? = needsRelay ? HLSOriginRelay(authorization: httpRequestAuthorization) : nil - relay?.admit(finalURL, httpHeaders: httpHeaders) + // Audit NET-109: the grant belongs to the URL the host handed over. A redirect target is + // only allowed, or a cross-host redirect would move the token to the edge and leave the + // host's own origin without it. + relay?.grantCredentials(to: originURL, httpHeaders: httpHeaders) + relay?.allow(finalURL) let server = HLSLocalServer(provider: provider, relay: relay) do { try server.start() @@ -176,7 +187,8 @@ enum RemoteHLSSubtitleProxy { // Decode up front: the rendition is fetched the moment the host selects it, and a whole-program // .vtt is fetched once and never again. provider.startFill() - return Prepared(server: server, provider: provider, masterURL: masterURL) + return Prepared(server: server, provider: provider, masterURL: masterURL, + renditionNames: rewritten.renditionNames) } // MARK: - Playlist reads @@ -190,21 +202,30 @@ enum RemoteHLSSubtitleProxy { configuration: config, delegate: EngineTLS.sessionDelegate, delegateQueue: nil) } + /// Matches the sibling HLS fetchers (`HLSCarriageProbe`, `HLSVODIngestReader`). This proxy fetches + /// an origin master and one variant before AVPlayer ever opens the source, so an origin that + /// answers with a fast, unbounded stream under the m3u8 URL could otherwise buffer until jetsam + /// (audit NAT-5). + private static let maximumPlaylistBytes = 2 * 1024 * 1024 + /// Returns the body and the URL it finally came from; every relative URI in the playlist resolves /// against the latter, so a redirecting origin (Plex's transcode handoff) still rewrites correctly. private static func fetchPlaylist(_ url: URL, session: URLSession, headers: [String: String], relay: HLSOriginRelay?) async throws -> (String, URL) { - if let relay { return try await relay.fetchPlaylist(url, headers: headers) } + if let relay { + return try await relay.fetchPlaylist(url, headers: headers, maximumBytes: maximumPlaylistBytes) + } var request = URLRequest(url: url) for (field, value) in headers { request.setValue(value, forHTTPHeaderField: field) } let data: Data let response: URLResponse do { - (data, response) = try await session.data(for: request) + (data, response) = try await BoundedPlaylistFetch.data( + for: request, session: session, limit: maximumPlaylistBytes) } catch { - throw Refusal.fetchFailed("\(error.localizedDescription)") + throw Refusal.fetchFailed("\(error)") } if let http = response as? HTTPURLResponse, !(200..<300).contains(http.statusCode) { throw Refusal.fetchFailed("HTTP \(http.statusCode)") @@ -228,23 +249,44 @@ enum RemoteHLSSubtitleProxy { private static func programDuration(of playlist: HLSPlaylist, at url: URL, session: URLSession, - headers: [String: String], + credentials: CredentialScope, relay: HLSOriginRelay?) async throws -> Double { switch playlist { case .media(let media): guard media.hasEndList else { throw Refusal.notVOD } - return media.segments.reduce(0) { $0 + $1.duration } + return try sumSegmentDurations(media.segments) case .master(let master): guard let variant = master.variants.first, let variantURL = HLSPlaylistParser.resolve(uri: variant.uri, against: url) else { throw Refusal.unusablePlaylist("master declares no resolvable variant") } - let (body, finalURL) = try await fetchPlaylist(variantURL, session: session, headers: headers, relay: relay) + let (body, finalURL) = try await fetchPlaylist( + variantURL, session: session, headers: credentials.headers(for: variantURL), relay: relay) guard case .media(let media) = try parse(body, at: finalURL) else { throw Refusal.unusablePlaylist("variant is not a media playlist") } guard media.hasEndList else { throw Refusal.notVOD } - return media.segments.reduce(0) { $0 + $1.duration } + return try sumSegmentDurations(media.segments) + } + } + + /// Longest program this proxy will serve as a whole-program WebVTT rendition. + static let maxProgramDurationSeconds: Double = MediaDurationCeiling.seconds + + /// A hostile or malformed EXTINF (`inf`, negative, or a huge total) reaches `Int(Double)` in + /// `wholeSecondsCovering` downstream and traps (audit NAT-1); refuse it here instead. Internal + /// rather than private so the parsing rule is testable without a network fixture. + static func sumSegmentDurations(_ segments: [HLSMediaSegment]) throws -> Double { + var total = 0.0 + for segment in segments { + guard segment.duration.isFinite, segment.duration >= 0 else { + throw Refusal.unusablePlaylist("EXTINF is not a finite, non-negative duration") + } + total += segment.duration + } + guard total.isFinite, total <= maxProgramDurationSeconds else { + throw Refusal.unusablePlaylist("program duration exceeds \(maxProgramDurationSeconds)s") } + return total } } diff --git a/Sources/AetherEngine/Native/SoftwareEndOfMediaClock.swift b/Sources/AetherEngine/Native/SoftwareEndOfMediaClock.swift index 596ea77cf..7ac1a6f4f 100644 --- a/Sources/AetherEngine/Native/SoftwareEndOfMediaClock.swift +++ b/Sources/AetherEngine/Native/SoftwareEndOfMediaClock.swift @@ -26,4 +26,23 @@ enum SoftwareEndOfMediaClock { guard remaining > 0 else { return 0 } return min(remaining, maxTailPlayoutSeconds) } + + /// Where a late park puts the clock: back on `latest` when the clock has run past it, nil (park + /// where it stands) otherwise or when either value is unusable. + static func parkSeconds(clockSeconds: Double, notAfter latest: Double) -> Double? { + guard clockSeconds.isFinite, latest.isFinite, clockSeconds > latest else { return nil } + return latest + } + + /// How often a deferred park re-reads a clock that is not running (paused during the tail). + static let stoppedClockPollSeconds: TimeInterval = 0.1 + + /// Wall seconds before a deferred park looks again, nil once the clock has reached `target` or + /// either value is unusable. The tail is source time, so it plays out over `remaining / rate` + /// wall seconds; a stopped clock is polled, since parking it where it stands cuts the tail. + static func parkWaitSeconds(clockSeconds: Double, rate: Float, target: Double) -> TimeInterval? { + guard clockSeconds.isFinite, target.isFinite, clockSeconds < target else { return nil } + guard rate > 0 else { return stoppedClockPollSeconds } + return (target - clockSeconds) / Double(rate) + } } diff --git a/Sources/AetherEngine/Native/SoftwarePacketCoverage.swift b/Sources/AetherEngine/Native/SoftwarePacketCoverage.swift index 3c9b23ad2..bd1e2998c 100644 --- a/Sources/AetherEngine/Native/SoftwarePacketCoverage.swift +++ b/Sources/AetherEngine/Native/SoftwarePacketCoverage.swift @@ -20,6 +20,7 @@ struct SoftwarePacketCoverage: Sendable { } var rangeCount: Int { ranges.count } + var isFull: Bool { ranges.count >= maximumRangeCount } /// Adds [pts, pts + duration). Invalid or capacity-exceeding input leaves existing coverage /// unchanged. Rejecting a sparse extension is conservative: it never claims a missing packet. diff --git a/Sources/AetherEngine/Native/SoftwarePacketDiskFIFO.swift b/Sources/AetherEngine/Native/SoftwarePacketDiskFIFO.swift index 26bb7b548..2126cbf78 100644 --- a/Sources/AetherEngine/Native/SoftwarePacketDiskFIFO.swift +++ b/Sources/AetherEngine/Native/SoftwarePacketDiskFIFO.swift @@ -142,10 +142,7 @@ final class SoftwarePacketDiskFIFO: @unchecked Sendable { guard let writer else { throw Failure.corruptRecord } let cursor = Cursor(chunkID: tailID, generation: generation, offset: tailBytes, recordIndex: writtenRecordCount, payloadPrefix: writtenPayloadBytes) - var length = UInt64(data.count).bigEndian - let header = withUnsafeBytes(of: &length) { Data($0) } - try writer.write(contentsOf: header) - try writer.write(contentsOf: data) + try Self.writeRecord(data, to: writer.fileDescriptor) tailBytes += UInt64(recordBytes) recordCount = nextCount payloadBytes = nextPayloadBytes @@ -174,8 +171,7 @@ final class SoftwarePacketDiskFIFO: @unchecked Sendable { } guard let reader else { throw Failure.corruptRecord } guard headOffset <= limit, limit - headOffset >= 8 else { throw Failure.corruptRecord } - let header = try readExactly(8, from: reader) - let length = header.reduce(UInt64(0)) { ($0 << 8) | UInt64($1) } + let length = try Self.readRecordLength(from: reader) guard length <= UInt64(payloadBytes), length <= limit - headOffset - 8 else { throw Failure.corruptRecord } @@ -221,8 +217,7 @@ final class SoftwarePacketDiskFIFO: @unchecked Sendable { let size = try replacement.seekToEnd() guard cursor.offset <= size, size - cursor.offset >= 8 else { throw Failure.corruptRecord } try replacement.seek(toOffset: cursor.offset) - let header = try readExactly(8, from: replacement) - let length = header.reduce(UInt64(0)) { ($0 << 8) | UInt64($1) } + let length = try Self.readRecordLength(from: replacement) guard length <= size - cursor.offset - 8, length <= UInt64(writtenPayloadBytes - cursor.payloadPrefix) else { throw Failure.corruptRecord @@ -242,6 +237,59 @@ final class SoftwarePacketDiskFIFO: @unchecked Sendable { } } + /// AE#605: walks retained records from `cursor` on without touching the consumer's reader, so a + /// scrub still can decode out of history while playback keeps reading its own position. + /// + /// The lock is held only to validate the cursor and to snapshot the tail. The disk reads run + /// outside it on handles of their own, because a GOP is megabytes and the producer and the + /// consumer both serialize on this lock. That is safe for two reasons: an unlinked chunk stays + /// readable through a handle opened before the unlink, and the tail is read only up to the + /// length snapshotted here, never into a record still being written. What it cannot see is a + /// reset recreating the same chunk names underneath it, so the generation is checked again + /// once the walk ends, and a walk that raced a reset throws `invalidCursor` AFTER visiting: the + /// caller discards whatever it collected. `visit` returns false to stop early. Failures never + /// poison the store: a still is optional, playback is not. + func readHistory(from cursor: Cursor, visit: (Data) throws -> Bool) throws { + lock.lock() + do { try requireUsable() } catch { lock.unlock(); throw error } + guard retainConsumed else { lock.unlock(); throw Failure.retentionDisabled } + guard cursor.generation == generation, chunks > 0, + cursor.chunkID >= oldestChunkID, cursor.chunkID <= tailID, + cursor.recordIndex >= 0, cursor.recordIndex < writtenRecordCount else { + lock.unlock() + throw Failure.invalidCursor + } + let lastChunk = tailID + let lastChunkBytes = tailBytes + lock.unlock() + + var chunkID = cursor.chunkID + var offset = cursor.offset + walk: while chunkID <= lastChunk { + guard let handle = try? FileHandle(forReadingFrom: chunkURL(chunkID)) else { + throw Failure.invalidCursor + } + defer { try? handle.close() } + let limit = chunkID == lastChunk ? lastChunkBytes : try handle.seekToEnd() + try handle.seek(toOffset: offset) + while offset < limit { + guard limit - offset >= 8 else { throw Failure.corruptRecord } + let length = try Self.readRecordLength(from: handle) + guard length <= limit - offset - 8 else { throw Failure.corruptRecord } + let record = try readExactly(Int(length), from: handle) + offset += 8 + length + if try !visit(record) { break walk } + } + chunkID += 1 + offset = 0 + } + + lock.lock() + let stillValid = cursor.generation == generation && !isClosed + lock.unlock() + guard stillValid else { throw Failure.invalidCursor } + } + /// Evict only complete history chunks strictly before the reader, oldest first. A budget is /// not permission to discard unread packets or the current reader/writer chunk; the resident /// count can therefore remain above budget until the consumer advances. No packet/chunk index @@ -359,18 +407,73 @@ final class SoftwarePacketDiskFIFO: @unchecked Sendable { chunks += 1 } + /// Audit PERF-109: one allocation and no copy. `read(upToCount:)` allocated a `Data` per call and + /// `append` copied it again into a second one, per payload. private func readExactly(_ length: Int, from handle: FileHandle) throws -> Data { - var result = Data() - result.reserveCapacity(length) - while result.count < length { - guard let part = try handle.read(upToCount: length - result.count), !part.isEmpty else { - throw Failure.corruptRecord - } - result.append(part) + guard length > 0 else { return Data() } + var result = Data(count: length) + try result.withUnsafeMutableBytes { raw in + try Self.readFully(into: raw, from: handle.fileDescriptor) } return result } + private static func readRecordLength(from handle: FileHandle) throws -> UInt64 { + var big: UInt64 = 0 + try withUnsafeMutableBytes(of: &big) { try readFully(into: $0, from: handle.fileDescriptor) } + return UInt64(bigEndian: big) + } + + /// EOF before the buffer fills is a record the file does not hold. + private static func readFully(into buffer: UnsafeMutableRawBufferPointer, from fd: Int32) throws { + guard let base = buffer.baseAddress else { return } + var done = 0 + while done < buffer.count { + let n = Darwin.read(fd, base + done, buffer.count - done) + if n > 0 { done += n; continue } + if n < 0, errno == EINTR { continue } + if n < 0 { throw posixError() } + throw Failure.corruptRecord + } + } + + /// Audit PERF-109: the length and the payload leave in one `writev`, not two writes with a + /// header `Data` allocated between them. A short count (full volume, quota) finishes with plain + /// writes, and any failure throws into the caller's latch, so a half-written record is never + /// appended to. + private static func writeRecord(_ payload: Data, to fd: Int32) throws { + var header = UInt64(payload.count).bigEndian + try withUnsafeBytes(of: &header) { head in + try payload.withUnsafeBytes { body in + let total = head.count + body.count + var sent = 0 + if body.count > 0 { + sent = try withUnsafeTemporaryAllocation(of: iovec.self, capacity: 2) { vec in + vec[0] = iovec(iov_base: UnsafeMutableRawPointer(mutating: head.baseAddress), + iov_len: head.count) + vec[1] = iovec(iov_base: UnsafeMutableRawPointer(mutating: body.baseAddress), + iov_len: body.count) + while true { + let n = Darwin.writev(fd, vec.baseAddress, 2) + if n >= 0 { return n } + if errno != EINTR { throw posixError() } + } + } + } + while sent < total { + let (base, offset) = sent < head.count + ? (head.baseAddress!, sent) : (body.baseAddress!, sent - head.count) + let limit = sent < head.count ? head.count : body.count + let n = Darwin.write(fd, base + offset, limit - offset) + if n > 0 { sent += n; continue } + if n < 0, errno == EINTR { continue } + if n < 0 { throw posixError() } + throw Failure.capacityExceeded + } + } + } + } + private func currentReadLimit() throws -> UInt64 { if reader == nil { reader = try FileHandle(forReadingFrom: chunkURL(headID)) } guard let reader else { throw Failure.corruptRecord } diff --git a/Sources/AetherEngine/Native/SoftwarePacketReadAhead.swift b/Sources/AetherEngine/Native/SoftwarePacketReadAhead.swift index 135c579f2..fa7a6ad8a 100644 --- a/Sources/AetherEngine/Native/SoftwarePacketReadAhead.swift +++ b/Sources/AetherEngine/Native/SoftwarePacketReadAhead.swift @@ -65,8 +65,8 @@ final class SoftwarePacketReadAhead: @unchecked Sendable { /// limit in force on material the coverage model cannot describe. private var storedVideoSeconds: Double? private var consumedVideoSeconds: Double? - private var videoCoverage = SoftwarePacketCoverage() - private var audioCoverage = SoftwarePacketCoverage() + private var videoCoverage: SoftwarePacketCoverage + private var audioCoverage: SoftwarePacketCoverage private var presentationCoverage: SoftwareVideoPacketCoverage? private struct Keyframe { let seconds: Double @@ -79,6 +79,7 @@ final class SoftwarePacketReadAhead: @unchecked Sendable { init(video: Stream, audio: Stream?, byteBudget: Int, forwardSeconds: Double, initialSourceClock: Double, fifo: SoftwarePacketDiskFIFO, videoReorderDepth: Int? = nil, + coverageRangeCap: Int = 4096, beforeConsumerOperation: (@Sendable () -> Void)? = nil, readSource: @escaping @Sendable (@Sendable () -> Bool) throws -> SoftwareStoredPacket?) { self.video = video @@ -87,9 +88,12 @@ final class SoftwarePacketReadAhead: @unchecked Sendable { self.forwardSeconds = max(1, forwardSeconds) self.sourceClock = initialSourceClock self.fifo = fifo + self.videoCoverage = SoftwarePacketCoverage(maximumRangeCount: coverageRangeCap) + self.audioCoverage = SoftwarePacketCoverage(maximumRangeCount: coverageRangeCap) self.presentationCoverage = videoReorderDepth.map { SoftwareVideoPacketCoverage(timeBaseNumerator: video.numerator, - timeBaseDenominator: video.denominator, reorderDepth: $0) + timeBaseDenominator: video.denominator, reorderDepth: $0, + maximumRangeCount: coverageRangeCap) } self.beforeConsumerOperation = beforeConsumerOperation self.readSource = readSource @@ -216,6 +220,54 @@ final class SoftwarePacketReadAhead: @unchecked Sendable { return false } + /// AE#605: the retained video packets a scrub still at `seconds` needs, keyframe first, or nil + /// when the store cannot answer it. Any thread except main: it reads the disk. + /// + /// Eligibility is the cached seek's own rule, coverage past the target plus a retained keyframe + /// at or before it, so a still is offered exactly where a commit to that position would be a + /// cache hit, and the card never shows a picture the seek then has to fetch. Unlike the live + /// ring there is no clamp at the end: a VOD target past the frontier has a real frame the store + /// does not hold yet, and the frame before it is the wrong answer. The consumer cursor is not + /// touched, so playback reads on from where it stood. + func stillRun(atSeconds seconds: Double, maxPackets: Int, maxSpanSeconds: Double, + reorderTail: Int) -> [SoftwareStoredPacket]? { + guard seconds.isFinite, maxPackets > 0 else { return nil } + condition.lock() + let eligible = !closed && !sourceRepositioning && !resetPending && failure == nil + && (frontierLocked(at: seconds).map { $0 > seconds } ?? false) + let anchor = eligible + ? keyframes.filter { $0.seconds <= seconds }.max { $0.seconds < $1.seconds } : nil + condition.unlock() + guard let anchor, seconds - anchor.seconds <= maxSpanSeconds else { return nil } + + var run: [SoftwareStoredPacket] = [] + var reached = false + var tail = reorderTail + var overflow = false + do { + try fifo.readHistory(from: anchor.cursor) { data in + let packet = try SoftwareStoredPacket.decode(data) + guard packet.streamIndex == video.index else { return true } + if run.isEmpty, packet.flags & 1 == 0 { overflow = true; return false } + run.append(packet) + if run.count > maxPackets { overflow = true; return false } + if reached { + tail -= 1 + return tail > 0 + } + if let pts = videoSeconds(pts: packet.pts), pts >= seconds { + reached = true + return tail > 0 + } + return true + } + } catch { + return nil + } + guard reached, !overflow else { return nil } + return run + } + func endSeek(_ token: UInt64, sourceClock: Double) { condition.lock(); defer { condition.unlock() } guard token == generation, !closed else { return } @@ -320,6 +372,21 @@ final class SoftwarePacketReadAhead: @unchecked Sendable { video: videoEnd, audio: audioEnd, requiresAudio: audio != nil) } + /// The playhead in a stream's ticks, rounded down so a prune never reaches past it. + /// + /// #613: coverage keeps its history behind the playhead for backward cached seeks, so a long + /// session fills the range cap. A full coverage used to stop describing new packets (duration + /// model) or invalidate itself (successor model), which left every frontier after the cap nil. + /// Forgetting what lies wholly behind the playhead instead costs a backward cached seek into that + /// history, which then goes cold. + private func playheadTick(_ stream: Stream) -> Int64? { + guard stream.numerator > 0, stream.denominator > 0 else { return nil } + let ticks = (sourceClock * Double(stream.denominator) / Double(stream.numerator)) + .rounded(.down) + guard ticks.isFinite, ticks > -9.0e18, ticks < 9.0e18 else { return nil } + return Int64(ticks) + } + private func clearSourceMetadataLocked() { count = 0; bytes = 0; residentBytes = 0 ended = false; failure = nil @@ -418,8 +485,14 @@ final class SoftwarePacketReadAhead: @unchecked Sendable { copyDiskStateLocked(state) if packet.streamIndex == video.index { if presentationCoverage != nil { + if presentationCoverage?.isFull == true, let tick = playheadTick(video) { + presentationCoverage?.prune(before: tick) + } presentationCoverage?.insert(pts: packet.pts) } else { + if videoCoverage.isFull, let tick = playheadTick(video) { + videoCoverage.prune(before: tick) + } videoCoverage.insert(pts: packet.pts, duration: packet.duration) } if let seconds = videoSeconds(pts: packet.pts) { storedVideoSeconds = seconds } @@ -429,7 +502,10 @@ final class SoftwarePacketReadAhead: @unchecked Sendable { keyframes.removeFirst(min(1024, keyframes.count)) } } - } else if packet.streamIndex == audio?.index { + } else if let audio, packet.streamIndex == audio.index { + if audioCoverage.isFull, let tick = playheadTick(audio) { + audioCoverage.prune(before: tick) + } audioCoverage.insert(pts: packet.pts, duration: packet.duration) } } diff --git a/Sources/AetherEngine/Native/SoftwarePathEscalation.swift b/Sources/AetherEngine/Native/SoftwarePathEscalation.swift index 241e7a2c8..c4c61e744 100644 --- a/Sources/AetherEngine/Native/SoftwarePathEscalation.swift +++ b/Sources/AetherEngine/Native/SoftwarePathEscalation.swift @@ -1,5 +1,32 @@ import Foundation +/// A native session the engine rebuilt on the software path because AVPlayer refused its media +/// (AE#561), published through `AetherEngine.softwarePathEscalations` the moment the rebuild is +/// taken (AE#629). +/// +/// The failure it carries is the one the engine absorbed instead of surfacing, so a host with a +/// fallback ladder of its own can count it, log it, or decline the next one through +/// `LoadOptions.escalatesToSoftwarePath`. If the rebuild then fails, that failure arrives as `.error` +/// the usual way; if it succeeds, `videoRoute` moves to `.software`. +public struct SoftwarePathEscalationEvent: Sendable, Equatable { + /// The failure the rebuild absorbed. `kind` is `.nativeItemFailed`; a failure the engine + /// inferred from a frozen position across its own revives carries no underlying code. + public let absorbedFailure: PlaybackErrorInfo + /// Where the session was when the native path gave up, in seconds: the position the rebuild + /// resumes at, which during a mount is the one the mount was handed rather than AVPlayer's clock. + public let positionSeconds: Double + /// True when a `load()` was still waiting on this session's startup. That `load()` keeps waiting + /// across the rebuild and returns when the rebuilt session is up, instead of throwing the + /// `CancellationError` a load superseded by the host would throw. + public let duringStartup: Bool + + public init(absorbedFailure: PlaybackErrorInfo, positionSeconds: Double, duringStartup: Bool) { + self.absorbedFailure = absorbedFailure + self.positionSeconds = positionSeconds + self.duringStartup = duringStartup + } +} + /// The last rung under a native session AVPlayer will not play: hand the source to the engine's own /// decoder instead of ending the session (AE#561). /// @@ -56,11 +83,28 @@ enum SoftwarePathEscalation { let preferredDecodePath: DecodePath /// The remote-HLS bypass, where the engine decodes nothing at all. let nativeRemoteHLS: Bool + /// `LoadOptions.escalatesToSoftwarePath`: false when the host re-plans a failing title itself. + var hostAllowsEscalation: Bool = true + } + + /// The rebuild a still-waiting `load()` hands its wait to (AE#629). `supersededGeneration` is the + /// load generation the rebuild's teardown ended, so only the load that owned it can follow. + struct Takeover { + let supersededGeneration: UInt64 + let rebuild: Task + /// The claim runs immediately before the rebuild's own `stopInternal`, which moves the generation by one. + var rebuildGeneration: UInt64 { supersededGeneration &+ 1 } } /// The domain of a media failure, i.e. AVFoundation could not make sense of what it was served. static let mediaErrorDomain = "CoreMediaErrorDomain" + /// AE#627: a live join that read video for the whole keyframe wait and found no picture the native + /// route can open a segment on (a feed without IDRs or recovery points, or with gradual refresh + /// only). Like a CoreMedia failure it is a verdict on the media, and libavcodec starts on such a + /// stream by decoding through it. + static let liveJoinErrorDomain = "AetherEngine.LiveJoin" + /// Whether a failed native item is worth handing to the engine's own decoder. /// /// The domain is the discriminator. A CoreMedia failure is a verdict on the MEDIA, which is the @@ -70,10 +114,12 @@ enum SoftwarePathEscalation { /// which is never a reason to swallow a failure. static func shouldEscalate(errorDomain: String?, availability: Availability?) -> Bool { guard let availability, !availability.alreadyEscalated else { return false } + // AE#629: a host with its own fallback ladder needs the failure, not a rescue it cannot see. + guard availability.hostAllowsEscalation else { return false } // Already there, or the host asked for a path this cannot improve on. guard availability.preferredDecodePath == .automatic else { return false } // The bypass has no local muxer and decodes nothing here, so #461 ignores the option anyway. guard !availability.nativeRemoteHLS else { return false } - return errorDomain == mediaErrorDomain + return errorDomain == mediaErrorDomain || errorDomain == liveJoinErrorDomain } } diff --git a/Sources/AetherEngine/Native/SoftwarePlaybackHost.swift b/Sources/AetherEngine/Native/SoftwarePlaybackHost.swift index c45c3986f..2d3ebef18 100644 --- a/Sources/AetherEngine/Native/SoftwarePlaybackHost.swift +++ b/Sources/AetherEngine/Native/SoftwarePlaybackHost.swift @@ -11,6 +11,18 @@ import AetherLibavutil /// Intentionally skips EAC3+JOC and DV HDMI handshake (AV1 sources rarely carry Atmos or DV). @MainActor final class SoftwarePlaybackHost { + /// Codecs whose presentation reordering has an established bound get the read-ahead's + /// successor-PTS coverage; every other codec keeps strict packet-duration coverage. HEVC's bound + /// is FFmpeg's own: `sps_max_num_reorder_pics > HEVC_MAX_DPB_SIZE - 1` (15) is rejected + /// (n8.1.2 hevc/ps.c:1397-1403), inside the 32-timestamp queue. On the duration model a Matroska + /// file muxed with 41 ms durations against 41/42 ms deltas split at every 42 ms step (#613). + nonisolated static func presentationReorderDepth(codecID: UInt32) -> Int? { + switch codecID { + case AV_CODEC_ID_H264.rawValue, AV_CODEC_ID_HEVC.rawValue: return 32 + default: return nil + } + } + // MARK: - Published state (mirrors NativeAVPlayerHost surface) @@ -42,6 +54,10 @@ final class SoftwarePlaybackHost { demuxer?.avioBytesFetched } + /// AE#514: bytes of the played streams by presentation time on the source axis (the axis of + /// `sourceClockSeconds`), fed by the read loops once the timeline fold has been applied. + nonisolated let playedMediaLedger = PlayedMediaLedger() + @Published private(set) var isReady: Bool = false @Published private(set) var currentTime: Double = 0 /// Raw synchronizer clock in the SOURCE axis (same axis as demuxed packet PTS and @@ -81,6 +97,9 @@ final class SoftwarePlaybackHost { /// upgrade the published `videoFormat` from `.hdr10` → `.hdr10Plus`. nonisolated(unsafe) var onFirstHDR10PlusDetected: (@Sendable () -> Void)? + /// AE#658: forwarded from the video decoder, off-main. + nonisolated(unsafe) var onDecodedVideoFormat: (@Sendable (DecodedVideoFormat) -> Void)? + /// #131: forwarded from the video decoder; decoded-frame A53 cc_data triplets, presentation order. nonisolated(unsafe) var onA53Captions: (@Sendable ([CCDataParser.CCTriplet], Double) -> Void)? @@ -93,8 +112,8 @@ final class SoftwarePlaybackHost { var displayLayer: AVSampleBufferDisplayLayer { renderer.displayLayer } /// SW-PiP Phase C: engine-fed cue mirror + PiP gate for the renderer's frame compositor. - func updateSubtitleCompositor(cues: [SubtitleCue], enabled: Bool) { - renderer.subtitleCompositor.update(cues: cues, enabled: enabled) + func updateSubtitleCompositor(cues: [SubtitleCue], enabled: Bool, delaySeconds: Double) { + renderer.subtitleCompositor.update(cues: cues, enabled: enabled, delaySeconds: delaySeconds) } // MARK: - Internals @@ -106,6 +125,8 @@ final class SoftwarePlaybackHost { private var audioOutput: AudioOutput? private var demuxer: Demuxer? private var vodPacketReadAhead: SoftwarePacketReadAhead? + /// #687: this host's entries in the process-wide retention ledger, released on `stop()`. + private var retentionClaims: [RetentionClaims.Claim] = [] /// The same public buffered-position axis as the live and native hosts, but backed by /// actual compressed A/V packet coverage. nil when no continuous cache span contains the clock. @@ -300,6 +321,17 @@ final class SoftwarePlaybackHost { return max(0, newestSourcePts - sessionStartPts) } + /// The DVR ring's rewind floor on the session axis once byte or time eviction has moved it past + /// the start; nil before that and when no ring is armed. Published as the resident floor so the + /// seekable range follows what the ring still holds (audit VPERF-101). + nonisolated var dvrResidentFloorSessionSeconds: Double? { + guard let ring = dvrRing else { return nil } + liveEdgeLock.lock() + let start = sessionStartPts + liveEdgeLock.unlock() + return ring.residentFloorSessionSeconds(sessionStartPts: start) + } + // MARK: - Live reader/feeder split (DVR sessions) // // DVR live sessions: reader (demuxQueue) fills ring regardless of play/pause; feeder (feedQueue) decodes from ring cursor with renderer back-pressure. @@ -644,8 +676,13 @@ final class SoftwarePlaybackHost { /// AE#462: how this host's audio ended up, from the two facts that decide it. Silence has two /// causes here exactly as it does in the loopback cascade, and only one of them is a reason for /// a host to demote to a source that carries the audio differently. - nonisolated static func audioDelivery(resolvedAudioIndex: Int32, decoderOpened: Bool) -> AudioDelivery { - guard resolvedAudioIndex >= 0 else { return .noAudioInSource } + /// + /// AE#641: an unresolved index is not a source without audio. On VOD nothing falls back past + /// `av_find_best_stream`, which passes over an audio stream whose parameters the probe left empty, + /// so the source's own streams decide which of the two silences this is. + nonisolated static func audioDelivery(resolvedAudioIndex: Int32, decoderOpened: Bool, + sourceCarriesAudio: Bool) -> AudioDelivery { + guard resolvedAudioIndex >= 0 else { return sourceCarriesAudio ? .droppedNoPipeline : .noAudioInSource } return decoderOpened ? .decoded : .droppedNoPipeline } @@ -706,6 +743,22 @@ final class SoftwarePlaybackHost { ) == .stop } + /// How long the parked demux loop waits before it re-checks the renderer, in seconds. The renderer + /// drains at frame cadence and the audio lead shrinks with the clock, and nothing signals either, + /// so this is a timed wait (audit PERF-110: the fixed 5 ms sleep was 200 wakeups a second in every + /// software VOD session). It is bounded by the time the read gate needs to open (the lead's excess + /// over its target at the playback rate), never below 5 ms, and never above 20 ms: a renderer queue + /// of about ten frames cannot drain in that, so the parked video still reaches the decoder in time. + nonisolated static func parkedRendererWaitSeconds( + clockArmed: Bool, lastAudioPts: Double, clockSeconds: Double, rate: Float + ) -> TimeInterval { + let floor = 0.005, ceiling = 0.020 + guard clockArmed, lastAudioPts.isFinite, clockSeconds.isFinite, rate > 0 else { return floor } + let untilGateOpens = (lastAudioPts - clockSeconds - AudioLookaheadPolicy.targetLeadSeconds) / Double(rate) + guard untilGateOpens.isFinite else { return floor } + return min(max(untilGateOpens, floor), ceiling) + } + // MARK: - Load func load( @@ -738,8 +791,19 @@ final class SoftwarePlaybackHost { let scratch = baseDir.appendingPathComponent("dvr-\(UUID().uuidString)", isDirectory: true) do { try FileManager.default.createDirectory(at: scratch, withIntermediateDirectories: true) - self.dvrRing = try PacketRingBuffer(windowSeconds: window, scratch: scratch) - EngineLog.emit("[SWHost] DVR ring armed window=\(String(format: "%.0f", window))s scratch=\(scratch.lastPathComponent)", category: .swPlayback) + // Audit VPERF-101: the same allowance the native path gives this session, so a long + // window on a small volume shrinks instead of filling the disk. + let available = (try? baseDir.resourceValues(forKeys: [.volumeAvailableCapacityKey]))? + .volumeAvailableCapacity.map(Int64.init) + let claim = RetentionClaims.shared.claim(volumeAvailableBytes: available) { + PacketRingBuffer.liveByteBudget(volumeAvailableBytes: $0) + } + let budget = claim.bytes + let ring = try PacketRingBuffer(windowSeconds: window, scratch: scratch, byteBudget: budget) + claim.track { [weak ring] in ring?.diskBytes ?? 0 } + self.retentionClaims.append(claim) + self.dvrRing = ring + EngineLog.emit("[SWHost] DVR ring armed window=\(String(format: "%.0f", window))s budget=\(budget >> 20)MiB scratch=\(scratch.lastPathComponent)", category: .swPlayback) } catch { EngineLog.emit("[SWHost] DVR ring create failed (\(error)); live-only fallback", category: .swPlayback) self.dvrRing = nil @@ -801,6 +865,9 @@ final class SoftwarePlaybackHost { ) } + // Audit PERF-104: only the VideoToolbox decoder emits out of presentation order. + renderer.setReorderDepth(SampleBufferRenderer.reorderDepth(forHardwareDecoder: videoDecoder is HardwareVideoDecoder)) + // Flip display layer into HDR mode before frames arrive; without this preferredDynamicRange stays .standard and PQ/HLG renders desaturated. if let codecpar = vStream.pointee.codecpar { let trc = codecpar.pointee.color_trc @@ -827,7 +894,7 @@ final class SoftwarePlaybackHost { if self.bumpFramesEnqueued() == 0 { self.noteFirstFrameEnqueuedForDisplayFallback() if self.takePausedBeforeFirstFrame() { - // The reorder buffer holds four frames before it hands one to the layer, and the + // The reorder buffer holds frames before it hands one to the layer, and the // loops park again from here: without the drain the frame never leaves it. self.renderer.drainReorderBuffer() self.presentFirstFrameUnderPause(pts: pts.seconds, generation: self.decodeGeneration) @@ -846,6 +913,9 @@ final class SoftwarePlaybackHost { videoDecoder.onFirstHDR10PlusDetected = { [weak self] in self?.onFirstHDR10PlusDetected?() } + videoDecoder.onDecodedFormat = { [weak self] format in + self?.onDecodedVideoFormat?(format) + } videoDecoder.onA53Captions = { [weak self] triplets, pts in self?.onA53Captions?(triplets, pts) } @@ -891,7 +961,8 @@ final class SoftwarePlaybackHost { // stream present but unusable (no codecpar) classifies as the drop it is rather than as a // source without audio. self.audioDelivery = Self.audioDelivery(resolvedAudioIndex: resolvedAudioIdx, - decoderOpened: self.audioDecoder != nil) + decoderOpened: self.audioDecoder != nil, + sourceCarriesAudio: dem.firstAudioStreamIndexByType >= 0) // #112 rework: capture embedded subtitle stream indices + time bases for the demux // loop's subtitle tap dispatch. var subIndices: Set = [] @@ -909,12 +980,19 @@ final class SoftwarePlaybackHost { // AudioOutput owns the AVSampleBufferRenderSynchronizer (master clock). Created unconditionally: video-only previously got no clock (frozen frame, currentTime=0). Layer attached in play() after the engine hangs it in the view hierarchy (attaching free-floating fails FigVideoQueueRemote -12080 on tvOS 26+). self.audioOutput = AudioOutput() + self.audioOutput?.volume = volume self.audioOutput?.setPresentationOffset(seconds: audioDelaySeconds) // AE#464 + // AE#395: the route this session plays into, the counterpart of the native host's line. Without + // it a silent software session on a long-latency route and an audible one on HDMI log the same. + if let route = AudioRouteDescription.current() { + EngineLog.emit("[SoftwarePlaybackHost] audioRoute \(route) (session start, live=\(isLive))", + category: .swPlayback) + } // Reset the live feeder state for the new session. resetFeederState() - if let start = startPosition, start > 0 { + if let start = startPosition, start.isFinite, start > 0 { // #254: same off-main, deadline-bounded reposition the transport seek uses. A resume into a // remote source that has to scan for its landing would otherwise block the main thread here. _ = await dem.seekBounded(to: start, timeout: Self.seekBudgetSeconds, on: seekQueue) @@ -945,19 +1023,21 @@ final class SoftwarePlaybackHost { denominator: $0.pointee.time_base.den) } : nil let initialSourceClock = initialClockTime.seconds - let videoReorderDepth: Int? = vCodecID == AV_CODEC_ID_H264.rawValue ? 32 : nil - let cacheResult = await Task.detached(priority: .utility) { () throws -> SoftwarePacketReadAhead? in + let videoReorderDepth = Self.presentationReorderDepth(codecID: vCodecID) + let cacheResult = await Task.detached(priority: .utility) { () throws -> (SoftwarePacketReadAhead, RetentionClaims.Claim)? in let temp = URL(fileURLWithPath: NSTemporaryDirectory(), isDirectory: true) let available = (try? temp.resourceValues(forKeys: [.volumeAvailableCapacityKey]))? .volumeAvailableCapacity.map(Int64.init) let segments = HLSVideoEngine.clampedForwardWindow(forwardBufferSegments) - let bytes = HLSVideoEngine.sessionRetentionBudgetBytes( - volumeAvailableBytes: available, - capRelaxed: HLSVideoEngine.retentionCapRelaxed(forwardWindowSegments: segments)) - guard bytes > 0 else { return Optional.none } + let capRelaxed = HLSVideoEngine.retentionCapRelaxed(forwardWindowSegments: segments) + let claim = RetentionClaims.shared.claim(volumeAvailableBytes: available) { + HLSVideoEngine.sessionRetentionBudgetBytes(volumeAvailableBytes: $0, capRelaxed: capRelaxed) + } + let bytes = claim.bytes + guard bytes > 0 else { return nil } let fifo = try SoftwarePacketDiskFIFO( chunkTargetBytes: min(4 << 20, max(8, bytes)), retainConsumed: true) - return SoftwarePacketReadAhead( + let readAhead = SoftwarePacketReadAhead( video: video, audio: audio, byteBudget: bytes, forwardSeconds: Double(segments) * 4, initialSourceClock: initialSourceClock, fifo: fifo, @@ -967,10 +1047,14 @@ final class SoftwarePlaybackHost { defer { av_packet_unref(packet); av_packet_free_safe(packet) } return try SoftwareStoredPacket(copying: packet) } + claim.track { [weak readAhead] in readAhead?.snapshot.residentBytes ?? 0 } + return (readAhead, claim) }.result let readAhead: SoftwarePacketReadAhead? switch cacheResult { - case .success(let cache): readAhead = cache + case .success(let cache): + readAhead = cache?.0 + if !stopRequested, let claim = cache?.1 { retentionClaims.append(claim) } case .failure: // A cache-directory failure must not stop a source that the old direct loop can // still play. Runtime spool corruption is explicit, never silently skipped. @@ -1151,17 +1235,19 @@ final class SoftwarePlaybackHost { lastAudioPts: demuxDiag.snapshot.lastAudioPts ) guard tail > 0 else { return parkClockNow() } + // The deferral aims at the end of the queued tail; a task that runs late must not park past it. + let target = aOut.currentTimeSeconds + tail let generation = seekGeneration Task { @MainActor [weak self] in try? await Task.sleep(nanoseconds: UInt64(tail * 1_000_000_000)) guard let self, self.admitsRead(generation: generation) else { return } - self.parkClockNow() + self.parkClockNow(notAfter: target) } } - private func parkClockNow() { + private func parkClockNow(notAfter latest: Double = .infinity) { guard !stopRequested, let aOut = audioOutput else { return } - aOut.pause() + aOut.pause(notAfter: latest) rate = 0 EngineLog.emit( "[SWHost] end of media: clock parked at " @@ -1431,6 +1517,40 @@ final class SoftwarePlaybackHost { } } + /// AE#605: true when this VOD session keeps a packet cache a still can be decoded from. A local + /// file has none (it is read directly, see the spool's setup), and neither does a session whose + /// cache could not be built. + var servesPacketCacheStills: Bool { !isLive && vodPacketReadAhead != nil } + + /// AE#605: the VOD twin of `liveScrubStill`, decoded out of the retained packet cache. + /// + /// Same queue, same newest-wins ticket and same extractor as the live still, and the same session + /// axis conversion `seek` uses, so the card and the commit name one moment. The disk read runs on + /// the still queue too, never on the demux or feed loop. + func vodScrubStill(atSessionSeconds seconds: Double, maxWidth: Int) async -> CGImage? { + guard !isLive, let cache = vodPacketReadAhead, let extractor = resolveStillExtractor() else { + return nil + } + let targetSource = sourceSeconds(forSession: seconds) + let limits = SoftwareStillExtractor.Limits.vod + let requests = stillRequests + let ticket = requests.next() + return await withCheckedContinuation { continuation in + stillQueue.async { + guard ticket == requests.latest, + let run = cache.stillRun(atSeconds: targetSource, + maxPackets: limits.maxPackets, + maxSpanSeconds: limits.maxSpanSeconds, + reorderTail: limits.reorderTail) else { + continuation.resume(returning: nil) + return + } + continuation.resume( + returning: extractor.still(from: run, targetPts: targetSource, maxWidth: maxWidth)) + } + } + } + /// Live DVR rewind: reseeds decoder from the ring (source PTS axis; maps via sessionStartPts) without touching the live demuxer. After return, the loop reads new packets forward and plays back to live. private func seekLiveDVR(to targetSession: Double, ring: PacketRingBuffer, wasPlaying: Bool) async { let targetSource = sourceSeconds(forSession: targetSession) @@ -1469,7 +1589,7 @@ final class SoftwarePlaybackHost { /// Reconstruct AVPacket from ring entry, convert PTS from seconds back to stream time_base, and route to decoders. Returns true when audio buffers were enqueued (used for feeder clock arming). @discardableResult - nonisolated private static func feedRingPacket( + nonisolated static func feedRingPacket( _ pkt: PacketRingBuffer.Packet, videoDecoder: any VideoDecodingPipeline, audioDecoder: AudioDecoder?, @@ -1479,6 +1599,7 @@ final class SoftwarePlaybackHost { videoTimeBaseSeconds: Double, audioTimeBaseSeconds: Double, audioTapSink: (@Sendable (CMSampleBuffer) -> Void)?, + audioEpoch: UInt64, noteDecodeGeneration: @Sendable () -> Void ) -> Bool { let tbSec = pkt.isVideo ? videoTimeBaseSeconds : audioTimeBaseSeconds @@ -1494,7 +1615,7 @@ final class SoftwarePlaybackHost { memcpy(dst, base, pkt.bytes.count) } } - p.pointee.pts = Int64((pkt.pts / tbSec).rounded()) + p.pointee.pts = SourceTimestampBounds.roundedTicks(pkt.pts / tbSec) ?? Int64.min p.pointee.dts = p.pointee.pts p.pointee.flags = pkt.isKeyframe ? AV_PKT_FLAG_KEY : 0 p.pointee.stream_index = pkt.isVideo ? videoStreamIndex : audioStreamIndex @@ -1508,8 +1629,11 @@ final class SoftwarePlaybackHost { } else if let aDec = audioDecoder, let aOut = audioOutput { var enqueued = false for buf in aDec.decode(packet: p) { - audioTapSink?(buf) // #95: mirror before enqueue - aOut.enqueue(sampleBuffer: buf) + // DEC-106 on the DVR path: `audioEpoch` was read before the ring read, so a rewind's flush + // retires what this packet decodes to. A pre-seek buffer kept at the head of the fresh queue + // carries a higher stamp than the new clock and mutes audio for the rewind distance. + guard aOut.enqueue(sampleBuffer: buf, ifEpoch: audioEpoch) else { return enqueued } + audioTapSink?(buf) // #95: mirrored behind the accept enqueued = true } return enqueued @@ -1537,6 +1661,8 @@ final class SoftwarePlaybackHost { } dvrRing?.close() dvrRing = nil + retentionClaims.forEach { $0.release() } + retentionClaims.removeAll() liveEdgeLock.lock() sessionStartPts = .nan newestSourcePts = .nan @@ -1560,9 +1686,9 @@ final class SoftwarePlaybackHost { isVideoReadyForDisplay = false } - var volume: Float { - get { audioOutput?.volume ?? 1.0 } - set { audioOutput?.volume = newValue } + /// #660: held here, not only on the output, because the engine sets it before `load()` builds one. + var volume: Float = 1.0 { + didSet { audioOutput?.volume = volume } } // MARK: - Demux loop @@ -1614,6 +1740,7 @@ final class SoftwarePlaybackHost { let getSubtitleTapSink: @Sendable () -> ((@Sendable (Int32, UnsafeMutablePointer, AVRational, Bool) -> Void)?) = { [weak self] in self?.subtitleTapSink } + let playedMediaLedger = playedMediaLedger // AE#560: both read loops hand every source packet to this before any branching. let recordingTap: @Sendable (UnsafeMutablePointer) -> Void = { [weak self] pkt in self?.tapForRecording(pkt) @@ -1719,7 +1846,8 @@ final class SoftwarePlaybackHost { subtitleTimeBases: subTimeBases, splitDisplaySetSubtitleStreamIndices: subSplitSetIndices, subtitleTapSink: getSubtitleTapSink, - recordingTap: recordingTap + recordingTap: recordingTap, + playedMedia: playedMediaLedger ) } let lookahead = audioLookahead @@ -1794,7 +1922,8 @@ final class SoftwarePlaybackHost { subtitleTimeBases: subTimeBases, splitDisplaySetSubtitleStreamIndices: subSplitSetIndices, subtitleTapSink: getSubtitleTapSink, - recordingTap: recordingTap + recordingTap: recordingTap, + playedMedia: playedMediaLedger ) } } @@ -1818,7 +1947,8 @@ final class SoftwarePlaybackHost { subtitleTimeBases: [Int32: AVRational] = [:], splitDisplaySetSubtitleStreamIndices: Set = [], subtitleTapSink: @Sendable () -> ((@Sendable (Int32, UnsafeMutablePointer, AVRational, Bool) -> Void)?) = { nil }, - recordingTap: @escaping @Sendable (UnsafeMutablePointer) -> Void = { _ in } + recordingTap: @escaping @Sendable (UnsafeMutablePointer) -> Void = { _ in }, + playedMedia: PlayedMediaLedger? = nil ) { let discontinuityThresholdSeconds = 10.0 var prevRawVideoPtsSec = Double.nan @@ -1838,6 +1968,9 @@ final class SoftwarePlaybackHost { do { packet = try demuxer.readPacket() } catch { + // Audit SEG-104: a stop closes the demuxer, which aborts a parked read. That is the + // stop arriving, not a lost source. + if stopRequested() { return false } EngineLog.emit("[SWHost] live reader read failed: \(error)", category: .swPlayback) onError("Playback error: \(error.localizedDescription)") onSourceEnded() @@ -1918,9 +2051,11 @@ final class SoftwarePlaybackHost { let tbSec = (streamIdx == videoStreamIndex) ? videoTimeBaseSeconds : audioTimeBaseSeconds if tbSec > 0 { - let offsetTicks = Int64((discontinuityOffsetSec / tbSec).rounded()) - if packet.pointee.pts != Int64.min { packet.pointee.pts -= offsetTicks } - if packet.pointee.dts != Int64.min { packet.pointee.dts -= offsetTicks } + // Audit NAT-101: the offset spans twice the input range and converts per stream + // base, so a packet on the far side of a seam can still leave Int64. + let offsetTicks = SourceTimestampBounds.roundedTicks(discontinuityOffsetSec / tbSec) + packet.pointee.pts = SourceTimestampBounds.shifted(packet.pointee.pts, back: offsetTicks) + packet.pointee.dts = SourceTimestampBounds.shifted(packet.pointee.dts, back: offsetTicks) } } @@ -1932,12 +2067,13 @@ final class SoftwarePlaybackHost { if rawPts != Int64.min, tbSec > 0 { let ptsSec = Double(rawPts) * tbSec noteEdge(ptsSec) + playedMedia?.record(isVideo ? .video : .audio, pts: ptsSec, bytes: Int(packet.pointee.size)) if let data = packet.pointee.data, packet.pointee.size > 0 { - let bytes = Data(bytes: data, count: Int(packet.pointee.size)) let isKey = isVideo && (packet.pointee.flags & AV_PKT_FLAG_KEY) != 0 // Best-effort: a write failure just shrinks the // rewind window, it must not stall the reader. - try? ring.append(pts: ptsSec, isKeyframe: isKey, isVideo: isVideo, bytes: bytes) + try? ring.append(pts: ptsSec, isKeyframe: isKey, isVideo: isVideo, + bytes: UnsafeRawBufferPointer(start: data, count: Int(packet.pointee.size))) condition.lock() condition.broadcast() condition.unlock() @@ -2001,7 +2137,7 @@ final class SoftwarePlaybackHost { var hadLead = false var rebuffering = false var lastLowLeadLog = DispatchTime(uptimeNanoseconds: 0) - func pumpAudio() { + func pumpAudio(epoch audioEpoch: UInt64) { guard let aDec = audioDecoder, let aOut = audioOutput, audioStreamIndex >= 0 else { return } var seq = audioLookahead.align(to: readCursor()) func pumpIteration() -> Bool { @@ -2013,13 +2149,10 @@ final class SoftwarePlaybackHost { clockSeconds: aOut.currentTimeSeconds ) == .feed else { return false } guard seq < ring.seqBounds.end else { return false } // live edge: nothing to pump yet - guard let pkt = ring.packet(atSeq: seq) else { - // Evicted/unreadable under the pump: skip, same as the combined loop. - guard audioLookahead.advance(from: seq, fedPTS: nil) else { return false } - seq += 1 - return true - } - if pkt.isVideo { + // Audit PERF-101: the kind is in the index, so the video packets this pump skips are + // never read off the disk (the feeder reads each of them once, for decode). + guard ring.isVideo(atSeq: seq) == false, let pkt = ring.packet(atSeq: seq) else { + // Video, or evicted/unreadable under the pump: skip, same as the combined loop. guard audioLookahead.advance(from: seq, fedPTS: nil) else { return false } seq += 1 return true @@ -2034,6 +2167,7 @@ final class SoftwarePlaybackHost { videoTimeBaseSeconds: videoTimeBaseSeconds, audioTimeBaseSeconds: audioTimeBaseSeconds, audioTapSink: audioTapSink(), + audioEpoch: audioEpoch, noteDecodeGeneration: noteDecodeGeneration ) if !armed { @@ -2107,6 +2241,10 @@ final class SoftwarePlaybackHost { } func feederIteration() -> Bool { + // Read before anything else: a DVR seek clears `isPlaying`, then flushes the audio output, then + // moves the cursor, so an epoch read ahead of the playing check is retired by any flush that + // lands after the cursor this iteration reads (audit DEC-106). + let audioEpoch = audioOutput?.epoch ?? 0 if !isPlaying() { condition.lock() while !isPlaying() && !stopRequested() { @@ -2119,7 +2257,7 @@ final class SoftwarePlaybackHost { } // Keep the audio renderer topped up before (possibly expensive) video work. - pumpAudio() + pumpAudio(epoch: audioEpoch) let cursor = readCursor() let bounds = ring.seqBounds @@ -2134,6 +2272,12 @@ final class SoftwarePlaybackHost { clampCursor(cursor, bounds.first) return true } + // Audio the pump already delivered: consume the slot without reading it back, the + // pump read it once already (audit PERF-101). + if cursor < audioLookahead.current, ring.isVideo(atSeq: cursor) == false { + advanceCursor(cursor) + return true + } guard let pkt = ring.packet(atSeq: cursor) else { // Resident entry whose file read failed (disk hiccup, // pruned mid-read): skip it, or the feeder would spin on @@ -2172,7 +2316,7 @@ final class SoftwarePlaybackHost { autoreleasepool { Thread.sleep(forTimeInterval: 0.005) waitTicks += 1 - if waitTicks % 20 == 0 { pumpAudio() } + if waitTicks % 20 == 0 { pumpAudio(epoch: audioEpoch) } // #337: the pump is what can still arm the clock from here, so its spent // pre-arm budget is what turns this park terminal (an audio track that // never decodes a buffer). The renderer cannot drain at a stopped clock, @@ -2213,6 +2357,7 @@ final class SoftwarePlaybackHost { videoTimeBaseSeconds: videoTimeBaseSeconds, audioTimeBaseSeconds: audioTimeBaseSeconds, audioTapSink: audioTapSink(), + audioEpoch: audioEpoch, noteDecodeGeneration: noteDecodeGeneration ) @@ -2302,7 +2447,8 @@ final class SoftwarePlaybackHost { subtitleTimeBases: [Int32: AVRational] = [:], splitDisplaySetSubtitleStreamIndices: Set = [], subtitleTapSink: @Sendable () -> ((@Sendable (Int32, UnsafeMutablePointer, AVRational, Bool) -> Void)?) = { nil }, - recordingTap: @escaping @Sendable (UnsafeMutablePointer) -> Void = { _ in } + recordingTap: @escaping @Sendable (UnsafeMutablePointer) -> Void = { _ in }, + playedMedia: PlayedMediaLedger? = nil ) { // Clock arming: one-shot latch (seekClock is not idempotent -- re-calling snaps clock back to initialClockTime). Shared with host so a seek before first audio isn't overridden by a late re-arm. @@ -2348,6 +2494,12 @@ final class SoftwarePlaybackHost { var everHadLead = false var parkedSeekGeneration = seekGeneration() let decoupleAudio = !isLive && audioDecoder != nil && audioOutput != nil + // AE#395: the diagnostic line's audio marker, written on every route. `lastEnqueuedAudioPtsSec` + // is the pacing input and only exists where audio is decoupled, so a live session's line read + // `aLead=-` throughout, on exactly the sessions where the lead decides whether a long-latency + // route plays anything. The generation is the one the buffer was produced under (AE#479). + var diagAudioPtsSec = Double.nan + var diagAudioPtsGeneration = parkedSeekGeneration func freeParkedVideo() { for p in parkedVideo { @@ -2454,10 +2606,21 @@ final class SoftwarePlaybackHost { releaseRebufferHold("the renderer needs a running clock to take parked video") armFromParkedVideoIfStuck() drainParkedVideoNonblocking() - diag?.update(lastAudioPts: lastEnqueuedAudioPtsSec, + diag?.update(lastAudioPts: diagAudioPtsSec, parked: parkedVideo.count, rebuffering: rebuffering, - generation: parkedSeekGeneration) - if stillWaiting() { Thread.sleep(forTimeInterval: 0.005) } + generation: diagAudioPtsGeneration) + if stillWaiting() { + // The condition is broadcast on play, stop, background, seek-settled and feed-cursor + // changes, so those cut the wait short where the sleep used to ride them out. + let wait = Self.parkedRendererWaitSeconds( + clockArmed: clockArmed(), lastAudioPts: lastEnqueuedAudioPtsSec, + clockSeconds: audioOutput?.currentTimeSeconds ?? .nan, rate: currentRate()) + condition.lock() + autoreleasepool { + _ = condition.wait(until: Date(timeIntervalSinceNow: wait)) + } + condition.unlock() + } } } @@ -2555,6 +2718,8 @@ final class SoftwarePlaybackHost { parkedSeekGeneration = gen freeParkedVideo() lastEnqueuedAudioPtsSec = .nan + diagAudioPtsSec = .nan + diagAudioPtsGeneration = gen rebuffering = false // The lead is zero again after a seek, so the latch has to earn itself back: // keeping it set pauses the clock for a rebuffer on the first post-seek check. @@ -2580,6 +2745,8 @@ final class SoftwarePlaybackHost { // inside the decoder rather than leaving the caller to re-check a value it cannot hold // across the call. var epochBeforeRead = videoDecoder.feedEpoch + // Audit DEC-106: the audio side of the same rule, compared under the output's lock by `enqueue`. + let audioEpochBeforeRead = audioOutput?.epoch ?? 0 let packet: UnsafeMutablePointer? do { if let readAhead { @@ -2679,9 +2846,21 @@ final class SoftwarePlaybackHost { let tbSec = (streamIdx == videoStreamIndex) ? videoTimeBaseSeconds : audioTimeBaseSeconds if tbSec > 0 { - let offsetTicks = Int64((discontinuityOffsetSec / tbSec).rounded()) - if packet.pointee.pts != Int64.min { packet.pointee.pts -= offsetTicks } - if packet.pointee.dts != Int64.min { packet.pointee.dts -= offsetTicks } + // Audit NAT-101: the offset spans twice the input range and converts per stream + // base, so a packet on the far side of a seam can still leave Int64. + let offsetTicks = SourceTimestampBounds.roundedTicks(discontinuityOffsetSec / tbSec) + packet.pointee.pts = SourceTimestampBounds.shifted(packet.pointee.pts, back: offsetTicks) + packet.pointee.dts = SourceTimestampBounds.shifted(packet.pointee.dts, back: offsetTicks) + } + } + + // AE#514: on the folded axis the clock runs on, which is what the sampler reads the playhead off. + if let playedMedia, streamIdx == videoStreamIndex || streamIdx == audioStreamIndex { + let tbSec = streamIdx == videoStreamIndex ? videoTimeBaseSeconds : audioTimeBaseSeconds + let ticks = packet.pointee.pts != Int64.min ? packet.pointee.pts : packet.pointee.dts + if ticks != Int64.min, tbSec > 0 { + playedMedia.record(streamIdx == videoStreamIndex ? .video : .audio, + pts: Double(ticks) * tbSec, bytes: Int(packet.pointee.size)) } } @@ -2698,13 +2877,13 @@ final class SoftwarePlaybackHost { if let ring { let isKey = isVideo && (packet.pointee.flags & AV_PKT_FLAG_KEY) != 0 if let data = packet.pointee.data, packet.pointee.size > 0 { - let bytes = Data(bytes: data, count: Int(packet.pointee.size)) // Append is a small file write; off-main and // internally locked, so it never touches the // decoders' state. Best-effort: a write failure // just shrinks the rewind window, it must not // stall live playback. - try? ring.append(pts: ptsSec, isKeyframe: isKey, isVideo: isVideo, bytes: bytes) + try? ring.append(pts: ptsSec, isKeyframe: isKey, isVideo: isVideo, + bytes: UnsafeRawBufferPointer(start: data, count: Int(packet.pointee.size))) } } } @@ -2836,12 +3015,20 @@ final class SoftwarePlaybackHost { return true } for buf in buffers { - tapSink?(buf) // #95: mirror before enqueue - aOut.enqueue(sampleBuffer: buf) + guard aOut.enqueue(sampleBuffer: buf, ifEpoch: audioEpochBeforeRead) else { + av_packet_unref(packet) + av_packet_free_safe(packet) + return true + } + tapSink?(buf) // #95: mirrored behind the accept, so a refused buffer is not transcribed } - if decoupleAudio, let last = buffers.last { + if let last = buffers.last { let pts = CMSampleBufferGetPresentationTimeStamp(last) - if pts.isValid { lastEnqueuedAudioPtsSec = pts.seconds } + if pts.isValid { + if decoupleAudio { lastEnqueuedAudioPtsSec = pts.seconds } + diagAudioPtsSec = pts.seconds + diagAudioPtsGeneration = genBeforeRead + } } // Arm clock on first decoded audio buffer; latch so subsequent packets don't snap clock back. if !clockArmed(), !buffers.isEmpty { @@ -2873,10 +3060,10 @@ final class SoftwarePlaybackHost { let keepGoing: Bool = autoreleasepool { demuxIteration() } - diag?.update(lastAudioPts: lastEnqueuedAudioPtsSec, + diag?.update(lastAudioPts: diagAudioPtsSec, parked: parkedVideo.count, rebuffering: rebuffering, - generation: parkedSeekGeneration) + generation: diagAudioPtsGeneration) if !keepGoing { break } } freeParkedVideo() @@ -2943,6 +3130,15 @@ final class SoftwarePlaybackHost { /// the line stops rather than repeating itself at 1 Hz for as long as the host holds the engine. private var didEmitParkedDiag = false + /// AE#395: a DVR session feeds audio from the ring pump, which never writes `demuxDiag`, so its + /// line read `aLead=-` throughout. The pump's own fed PTS is the marker it paces on, and a DVR + /// seek clears it, so it carries no pre-seek value. + nonisolated static func diagAudioMarker( + demuxLoopPts: Double, dvrPumpPts: Double, isDVRSession: Bool + ) -> Double { + isDVRSession ? dvrPumpPts : demuxLoopPts + } + /// 1 Hz [SWDiag] line: clock + clock delta, decoded-audio and video lead over the clock, parked /// video PACKET FIFO depth, rebuffer state, frames produced vs frames handed to the layer with /// the spacing of the timestamps they carried, and the display layer's OWN drop counter with its @@ -2972,7 +3168,11 @@ final class SoftwarePlaybackHost { if prevClock.isFinite, clock == prevClock { didEmitParkedDiag = true } } diagPrevClock = clock - let lead = d.lastAudioPts.isFinite && clock.isFinite ? d.lastAudioPts - clock : Double.nan + let audioMarker = Self.diagAudioMarker( + demuxLoopPts: d.lastAudioPts, + dvrPumpPts: audioLookahead.lastFedAudioPTS, + isDVRSession: isLive && dvrRing != nil) + let lead = audioMarker.isFinite && clock.isFinite ? audioMarker - clock : Double.nan let enqueued = framesEnqueued let dEnq = enqueued - diagPrevEnqueued diagPrevEnqueued = enqueued @@ -3023,6 +3223,9 @@ final class SoftwarePlaybackHost { // deactivated audio session are the same "dclk=0.00" and different defects. + "rate=\(String(format: "%.2f", self.audioOutput?.rate ?? 0)) " + "aLead=\(lead.isFinite ? String(format: "%.2f", lead) : "-") " + // AE#395: status/sufficient/error of the audio renderer, the one stage after the feed + // that nothing on this line described. + + "aRend=\(self.audioOutput?.diagRendererState ?? "-") " + "vLead=\(videoLead.map { String(format: "%.2f", $0) } ?? "-") " + "parkedPkts=\(d.parked) rebuf=\(d.rebuffering ? "y" : "n") " + (d.sourceExhausted ? "eof=y " : "") diff --git a/Sources/AetherEngine/Native/SoftwareStillExtractor.swift b/Sources/AetherEngine/Native/SoftwareStillExtractor.swift index 2234cc82d..ab9d28d21 100644 --- a/Sources/AetherEngine/Native/SoftwareStillExtractor.swift +++ b/Sources/AetherEngine/Native/SoftwareStillExtractor.swift @@ -28,6 +28,10 @@ final class SoftwareStillExtractor: @unchecked Sendable { /// Packets are stored in decode order, so the frame at the target can sit behind the first /// packet that reaches it. Two B-frames is the common broadcast shape; four covers the rest. var reorderTail: Int = 4 + + /// AE#605: a file's GOP is not a broadcast one. x264's default keyint is 250 pictures, + /// ten seconds at 25 fps and four at 60, and B-pyramids reorder deeper than two B-frames. + static let vod = Limits(maxPackets: 900, maxSpanSeconds: 12, reorderTail: 16) } private let decoder = SoftwareVideoDecoder() @@ -72,6 +76,30 @@ final class SoftwareStillExtractor: @unchecked Sendable { reorderTail: limits.reorderTail), !run.isEmpty else { return nil } + return decodeRun(targetPts: targetPts, maxWidth: maxWidth) { + for packet in run { feed(packet) } + } + } + + /// AE#605: the same still out of a software VOD session's retained packets. They arrive as the + /// demuxer produced them, envelope and all, so they are replayed as is: a VOD stream carries real + /// decode timestamps and side data that the ring's pts-only shape has no room for. + func still(from run: [SoftwareStoredPacket], targetPts: Double, maxWidth: Int) -> CGImage? { + guard isOpen, maxWidth > 0, let first = run.first, first.flags & AV_PKT_FLAG_KEY != 0 else { + return nil + } + return decodeRun(targetPts: targetPts, maxWidth: maxWidth) { + for stored in run { + guard let p = try? stored.makeAVPacket() else { continue } + var packet: UnsafeMutablePointer? = p + defer { trackedPacketFree(&packet) } + p.pointee.stream_index = videoStreamIndex + decoder.decode(packet: p, epoch: nil) + } + } + } + + private func decodeRun(targetPts: Double, maxWidth: Int, feedRun: () -> Void) -> CGImage? { let collector = FrameCollector(target: targetPts) decoder.onFrame = { pixelBuffer, pts, _ in collector.append(pixelBuffer: pixelBuffer, seconds: pts.seconds) @@ -79,9 +107,7 @@ final class SoftwareStillExtractor: @unchecked Sendable { decoder.flush(resetFilterGraph: false) defer { decoder.onFrame = nil } - for packet in run { - feed(packet) - } + feedRun() guard let best = collector.best else { return nil } return Self.image(from: best, maxWidth: maxWidth) @@ -102,7 +128,7 @@ final class SoftwareStillExtractor: @unchecked Sendable { memcpy(dst, base, packet.bytes.count) } } - p.pointee.pts = Int64((packet.pts / timeBaseSeconds).rounded()) + p.pointee.pts = SourceTimestampBounds.roundedTicks(packet.pts / timeBaseSeconds) ?? Int64.min p.pointee.dts = p.pointee.pts p.pointee.flags = packet.isKeyframe ? AV_PKT_FLAG_KEY : 0 p.pointee.stream_index = videoStreamIndex diff --git a/Sources/AetherEngine/Native/SoftwareStoredPacket.swift b/Sources/AetherEngine/Native/SoftwareStoredPacket.swift index 0a8c57a7d..3847285f8 100644 --- a/Sources/AetherEngine/Native/SoftwareStoredPacket.swift +++ b/Sources/AetherEngine/Native/SoftwareStoredPacket.swift @@ -2,8 +2,8 @@ import Foundation /// Lossless packet envelope for the software VOD read-ahead store. No decoded pixels or /// source URL is retained. In particular DTS, duration and side data must survive a cache hit. -struct SoftwareStoredPacket: Codable, Sendable, Equatable { - struct SideData: Codable, Sendable, Equatable { +struct SoftwareStoredPacket: Sendable, Equatable { + struct SideData: Sendable, Equatable { let type: UInt32 let bytes: Data } @@ -18,13 +18,103 @@ struct SoftwareStoredPacket: Codable, Sendable, Equatable { let bytes: Data let sideData: [SideData] + enum EnvelopeError: Error { + case unknownVersion(UInt8) + case truncated + case trailingBytes(Int) + } + + // AE#592: a fixed little-endian header followed by the raw bytes. The binary property list + // this replaced uniqued every object through a Set, hashing the whole payload of every packet, + // and that was the largest single cost of the software route's producer. The spool lives only + // for its session, so the layout carries a version byte and no compatibility promise. + // + // u8 version | i64 pts, dts, duration, position | i32 streamIndex, flags, tbNum, tbDen + // | u32 sideDataCount | u64 payloadCount | payload | per side data: u32 type, u64 count, bytes + static let version: UInt8 = 1 + static let headerBytes = 1 + 4 * 8 + 4 * 4 + 4 + 8 + static let sideDataHeaderBytes = 4 + 8 + func encoded() throws -> Data { - let encoder = PropertyListEncoder() - encoder.outputFormat = .binary - return try encoder.encode(self) + let total = sideData.reduce(Self.headerBytes + bytes.count) { + $0 + Self.sideDataHeaderBytes + $1.bytes.count + } + var out = Data(capacity: total) + out.append(Self.version) + for value in [pts, dts, duration, position] { Self.append(value, to: &out) } + for value in [streamIndex, flags, timeBaseNumerator, timeBaseDenominator] { Self.append(value, to: &out) } + Self.append(UInt32(sideData.count), to: &out) + Self.append(UInt64(bytes.count), to: &out) + out.append(bytes) + for entry in sideData { + Self.append(entry.type, to: &out) + Self.append(UInt64(entry.bytes.count), to: &out) + out.append(entry.bytes) + } + return out } static func decode(_ data: Data) throws -> Self { - try PropertyListDecoder().decode(Self.self, from: data) + var reader = Reader(data: data) + let version: UInt8 = try reader.read() + guard version == Self.version else { throw EnvelopeError.unknownVersion(version) } + let pts: Int64 = try reader.read() + let dts: Int64 = try reader.read() + let duration: Int64 = try reader.read() + let position: Int64 = try reader.read() + let streamIndex: Int32 = try reader.read() + let flags: Int32 = try reader.read() + let timeBaseNumerator: Int32 = try reader.read() + let timeBaseDenominator: Int32 = try reader.read() + let sideDataCount: UInt32 = try reader.read() + let payload = try reader.bytes(count: try reader.read() as UInt64) + var sideData: [SideData] = [] + sideData.reserveCapacity(Int(min(sideDataCount, 64))) + for _ in 0..(_ value: T, to out: inout Data) { + withUnsafeBytes(of: value.littleEndian) { out.append(contentsOf: $0) } + } + + private struct Reader { + let data: Data + var offset: Int + + init(data: Data) { + self.data = data + self.offset = data.startIndex + } + + var remaining: Int { data.endIndex - offset } + + mutating func read() throws -> T { + let size = MemoryLayout.size + guard remaining >= size else { throw EnvelopeError.truncated } + var value: T = 0 + withUnsafeMutableBytes(of: &value) { raw in + data.copyBytes(to: raw.bindMemory(to: UInt8.self), from: offset.. Data { + guard count <= UInt64(remaining) else { throw EnvelopeError.truncated } + let end = offset + Int(count) + // Audit PERF-109: a view of the record, not a copy; the caller copies once into its AVPacket. + let slice = data[offset.. 16; the default holds 32 distinct timestamps as a conservative -/// field-picture margin. That codec picture bound does NOT prove arbitrary container PTS obey the -/// bound. A timestamp behind the emitted watermark therefore invalidates all coverage until reset. -/// Source: https://github.com/FFmpeg/FFmpeg/blob/n8.1.2/libavcodec/h264_ps.c#L174-L180 +/// rejects num_reorder_frames > 16 and hevc/ps.c rejects sps_max_num_reorder_pics > 15; the default +/// holds 32 distinct timestamps as a conservative field-picture margin. That codec picture bound +/// does NOT prove arbitrary container PTS obey the bound. A timestamp behind the emitted watermark +/// therefore invalidates all coverage until reset. +/// Sources: https://github.com/FFmpeg/FFmpeg/blob/n8.1.2/libavcodec/h264_ps.c#L174-L180 +/// https://github.com/FFmpeg/FFmpeg/blob/n8.1.2/libavcodec/hevc/ps.c#L1397-L1403 /// /// This metadata-only value neither owns packets nor synchronizes access. The owner must reset it /// together with its packet store on seek/flush/discard. Other codecs must keep their strict packet /// duration coverage unless their presentation reordering has a separately established bound. struct SoftwareVideoPacketCoverage: Sendable { - private var coverage = SoftwarePacketCoverage() + private var coverage: SoftwarePacketCoverage private var pending: [Int64] = [] private var emittedPTS: Int64? private let timeBaseNumerator: Int32 @@ -30,7 +33,9 @@ struct SoftwareVideoPacketCoverage: Sendable { private(set) var lateTimestampCount = 0 private(set) var isFinished = false - init(timeBaseNumerator: Int32, timeBaseDenominator: Int32, reorderDepth: Int = 32) { + init(timeBaseNumerator: Int32, timeBaseDenominator: Int32, reorderDepth: Int = 32, + maximumRangeCount: Int = 4096) { + coverage = SoftwarePacketCoverage(maximumRangeCount: maximumRangeCount) self.timeBaseNumerator = timeBaseNumerator self.timeBaseDenominator = timeBaseDenominator self.reorderDepth = min(32, max(1, reorderDepth)) @@ -41,6 +46,7 @@ struct SoftwareVideoPacketCoverage: Sendable { var pendingCount: Int { pending.count } var rangeCount: Int { coverage.rangeCount } + var isFull: Bool { coverage.isFull } /// Duration/DTS are deliberately not inputs. Duplicate pending PTS do not advance the reorder /// watermark. At most depth + 1 timestamps are held transiently, then the minimum is emitted. diff --git a/Sources/AetherEngine/Network/EngineTLS.swift b/Sources/AetherEngine/Network/EngineTLS.swift index 2bc6d017d..09240119a 100644 --- a/Sources/AetherEngine/Network/EngineTLS.swift +++ b/Sources/AetherEngine/Network/EngineTLS.swift @@ -37,6 +37,10 @@ public enum EngineTLS { /// for tasks that missed a per-task delegate. static let sessionDelegate = SessionTrustDelegate() + /// The redirect rule without the trust answer, for a session that must see + /// system trust alone (`HLSOriginRelay.systemTrustRefuses`). + static let redirectDelegate = RedirectScrubbingDelegate() + /// Single disposition shared by the session-level delegate and the /// per-task delegates in AVIOReader. Anything other than a server-trust /// challenge the host accepted is left to default handling, so client @@ -60,7 +64,31 @@ public enum EngineTLS { completionHandler(.useCredential, URLCredential(trust: trust)) } - final class SessionTrustDelegate: NSObject, URLSessionDelegate, @unchecked Sendable { + /// Audit NET-108: URLSession's own redirect copies every custom header + /// except `Authorization` to the new request (measured on CFNetwork 3896: + /// `X-Emby-Token`, `Cookie`, `Range` and `Referer` all reached a cross-host + /// target), so without this a 302 took the host's token wherever it + /// pointed. A task delegate that implements its own redirect method + /// (AVIOReader, SourcePrewarmFetcher) still answers for its task; one that + /// does not (the relay's pump) falls back to this. + class RedirectScrubbingDelegate: NSObject, URLSessionTaskDelegate, @unchecked Sendable { + func urlSession( + _ session: URLSession, + task: URLSessionTask, + willPerformHTTPRedirection response: HTTPURLResponse, + newRequest request: URLRequest, + completionHandler: @escaping (URLRequest?) -> Void + ) { + let original = task.originalRequest + completionHandler(RedirectHeaderPolicy.redirectRequest( + request, + originalURL: original?.url, + originalRange: original?.value(forHTTPHeaderField: "Range"), + extraHeaders: [:])) + } + } + + final class SessionTrustDelegate: RedirectScrubbingDelegate, @unchecked Sendable { func urlSession( _ session: URLSession, didReceive challenge: URLAuthenticationChallenge, diff --git a/Sources/AetherEngine/Network/HLSLocalServer.swift b/Sources/AetherEngine/Network/HLSLocalServer.swift index 3edb2331a..cf55ae87a 100644 --- a/Sources/AetherEngine/Network/HLSLocalServer.swift +++ b/Sources/AetherEngine/Network/HLSLocalServer.swift @@ -101,6 +101,12 @@ protocol HLSSegmentProvider: AnyObject { /// has the cues for a segment in the store by the time AVPlayer fetches it. func nativeSubtitleVTT(ordinal: Int, segmentIndex: Int) -> NativeSubtitleVTTResponse + /// AE#682: true when this session lists an I-frame rendition in its master. The two calls below + /// answer `iframe_init.mp4` and `iframe{N}.mp4`; both may block on a source read. + var iFrameRenditionServed: Bool { get } + func iFrameInitSegment() -> Data? + func iFrameSegment(at index: Int) -> Data? + /// Atomic snapshot at the top of each playlist build. discontinuitySequence = EXT-X-DISCONTINUITY-tagged segments that slid out of the window (RFC 8216 §6.2.2 requires incrementing it; omission slips AVPlayer's discontinuity tracking one window per boundary). firstVisible in the same snapshot: a separate lock acquisition let a concurrent slide produce MEDIA-SEQUENCE newer than the count. func notePlaylistBuild() -> (visibleCount: Int, firstVisible: Int, refreshCounter: Int, endlistAdded: Bool, discontinuitySequence: Int) @@ -165,6 +171,9 @@ extension HLSSegmentProvider { var nativeSubtitleDefaultOrdinal: Int { 0 } var nativeSubtitleWholeProgram: Bool { false } func nativeSubtitleVTT(ordinal: Int, segmentIndex: Int) -> NativeSubtitleVTTResponse { .missing } + var iFrameRenditionServed: Bool { false } + func iFrameInitSegment() -> Data? { nil } + func iFrameSegment(at index: Int) -> Data? { nil } var liveTargetSegmentDuration: Double? { nil } var liveRejoinStart: (segmentIndex: Int, secondsIntoSegment: Double)? { nil } func noteServedLiveRejoinPlacement(timeOffset: Double, firstVisible: Int) {} @@ -248,6 +257,59 @@ final class HLSLocalServer: @unchecked Sendable { return rest } + /// The request line as it is logged: the route after the session token, never the token itself + /// (audit SUB-107). + static func requestLineForLog(method: Substring, routePath: String, query: String, + version: Substring?) -> String { + var line = "\(method) \(routePath)" + if !query.isEmpty { line += "?\(query)" } + if let version { line += " \(version)" } + return escapedForLog(line) + } + + /// The headers whose values the first-request dump shows: the capability headers it exists for + /// (#50), and the address the client used (#86). Nothing that carries a credential. + static let loggedRequestHeaderValues: Set = [ + "accept", "host", "range", "user-agent", "x-playback-session-id", + ] + + /// Audit Vcred-102: on the #316 / AE#495 stand-in route AVPlayer sends the host's own headers to + /// this server too, credentials included (measured), so the dump names every header and prints a + /// value only where `loggedRequestHeaderValues` says it answers a question. + static func requestHeadersForLog(_ headerLines: [String]) -> String { + headerLines.map { line -> String in + guard let colon = line.firstIndex(of: ":") else { return "?" } + let name = line[.. String { + var out = "" + var count = 0 + for scalar in text.unicodeScalars { + if count >= limit { + out += "..." + break + } + if scalar.value < 0x20 || scalar.value == 0x7F { + out += String(format: "\\x%02X", scalar.value) + } else { + out.unicodeScalars.append(scalar) + } + count += 1 + } + return out + } + /// Kernel-assigned ephemeral port. Zero until start() succeeds. private(set) var port: UInt16 = 0 @@ -337,6 +399,12 @@ final class HLSLocalServer: @unchecked Sendable { private var shouldStop = false private var clientFds = Set() + /// Audit SUB-107: whether `pathToken` is registered with the log redactor. The token is the + /// capability that keeps LAN strangers off the stream, and every line that prints a server URL + /// (`load url=`, `asset.url=`, the #316 serving line) would otherwise hand it to a shared log + /// for as long as the session lives. + private var tokenRegistered = false + /// Active connection count; engine memory probe watches for unexpectedly rising accumulation (AVPlayer normally holds 1-3 connections). var activeConnectionCount: Int { stateLock.lock() @@ -411,9 +479,36 @@ final class HLSLocalServer: @unchecked Sendable { /// that lasts: the client then reads a dead server. Measured with 192 pool workers blocked, a /// `DispatchQueue.global().async` block had not started after 35 s while a detached thread ran /// in 3 ms. Same reason the pump owns its thread since AE#286. - private static let maxConcurrentConnections = 32 + static let maxConcurrentConnections = 32 private var liveConnectionThreads = 0 + /// Audit NET-6: the listener has to answer the LAN while an AirPlay receiver fetches from it + /// (#86), and the session token is only read once a whole request head has arrived. So a peer + /// that is not loopback may hold at most this many of the slots, and the rest stay free for + /// the local player however many connections a LAN host opens. + static let maxNonLoopbackConnections = 24 + private var liveNonLoopbackConnections = 0 + + /// Audit NET-6: a connection that has not yet presented this session's token has this long, + /// from accept, to deliver a whole request head. Without it a trickling or idle LAN peer holds + /// its slot for as long as it keeps sending a byte inside the per-recv timeout. + static let unauthenticatedHeadSeconds: TimeInterval = 10 + /// Once a request's first byte has arrived, the rest of its head must follow within this long, + /// on a connection that already authenticated too. The idle wait before that byte stays at + /// `keepAliveIdleSeconds`, which AVPlayer's keep-alive gaps need. + static let requestHeadSeconds: TimeInterval = 10 + static let keepAliveIdleSeconds: TimeInterval = 60 + + /// Audit NET-13: one line per failed accept, when the process is out of descriptors, is a busy + /// loop that floods the host's log ring. Failures back off and their line is rate limited. + private static let acceptFailureBackoffMicroseconds: useconds_t = 100_000 + private var acceptFailureLog = LogThrottle(interval: 5) + private var refusalLog = LogThrottle(interval: 5) + /// Audit NET-111: every line a connection that never presented the token can cause shares this, so + /// a LAN peer looping short connections costs the host's log one line per interval, not two per + /// request. + private var strangerLog = LogThrottle(interval: 5) + // MARK: - Init /// When set (e.g. `aether-engine://engine/`), segment URIs in the playlist are absolute custom-scheme URLs routed through AVAssetResourceLoader. Nil emits relative URIs for the aetherctl HTTP workflow. @@ -427,22 +522,27 @@ final class HLSLocalServer: @unchecked Sendable { /// A relay-only server has no provider: nothing here produces segments, every byte comes /// from the origin, and the master is whatever the origin served. init(provider: HLSSegmentProvider? = nil, subResourceBaseURL: URL? = nil, - relay: HLSOriginRelay? = nil) { + relay: HLSOriginRelay? = nil, + unauthenticatedHeadSeconds: TimeInterval = HLSLocalServer.unauthenticatedHeadSeconds) { self.provider = provider self.subResourceBaseURL = subResourceBaseURL self.relay = relay + self.unauthenticatedHeadDeadline = unauthenticatedHeadSeconds } - /// Admits `origin` to the relay and returns the address standing in for it, for a player - /// pointed at the relay rather than at a provider's playlists. Nil before `start()` or with - /// no relay mounted. + private let unauthenticatedHeadDeadline: TimeInterval + + /// Allows `origin` on the relay and returns the address standing in for it, for a player + /// pointed at the relay rather than at a provider's playlists. Grants no credentials: that is + /// the caller's decision, for a URL the host handed over (audit NET-109). Nil before `start()` + /// or with no relay mounted. func relayURL(for origin: URL) -> URL? { - guard let relay, relay.admit(origin) != nil else { return nil } + guard let relay, relay.allow(origin) != nil else { return nil } stateLock.lock() let listeningPort = port stateLock.unlock() guard listeningPort > 0 else { return nil } - return HLSOriginRelay.localURL(for: origin, port: listeningPort, token: pathToken) + return relay.localURL(for: origin, port: listeningPort, token: pathToken) } // MARK: - Lifecycle @@ -472,6 +572,10 @@ final class HLSLocalServer: @unchecked Sendable { // Bind all interfaces (not just loopback) so an AirPlay receiver can reach the stream over the LAN // via the device's WiFi IP (#86, DrHurt). Local playback still uses 127.0.0.1; the URL host is only // swapped to the LAN IP while external playback is active. Ephemeral port, serves the current stream only. + // Audit NET-6 kept this rather than binding loopback until AirPlay engages: the wireless edge reloads + // onto a new server, but a readiness-gate swap (`airPlayHostSwapped`) and a held edge re-read after a + // reload hand the LAN URL to the server already listening. LAN exposure is bounded instead by the + // head deadlines and the loopback reserve in the accept loop. addr.sin_addr.s_addr = inet_addr("0.0.0.0") let bindResult = withUnsafePointer(to: &addr) { ptr -> Int32 in @@ -511,6 +615,9 @@ final class HLSLocalServer: @unchecked Sendable { listenFd = fd port = assignedPort shouldStop = false + if !tokenRegistered { + tokenRegistered = LogRedaction.register(pathToken) + } stateLock.unlock() EngineLog.emit("[HLSLocalServer] Listening on port \(assignedPort)", @@ -527,6 +634,7 @@ final class HLSLocalServer: @unchecked Sendable { shouldStop = true let fdToClose = listenFd listenFd = -1 + let closingPort = port port = 0 loggedMasterPlaylist = false loggedReducedMasterPlaylist = false @@ -534,8 +642,12 @@ final class HLSLocalServer: @unchecked Sendable { mediaPlaylistBuildCount = 0 let clients = clientFds clientFds.removeAll() - let closingPort = port + let unregisterToken = tokenRegistered + tokenRegistered = false stateLock.unlock() + // Last, so a line still in flight from a connection being shut down is redacted too; the + // token opens nothing once the listener is gone. + defer { if unregisterToken { LogRedaction.unregister(pathToken) } } // AE#597: the one line that says a listener went away. Without it a log cannot tell a // server that was released from one that outlived its session on a port of its own. EngineLog.emit( @@ -583,38 +695,53 @@ final class HLSLocalServer: @unchecked Sendable { if err == EINTR || err == EAGAIN || err == ECONNABORTED { continue } - EngineLog.emit("[HLSLocalServer] accept failed errno=\(err)", - category: .hlsServer) + // EMFILE / ENFILE leave the connection in the backlog, so the next accept fails at + // once; without the pause this thread spins. + stateLock.lock() + let logLine = acceptFailureLog.admit(now: Self.uptimeSeconds()) + stateLock.unlock() + if let suppressed = logLine { + EngineLog.emit("[HLSLocalServer] accept failed errno=\(err)" + + (suppressed > 0 ? " (\(suppressed) more since the last line)" : "") + + ", backing off", category: .hlsServer) + } + usleep(Self.acceptFailureBackoffMicroseconds) continue } + let isLoopbackPeer = clientAddr.sin_family == sa_family_t(AF_INET) + && (UInt32(bigEndian: clientAddr.sin_addr.s_addr) >> 24) == 127 // SO_NOSIGPIPE on the accepted socket too, otherwise a // closed-peer send still raises SIGPIPE on iOS. var on: Int32 = 1 _ = setsockopt(clientFd, SOL_SOCKET, SO_NOSIGPIPE, &on, socklen_t(MemoryLayout.size)) - // 60s idle timeout. AVPlayer's typical inter-request gap - // is single-digit seconds; 60s is comfortable headroom. - var timeout = timeval(tv_sec: 60, tv_usec: 0) - _ = setsockopt(clientFd, SOL_SOCKET, SO_RCVTIMEO, &timeout, - socklen_t(MemoryLayout.size)) + let acceptedAt = Self.uptimeSeconds() stateLock.lock() // A thread per connection has no ceiling of its own, where the pool's 64 workers were // one. AVPlayer keeps a handful open, so anything near this number is a client that // has stopped making sense and gets the socket closed rather than a thread. let atCapacity = liveConnectionThreads >= Self.maxConcurrentConnections + || (!isLoopbackPeer && liveNonLoopbackConnections >= Self.maxNonLoopbackConnections) + var refusalLine: Int? if !atCapacity { liveConnectionThreads += 1 + if !isLoopbackPeer { liveNonLoopbackConnections += 1 } clientFds.insert(clientFd) + } else { + refusalLine = refusalLog.admit(now: acceptedAt) } stateLock.unlock() if atCapacity { - EngineLog.emit( - "[HLSLocalServer] refusing fd=\(clientFd): " - + "\(Self.maxConcurrentConnections) connections already open", - category: .hlsServer) + if let suppressed = refusalLine { + EngineLog.emit( + "[HLSLocalServer] refusing fd=\(clientFd) (\(isLoopbackPeer ? "loopback" : "LAN") peer): " + + "connection slots full" + + (suppressed > 0 ? " (\(suppressed) more refused since the last line)" : ""), + category: .hlsServer) + } close(clientFd) continue } @@ -626,9 +753,10 @@ final class HLSLocalServer: @unchecked Sendable { defer { self?.stateLock.lock() self?.liveConnectionThreads -= 1 + if !isLoopbackPeer { self?.liveNonLoopbackConnections -= 1 } self?.stateLock.unlock() } - self?.handleConnection(clientFd) + self?.handleConnection(clientFd, acceptedAt: acceptedAt) } worker.name = "com.aetherengine.hls.conn.\(clientFd)" worker.qualityOfService = .userInitiated @@ -638,7 +766,7 @@ final class HLSLocalServer: @unchecked Sendable { // MARK: - Per-connection handler - private func handleConnection(_ fd: Int32) { + private func handleConnection(_ fd: Int32, acceptedAt: TimeInterval) { defer { stateLock.lock() clientFds.remove(fd) @@ -649,30 +777,54 @@ final class HLSLocalServer: @unchecked Sendable { } // HTTP/1.1 keep-alive loop: AVPlayer reuses connections across segment fetches. Connection:close per-request tried 2026-05-20; Instruments showed it shifted the leak from libnetwork into a 570 MiB Malloc heap bucket instead (strictly worse; reverted). + // Every request that comes back true passed the session token check in `processRequest`, + // so after the first one the connection has shown it was handed a URL by this engine. + var authenticated = false while true { stateLock.lock() let stopping = shouldStop stateLock.unlock() if stopping { return } - guard let request = readHTTPRequest(fd) else { return } - guard processRequest(request, on: fd) else { return } + let firstByteDeadline = authenticated + ? Self.uptimeSeconds() + Self.keepAliveIdleSeconds + : acceptedAt + unauthenticatedHeadDeadline + guard let request = readHTTPRequest( + fd, firstByteDeadline: firstByteDeadline, authenticated: authenticated) else { return } + guard processRequest(request, on: fd, authenticated: authenticated) else { return } + authenticated = true } } /// Read until end of HTTP headers (`\r\n\r\n`). Returns the raw /// request bytes (headers only, no body, since we only accept - /// GET). Returns nil on EOF, error, or oversize. - private func readHTTPRequest(_ fd: Int32) -> Data? { + /// GET). Returns nil on EOF, error, oversize, or a missed deadline. + /// + /// The deadlines are totals rather than per-recv timeouts (audit NET-6): the first byte must + /// arrive by `firstByteDeadline`, and the whole head within `requestHeadSeconds` of it, and an + /// unauthenticated connection also by `firstByteDeadline`. A peer sending one byte a minute + /// used to keep a slot for as long as the head had room. + private func readHTTPRequest(_ fd: Int32, firstByteDeadline: TimeInterval, + authenticated: Bool) -> Data? { var buffer = Data() var chunk = [UInt8](repeating: 0, count: 4096) + var deadline = firstByteDeadline while true { + let remaining = deadline - Self.uptimeSeconds() + guard remaining > 0, Self.setReceiveTimeout(fd, seconds: remaining) else { + // A stranger that never finished a head is expected noise on a LAN listener, and + // one line each would let it scroll the host's log ring. + EngineLog.emit("[HLSLocalServer] request head deadline passed fd=\(fd) bytes=\(buffer.count)" + + (authenticated ? "" : " (connection never authenticated)"), + category: .hlsServer, level: authenticated ? .info : .verbose) + return nil + } let n = chunk.withUnsafeMutableBufferPointer { ptr -> Int in recv(fd, ptr.baseAddress, ptr.count, 0) } if n == 0 { if buffer.isEmpty { return nil } - EngineLog.emit("[HLSLocalServer] peer EOF mid-request fd=\(fd)", - category: .hlsServer) + emitRequestProblem("[HLSLocalServer] peer EOF mid-request fd=\(fd)", + authenticated: authenticated) return nil } if n < 0 { @@ -680,25 +832,40 @@ final class HLSLocalServer: @unchecked Sendable { if err == EINTR { continue } if err == EAGAIN || err == EWOULDBLOCK { EngineLog.emit("[HLSLocalServer] recv timeout fd=\(fd)", - category: .hlsServer) + category: .hlsServer, level: authenticated ? .info : .verbose) return nil } - EngineLog.emit("[HLSLocalServer] recv error fd=\(fd) errno=\(err)", - category: .hlsServer) + emitRequestProblem("[HLSLocalServer] recv error fd=\(fd) errno=\(err)", + authenticated: authenticated) return nil } + if buffer.isEmpty { + deadline = min(deadline, Self.uptimeSeconds() + Self.requestHeadSeconds) + } buffer.append(chunk, count: n) if let end = findHeadersTerminator(buffer) { return buffer.prefix(end + 4) } if buffer.count > 8192 { - EngineLog.emit("[HLSLocalServer] request too large fd=\(fd) bytes=\(buffer.count)", - category: .hlsServer) + emitRequestProblem("[HLSLocalServer] request too large fd=\(fd) bytes=\(buffer.count)", + authenticated: authenticated) return nil } } } + private static func setReceiveTimeout(_ fd: Int32, seconds: TimeInterval) -> Bool { + let whole = Int(seconds) + var timeout = timeval( + tv_sec: whole, tv_usec: Int32(max(1, (seconds - Double(whole)) * 1_000_000))) + return setsockopt(fd, SOL_SOCKET, SO_RCVTIMEO, &timeout, + socklen_t(MemoryLayout.size)) == 0 + } + + private static func uptimeSeconds() -> TimeInterval { + Double(clock_gettime_nsec_np(CLOCK_UPTIME_RAW)) / 1_000_000_000 + } + private func findHeadersTerminator(_ buf: Data) -> Int? { guard buf.count >= 4 else { return nil } let needle: [UInt8] = [0x0D, 0x0A, 0x0D, 0x0A] @@ -714,21 +881,40 @@ final class HLSLocalServer: @unchecked Sendable { } } - private func processRequest(_ request: Data, on fd: Int32) -> Bool { + /// A line caused by a connection that has not presented the token: throttled, since anyone on the + /// LAN can cause it (audit NET-111). An authenticated connection's lines go out as they come. + private func emitRequestProblem(_ line: String, authenticated: Bool) { + guard !authenticated else { + EngineLog.emit(line, category: .hlsServer) + return + } + stateLock.lock() + let admitted = strangerLog.admit(now: Self.uptimeSeconds()) + stateLock.unlock() + guard let suppressed = admitted else { return } + EngineLog.emit( + line + (suppressed > 0 + ? " (\(suppressed) more from unauthenticated connections since the last line)" : ""), + category: .hlsServer) + } + + private func processRequest(_ request: Data, on fd: Int32, authenticated: Bool) -> Bool { byteCounterLock.lock() _requestCount &+= 1 byteCounterLock.unlock() guard let text = String(data: request, encoding: .utf8) else { - EngineLog.emit("[HLSLocalServer] non-UTF8 request bytes (\(request.count)B)", - category: .hlsServer) + emitRequestProblem("[HLSLocalServer] non-UTF8 request bytes (\(request.count)B)", + authenticated: authenticated) return false } let firstLine = text.components(separatedBy: "\r\n").first ?? "" let parts = firstLine.split(separator: " ", maxSplits: 2, omittingEmptySubsequences: true) guard parts.count >= 2 else { - EngineLog.emit("[HLSLocalServer] malformed request line: '\(firstLine)'", - category: .hlsServer) + emitRequestProblem( + "[HLSLocalServer] malformed request line: " + + "'\(Self.escapedForLog(firstLine, limit: Self.maximumStrangerText))'", + authenticated: authenticated) return false } let rawTarget = String(parts[1]) @@ -746,18 +932,24 @@ final class HLSLocalServer: @unchecked Sendable { // The listener is reachable from the whole LAN, so an unprefixed request is a scan or a // stale URL, never AVPlayer following a playlist we handed out. guard let routePath = Self.pathAfterToken(pathToken, in: path) else { - EngineLog.emit("[HLSLocalServer] rejected request without a valid session token: \(firstLine)", - category: .hlsServer) - _ = send404(fd: fd, path: path, reason: "bad session token") + // One line for the rejection and the 404 together: the path is the stranger's text. + emitRequestProblem( + "[HLSLocalServer] rejected request without a valid session token, -> 404: " + + Self.escapedForLog(firstLine, limit: Self.maximumStrangerText), + authenticated: authenticated) + _ = send404(fd: fd, path: path, reason: "bad session token", logged: false) return false } let normalizedPath = (routePath == "/audio.m3u8") ? "/media.m3u8" : routePath + let loggedRequest = Self.requestLineForLog( + method: parts[0], routePath: routePath, query: query, + version: parts.count > 2 ? parts[2] : nil) // #50 diag: promoted to .info so the host mirror names the failing path without a verbose build. Revert once #50 is root-caused. // AE#446: the fd is what says whether a blocking-reload hold is parking the connection the // next segment request needs. Same fd on both, and the segment could not be read until the // hold returned; different fds, and the client chose not to fetch. - EngineLog.emit("[HLSLocalServer] \(firstLine) fd=\(fd)", category: .hlsServer) + EngineLog.emit("[HLSLocalServer] \(loggedRequest) fd=\(fd)", category: .hlsServer) // #227 diag: name each distinct client once, so an AirPlay session shows whether the receiver fetches // for itself (its own LAN address appears) or the sender pulls everything (only 127.0.0.1 / own IP). if let peer = Self.peerAddress(of: fd) { @@ -765,7 +957,7 @@ final class HLSLocalServer: @unchecked Sendable { let isNewPeer = loggedPeers.insert(peer).inserted stateLock.unlock() if isNewPeer { - EngineLog.emit("[HLSLocalServer] #227 client \(peer) first request: \(firstLine)", category: .hlsServer) + EngineLog.emit("[HLSLocalServer] #227 client \(peer) first request: \(loggedRequest)", category: .hlsServer) } } // Dump request headers once per session; AVPlayer capability headers (Accept, Range, X-Playback-Session-Id) can influence silent variant rejection. @@ -775,7 +967,7 @@ final class HLSLocalServer: @unchecked Sendable { stateLock.unlock() if dumpHeaders { let allLines = text.components(separatedBy: "\r\n") - let headers = allLines.dropFirst().prefix(while: { !$0.isEmpty }).joined(separator: " | ") + let headers = Self.requestHeadersForLog(Array(allLines.dropFirst().prefix(while: { !$0.isEmpty }))) // #50 diag: once-per-session, promoted to .info to surface any // Range / capability header that explains the 404. Revert with the // arrival-line promotion above once #50 is root-caused. @@ -942,11 +1134,45 @@ final class HLSLocalServer: @unchecked Sendable { case .missing: return send404(fd: fd, path: normalizedPath, reason: "no subtitle segment for \(normalizedPath)") } - EngineLog.emit("[HLSLocalServer] served subtitle .vtt ord=\(parsed.ordinal) seg=\(seg) bytes=\(vtt.utf8.count)", category: .hlsServer, level: .verbose) + // Sodalite#156: an EMPTY segment is the reported defect and a populated one is routine, so + // only the empty case is worth a line a reporter will see. AVKit takes the whole forward + // window in one burst (~45 segments) and never re-fetches, so logging every one of them at + // a visible level would push the surrounding evidence out of the 300-line ring, and it is + // exactly the surrounding evidence that says WHY a segment came out empty. + let cues = vtt.components(separatedBy: "-->").count - 1 + if cues == 0 { + EngineLog.emit("[HLSLocalServer] served an EMPTY subtitle .vtt ord=\(parsed.ordinal) " + + "seg=\(seg) bytes=\(vtt.utf8.count); the receiver caches this segment " + + "as it is and never asks again", category: .hlsServer) + } else { + EngineLog.emit("[HLSLocalServer] served subtitle .vtt ord=\(parsed.ordinal) seg=\(seg) " + + "bytes=\(vtt.utf8.count) cues=\(cues)", category: .hlsServer, level: .verbose) + } return send200(fd: fd, path: normalizedPath, data: Data(vtt.utf8), contentType: "text/vtt") + case "/iframe.m3u8": + guard let prov = provider, prov.iFrameRenditionServed else { + return send404(fd: fd, path: normalizedPath, reason: "no I-frame rendition") + } + let body = Self.buildIFramePlaylistText(provider: prov, subResourceBaseURL: subResourceBaseURL) + return send200(fd: fd, path: normalizedPath, data: Data(body.utf8), + contentType: "application/vnd.apple.mpegurl") + + case "/iframe_init.mp4": + guard let data = provider?.iFrameInitSegment(), !data.isEmpty else { + return send404(fd: fd, path: normalizedPath, reason: "I-frame init unavailable") + } + return send200(fd: fd, path: normalizedPath, data: data, contentType: "video/mp4") + + case let p where p.hasPrefix("/iframe") && p.hasSuffix(".mp4"): + guard let index = Self.parseIFramePath(p), + let data = provider?.iFrameSegment(at: index), !data.isEmpty else { + return send404(fd: fd, path: normalizedPath, reason: "no I-frame for \(normalizedPath)") + } + return send200(fd: fd, path: normalizedPath, data: data, contentType: "video/mp4") + case "/init.mp4": stateLock.lock(); servedMediaBytes = true; stateLock.unlock() let data = provider?.initSegment() ?? Data() @@ -1240,10 +1466,12 @@ final class HLSLocalServer: @unchecked Sendable { } } - private func send404(fd: Int32, path: String, reason: String) -> Bool { + private func send404(fd: Int32, path: String, reason: String, logged: Bool = true) -> Bool { let response = Self.responseHeader(status: "404 Not Found", contentLength: 0, contentType: nil) - EngineLog.emit("[HLSLocalServer] -> 404 \(path) reason=\(reason)", - category: .hlsServer) + if logged { + EngineLog.emit("[HLSLocalServer] -> 404 \(path) reason=\(reason)", + category: .hlsServer) + } return writeAll(fd: fd, data: response, path: path) } @@ -1489,6 +1717,69 @@ final class HLSLocalServer: @unchecked Sendable { } lines.append("#EXT-X-STREAM-INF:\(streamInfAttrs.joined(separator: ","))") lines.append("media.m3u8") + if provider.iFrameRenditionServed { + // AE#682: BANDWIDTH is the variant's, an honest ceiling (one keyframe cannot outweigh the + // segment it opens). A value below the real peak logs -12318 on every fetch. + var iFrameAttrs = ["BANDWIDTH=\(bandwidth)", "CODECS=\"\(videoCodecs(of: codecs))\""] + if variant == .primary, let supplemental = provider.masterSupplementalCodecs { + iFrameAttrs.append("SUPPLEMENTAL-CODECS=\"\(supplemental)\"") + } + if let resolution = provider.masterResolution { + iFrameAttrs.append("RESOLUTION=\(resolution.width)x\(resolution.height)") + } + if let range = provider.masterVideoRange { + iFrameAttrs.append("VIDEO-RANGE=\(range.rawValue)") + } + iFrameAttrs.append("URI=\"iframe.m3u8\"") + lines.append("#EXT-X-I-FRAME-STREAM-INF:\(iFrameAttrs.joined(separator: ","))") + } + return lines.joined(separator: "\n") + "\n" + } + + /// The video entry of a master CODECS list. An I-frame variant carries no audio, and a CODECS + /// that names one makes AVPlayer look for a track the fragments do not have. + static func videoCodecs(of codecs: String) -> String { + let videoPrefixes = ["avc1", "avc3", "hvc1", "hev1", "dvh1", "dvhe", "av01", "vp09"] + let video = codecs.split(separator: ",") + .map { $0.trimmingCharacters(in: .whitespaces) } + .filter { entry in videoPrefixes.contains { entry.hasPrefix($0) } } + return video.isEmpty ? codecs : video.joined(separator: ",") + } + + /// "/iframe{N}.mp4" -> N. nil for the init, for a negative or non-numeric index, and for any + /// other path. + static func parseIFramePath(_ path: String) -> Int? { + guard path.hasPrefix("/iframe"), path.hasSuffix(".mp4") else { return nil } + let digits = path.dropFirst("/iframe".count).dropLast(".mp4".count) + guard !digits.isEmpty, digits.allSatisfy(\.isNumber) else { return nil } + return Int(digits) + } + + /// AE#682: one entry per plan segment, each a single keyframe in its own resource. Same count, + /// EXTINF and TARGETDURATION as the VOD media playlist, so both renditions describe one timeline. + /// VOD only; the caller never lists this rendition for a live or event session. + static func buildIFramePlaylistText(provider: HLSSegmentProvider, + subResourceBaseURL: URL? = nil) -> String { + let count = provider.segmentCount + var maxDuration: Double = 0 + for i in 0.. Int? { + if let lastEmitted, now - lastEmitted < interval { + suppressed += 1 + return nil + } + lastEmitted = now + defer { suppressed = 0 } + return suppressed + } +} diff --git a/Sources/AetherEngine/Network/HLSOriginRelay.swift b/Sources/AetherEngine/Network/HLSOriginRelay.swift index 534adabdf..387ed5afd 100644 --- a/Sources/AetherEngine/Network/HLSOriginRelay.swift +++ b/Sources/AetherEngine/Network/HLSOriginRelay.swift @@ -1,3 +1,4 @@ +import CryptoKit import Foundation /// AE#495: fetches a remote origin on the engine's behalf and sends every URI a playlist @@ -17,9 +18,19 @@ import Foundation final class HLSOriginRelay: @unchecked Sendable { /// The single route the player is ever pointed at, under the server's session token. - /// The origin rides in the query, so one route covers playlists, keys and segments. + /// A sealed reference to the origin rides in the query, so one route covers playlists, keys + /// and segments. static let route = "/aether-origin-relay" - private static let originQueryKey = "origin" + private static let referenceQueryKey = "ref" + + /// Audit NET-1: the origin URL carries the media server's access token, and the local URL is + /// logged on every request, handed to AirPlay receivers and written into rewritten playlists. + /// So the local URL names the origin only through a reference sealed with keys that live and + /// die with this relay. Sealing rather than an id table keeps the state constant: a VOD + /// playlist registers every segment URI at once, and an evicted table entry would be a 404 + /// in the middle of a film. + private let sealingKey = SymmetricKey(size: .bits256) + private let nonceKey = SymmetricKey(size: .bits256) /// What a relayed request produced, for the server to write. struct Response { @@ -32,15 +43,19 @@ final class HLSOriginRelay: @unchecked Sendable { private let stateLock = NSLock() - /// Origins this relay will fetch, as scheme://host:port. Seeded by `admit(_:)` and grown - /// as playlists reveal where their own sub-resources live, so a stream split across hosts - /// keeps working while a request naming somewhere nobody advertised is still refused. + /// Origins this relay will fetch, as scheme://host:port. Seeded by `allow(_:)` and + /// `grantCredentials(to:httpHeaders:)` and grown as playlists reveal where their own + /// sub-resources live, so a stream split across hosts keeps working while a request naming + /// somewhere nobody advertised is still refused. private var allowedOrigins = Set() - /// Sent upstream on every fetch. Origins that gate on Referer, User-Agent or - /// Authorization need these, and they can no longer ride on the asset because the asset - /// now points at the local server. - private var upstreamHeaders: [String: String] = [:] + /// Sent upstream on every fetch. Origins that gate on Referer, User-Agent or Authorization need + /// these, and they can no longer ride on the asset because the asset now points at the local + /// server. The anchors are the URLs the host itself pointed the relay at: credential headers + /// follow a fetch only to one of those origins with no TLS downgrade, and an origin a playlist + /// revealed or a redirect landed on gets the rest of the headers but not the token (audit NET-7, + /// NET-109). + private var credentials = CredentialScope(headers: [:], anchors: []) /// The NSURLError code of the last upstream handshake this relay lost to system trust, if any. /// @@ -74,14 +89,28 @@ final class HLSOriginRelay: @unchecked Sendable { private var authorizationWaits: [UUID: HTTPAuthorizationWait] = [:] private var pumps: [UUID: UpstreamPump] = [:] + private let heldBodyLimit: Int + private let pendingLimit: Int + + /// Audit NET-107: fetches cut off because the consumer fell `pendingLimit` bytes behind. Diagnostics and tests. + var cappedFetchCount: Int { + stateLock.lock(); defer { stateLock.unlock() } + return _cappedFetchCount + } + private var _cappedFetchCount = 0 + init(authorization: HTTPRequestAuthorization? = nil, authorizationTimeout: TimeInterval = 10, resourceTimeout: TimeInterval = 120, deadline: Date? = nil, rawResources: Bool = false, - onRequestFailure: (@Sendable () -> Void)? = nil) { + onRequestFailure: (@Sendable () -> Void)? = nil, + maximumHeldBodyBytes: Int = HLSOriginRelay.maximumHeldBodyBytes, + maximumPendingBytes: Int = HLSOriginRelay.maximumPendingBytes) { self.rawResources = rawResources self.onRequestFailure = onRequestFailure self.authorization = authorization self.authorizationTimeout = authorizationTimeout self.deadline = deadline + heldBodyLimit = maximumHeldBodyBytes + pendingLimit = maximumPendingBytes let config = URLSessionConfiguration.ephemeral config.requestCachePolicy = .reloadIgnoringLocalAndRemoteCacheData config.urlCache = nil @@ -149,10 +178,12 @@ final class HLSOriginRelay: @unchecked Sendable { config.urlCache = nil config.timeoutIntervalForRequest = trustProbeSeconds config.timeoutIntervalForResource = trustProbeSeconds - // No delegate on purpose. This session must answer the way AVPlayer's own networking does, - // which is system trust and nothing else; handing it `EngineTLS.sessionDelegate` would ask - // the host and get back the answer that hides what is being measured. - let session = URLSession(configuration: config) + // No trust delegate on purpose. This session must answer the way AVPlayer's own networking + // does, which is system trust and nothing else; handing it `EngineTLS.sessionDelegate` would + // ask the host and get back the answer that hides what is being measured. The redirect rule + // still applies (audit NET-108). + let session = URLSession( + configuration: config, delegate: EngineTLS.redirectDelegate, delegateQueue: nil) defer { session.invalidateAndCancel() } var request = URLRequest(url: origin) @@ -177,18 +208,45 @@ final class HLSOriginRelay: @unchecked Sendable { // MARK: - Admission - /// Lets this relay fetch `origin`, and adopts the headers a load carries. Returns the + /// Lets this relay fetch `origin` without granting it the host's credentials. Returns the /// origin key, or nil for a URL with no host to fetch from. @discardableResult - func admit(_ origin: URL, httpHeaders: [String: String] = [:]) -> String? { + func allow(_ origin: URL) -> String? { guard let key = Self.originKey(for: origin) else { return nil } stateLock.lock() allowedOrigins.insert(key) - if !httpHeaders.isEmpty { upstreamHeaders = httpHeaders } stateLock.unlock() return key } + /// Lets this relay fetch `origin` AND send it the credential headers, and adopts the headers a + /// load carries. Only for a URL the host itself handed over (audit NET-109): anything a + /// playlist or a redirect produced goes through `allow(_:)`. + @discardableResult + func grantCredentials(to origin: URL, httpHeaders: [String: String] = [:]) -> String? { + guard let key = Self.originKey(for: origin) else { return nil } + stateLock.lock() + allowedOrigins.insert(key) + let anchors = credentials.anchors.contains(origin) + ? credentials.anchors : credentials.anchors + [origin] + credentials = CredentialScope( + headers: httpHeaders.isEmpty ? credentials.headers : httpHeaders, anchors: anchors) + stateLock.unlock() + return key + } + + /// The URLs whose origins receive the host's credentials. + var credentialAnchors: [URL] { + stateLock.lock(); defer { stateLock.unlock() } + return credentials.anchors + } + + /// The headers a relayed fetch of `target` carries upstream. + func upstreamHeaders(for target: URL) -> [String: String] { + stateLock.lock(); defer { stateLock.unlock() } + return credentials.headers(for: target) + } + /// scheme://host:port for `url`, which is the granularity the allow list works at. static func originKey(for url: URL) -> String? { guard let scheme = url.scheme?.lowercased(), ["http", "https"].contains(scheme), @@ -201,47 +259,41 @@ final class HLSOriginRelay: @unchecked Sendable { // MARK: - Addressing - /// The local address standing in for `origin`. - static func localURL( + /// The local address standing in for `origin`. Does not admit it. + func localURL( for origin: URL, host: String = "127.0.0.1", port: UInt16, token: String ) -> URL? { - // Encoding everything outside the alphanumerics keeps the origin's own query, which - // on a Jellyfin stream carries the api key and the play session, from being read as - // part of this URL's query. - guard - let encoded = origin.absoluteString.addingPercentEncoding( - withAllowedCharacters: .alphanumerics) - else { return nil } - return URL(string: "http://\(host):\(port)/\(token)\(route)?\(originQueryKey)=\(encoded)") + guard let reference = seal(origin) else { return nil } + return URL( + string: "http://\(host):\(port)/\(token)\(Self.route)?\(Self.referenceQueryKey)=\(reference)") } - /// The origin a relay request names, or nil when the query does not carry one. + /// The origin a relay request names, or nil when the query carries no reference this relay + /// sealed. /// /// Every other field the client put on the URL is carried onto the origin's own query rather /// than dropped. AVPlayer appends `_HLS_msn` / `_HLS_part` / `_HLS_skip` to a playlist URL when /// the playlist advertises `CAN-BLOCK-RELOAD` (#441), and a reload that should have blocked /// until the next segment exists answers immediately without them, so the player asks again at /// once and the origin is polled as fast as the loopback can answer. - static func originURL(fromQuery query: String) -> URL? { + func originURL(fromQuery query: String) -> URL? { var origin: URL? var carried: [String] = [] for field in query.split(separator: "&") { let pair = field.split(separator: "=", maxSplits: 1) - guard pair.count == 2, pair[0] == originQueryKey else { + guard pair.count == 2, pair[0] == Self.referenceQueryKey else { carried.append(String(field)) continue } - guard let decoded = String(pair[1]).removingPercentEncoding, !decoded.isEmpty else { - return nil - } - origin = URL(string: decoded) + guard let opened = open(String(pair[1])) else { return nil } + origin = opened } guard let origin else { return nil } guard !carried.isEmpty, var components = URLComponents(url: origin, resolvingAgainstBaseURL: false) else { return origin } - // percentEncoded, because both halves are already encoded: the origin's query came out of - // the round trip through `localURL`, and the client's fields arrived off the wire. + // percentEncoded, because both halves are already encoded: the origin's query is the one + // it was sealed with, and the client's fields arrived off the wire. var fields: [String] = [] if let existing = components.percentEncodedQuery, !existing.isEmpty { fields.append(existing) } fields.append(contentsOf: carried) @@ -249,6 +301,36 @@ final class HLSOriginRelay: @unchecked Sendable { return components.url ?? origin } + /// AES-GCM under a nonce derived from the plaintext, so one origin always seals to the same + /// reference: a live playlist refreshed every few seconds names the same segment by the same + /// local URL each time. Equal plaintexts are the only nonce reuse, and they reveal nothing + /// beyond their equality. base64url without padding, which a query carries as it stands. + private func seal(_ origin: URL) -> String? { + let plaintext = Data(origin.absoluteString.utf8) + let derived = HMAC.authenticationCode(for: plaintext, using: nonceKey) + guard let nonce = try? AES.GCM.Nonce(data: Data(derived).prefix(12)), + let sealed = try? AES.GCM.seal(plaintext, using: sealingKey, nonce: nonce), + let combined = sealed.combined + else { return nil } + return combined.base64EncodedString() + .replacingOccurrences(of: "+", with: "-") + .replacingOccurrences(of: "/", with: "_") + .replacingOccurrences(of: "=", with: "") + } + + private func open(_ reference: String) -> URL? { + var base64 = reference + .replacingOccurrences(of: "-", with: "+") + .replacingOccurrences(of: "_", with: "/") + base64 += String(repeating: "=", count: (4 - base64.count % 4) % 4) + guard let combined = Data(base64Encoded: base64), + let box = try? AES.GCM.SealedBox(combined: combined), + let plaintext = try? AES.GCM.open(box, using: sealingKey), + let string = String(data: plaintext, encoding: .utf8), !string.isEmpty + else { return nil } + return URL(string: string) + } + /// The authority a rewritten playlist should point its sub-resources at: whatever the /// client used to reach the server, so a receiver that arrived on the LAN address keeps /// fetching from it rather than being sent to a loopback it resolves to itself (#86). @@ -293,12 +375,12 @@ final class HLSOriginRelay: @unchecked Sendable { func respond(query: String, host: String?, range: String?, port: UInt16, token: String, sink: Sink) -> Outcome? { - guard let origin = Self.originURL(fromQuery: query) else { return nil } + guard let origin = originURL(fromQuery: query) else { return nil } guard let key = Self.originKey(for: origin) else { return nil } stateLock.lock() let permitted = allowedOrigins.contains(key) - let headers = upstreamHeaders + let headers = credentials.headers(for: origin) stateLock.unlock() guard permitted else { EngineLog.emit( @@ -368,6 +450,18 @@ final class HLSOriginRelay: @unchecked Sendable { case failed } + /// Audit NET-10: a held body is buffered whole, and the resource timeout alone let a hostile or + /// broken origin grow one for two minutes at link rate. A playlist larger than this is not one a + /// player will get through; a held media body (a segment served without a length) gets the wider + /// cap, and an error body is dropped past it while its status still goes through. + static let maximumHeldPlaylistBytes = 16 * 1024 * 1024 + static let maximumHeldBodyBytes = 64 * 1024 * 1024 + + /// Audit NET-107: what one streamed fetch may hold for a consumer that is not draining. Above any + /// segment a player asks for in one request, so a consumer that is merely slow is never cut off, + /// and far enough under a session's memory that a stalled one cannot turn it into a heap. + static let maximumPendingBytes = 32 * 1024 * 1024 + /// The answers that mean "you are asking too often", which arm the pacer for this origin. private static let refusalStatuses: Set = [429, 503, 509] @@ -495,8 +589,10 @@ final class HLSOriginRelay: @unchecked Sendable { } /// Playlist preflight shares exactly the relay's redirect, challenge and authorization contract. - func fetchPlaylist(_ url: URL, headers: [String: String]) async throws -> (String, URL) { - let fetched = try await fetchWhole(url, headers: headers) + /// `maximumBytes` bounds the body as it arrives (audit NET-10), like the static-header preflight. + func fetchPlaylist(_ url: URL, headers: [String: String], + maximumBytes: Int? = nil) async throws -> (String, URL) { + let fetched = try await fetchWhole(url, headers: headers, maximumBytes: maximumBytes) guard let body = String(data: fetched.body, encoding: .utf8) else { throw URLError(.cannotDecodeContentData) } @@ -548,7 +644,12 @@ final class HLSOriginRelay: @unchecked Sendable { for: origin, label: "relay", timeout: slotWait) defer { OriginRequestBudget.shared.release(ticket) } - let pump = UpstreamPump(maximumBytes: maximumBytes) + let pump = UpstreamPump(hardCapBytes: pendingLimit) { [weak self] in + guard let self else { return } + self.stateLock.lock() + self._cappedFetchCount += 1 + self.stateLock.unlock() + } let id = UUID() stateLock.lock() guard !stopped, remainingBudget(upTo: 1) > 0 else { @@ -595,13 +696,27 @@ final class HLSOriginRelay: @unchecked Sendable { } let contentType = http.value(forHTTPHeaderField: "Content-Type") let contentRange = http.value(forHTTPHeaderField: "Content-Range") + // A length the origin did not state cannot be framed for the player without buffering the + // body to measure it, and a playlist has to be read whole to be rewritten at all. let declaredLength = http.expectedContentLength - let mustHold = forceHold || !(200..<300).contains(http.statusCode) - || (!rawResources && (declaredLength < 0 - || Self.looksLikePlaylist(url: responseURL, contentType: contentType))) + let isPlaylist = !rawResources && Self.looksLikePlaylist(url: responseURL, contentType: contentType) + let isSuccess = (200..<300).contains(http.statusCode) + let mustHold = forceHold || !isSuccess || isPlaylist || (!rawResources && declaredLength < 0) if mustHold { - let body = pump.awaitWholeBody() + // A caller's own bound (a bounded resource, a playlist preflight) replaces the defaults. + let limit = maximumBytes + ?? (isPlaylist ? min(Self.maximumHeldPlaylistBytes, heldBodyLimit) : heldBodyLimit) + guard let body = pump.awaitWholeBody(limit: limit) else { + EngineLog.emit( + "[HLSOriginRelay] \(origin.host ?? "origin") answered \(http.statusCode) with a body over " + + "\(limit) bytes; not holding it", category: .hlsServer) + if isSuccess { return .complete(.failed) } + return .complete(.held(Fetched(url: responseURL, status: http.statusCode, body: Data(), + contentType: contentType, contentRange: contentRange))) + } if let error = pump.awaitFailure() { + // A playlist read halfway is not a playlist, and the framing of a held answer is its + // own length, so there is nothing here worth passing on. note(failure: error, origin: origin) return .complete(.failed) } @@ -611,6 +726,8 @@ final class HLSOriginRelay: @unchecked Sendable { guard sink.head(http.statusCode, contentType ?? "application/octet-stream", contentRange, Int(declaredLength)) else { return .complete(.streamed(ok: false)) } let written = pump.drain(into: sink.body) + // A transport error after the head is already out leaves the body short of the length that + // was promised, so the answer cannot be finished; the server closes the connection on false. if let error = pump.awaitFailure() { note(failure: error, origin: origin) return .complete(.streamed(ok: false)) @@ -655,7 +772,7 @@ final class HLSOriginRelay: @unchecked Sendable { let rewriteOne: (String) -> String = { raw in if absoluteOnly, URL(string: raw)?.host == nil { return raw } guard let resolved = URL(string: raw, relativeTo: origin)?.absoluteURL, - let local = Self.localURL( + let local = self.localURL( for: resolved, host: authority, port: port, token: token) else { return raw } if let key = Self.originKey(for: resolved) { discovered.insert(key) } @@ -707,14 +824,23 @@ final class HLSOriginRelay: @unchecked Sendable { /// land. Holding them here instead, and writing from the delegate callback, would put a socket the /// player has stopped reading in front of every other task on this session's serial delegate queue. /// -/// The bound is what makes the handoff backpressure rather than an unbounded copy of the body: the -/// producer waits once the consumer is that far behind, which is the same shape the direct route has -/// when a socket stops draining. +/// Audit NET-107: the producer never waits. The delegate queue is serial across every task on the +/// session, so parking it inside `didReceive` for one consumer that stopped reading (an AirPlay +/// receiver on a slow link) held up the head and body of every other relayed fetch for a whole +/// park. Backpressure by `URLSessionTask.suspend()` is no way out: transports that ignore it were +/// measured (#220), and suspended flows correlated with every connection in the process going deaf +/// (#310). So the bound is a cap on what one fetch may hold: past it the fetch is cancelled, the +/// body the player was promised ends short and the server closes the connection, which is how a +/// truncated transfer already reads, and the player asks again. private final class UpstreamPump: NSObject, URLSessionDataDelegate, @unchecked Sendable { - /// One segment's worth of slack. Enough that a fast origin never waits on a loopback write, small - /// enough that a stalled player cannot turn a session into a heap of parked segments. - private static let highWaterBytes = 4 * 1024 * 1024 + private let hardCapBytes: Int + private let onCapped: @Sendable () -> Void + + init(hardCapBytes: Int, onCapped: @escaping @Sendable () -> Void) { + self.hardCapBytes = hardCapBytes + self.onCapped = onCapped + } private let condition = NSCondition() private var head: HTTPURLResponse? @@ -722,10 +848,7 @@ private final class UpstreamPump: NSObject, URLSessionDataDelegate, @unchecked S private var finished = false private var failure: Error? private var consumerGaveUp = false - private let maximumBytes: Int? - private var receivedBytes = 0 - - init(maximumBytes: Int? = nil) { self.maximumBytes = maximumBytes } + private var capped = false // MARK: - Consumer, on the server's worker thread @@ -737,14 +860,20 @@ private final class UpstreamPump: NSObject, URLSessionDataDelegate, @unchecked S return head } - /// Every byte of the body, for the answers that have to be read whole. - func awaitWholeBody() -> Data { + /// Every byte of the body, for the answers that have to be read whole, or nil once it passes + /// `limit`, which also stops the transfer. + func awaitWholeBody(limit: Int) -> Data? { + if let head = awaitHead(), head.expectedContentLength > Int64(limit) { + abandon() + return nil + } var body = Data() - _ = drain { chunk in + let complete = drain { chunk in + guard body.count + chunk.count <= limit else { return false } body.append(chunk) return true } - return body + return complete ? body : nil } /// Hands each chunk to `write` as it arrives, until the body ends or a write fails. Returns @@ -771,8 +900,7 @@ private final class UpstreamPump: NSObject, URLSessionDataDelegate, @unchecked S } } - /// Stops the producer waiting on a consumer that is no longer there. Without it a body parked at - /// the high-water mark holds the delegate queue for the life of the session. + /// Tells the producer to stop buffering for a consumer that is no longer there. func abandon() { condition.lock() consumerGaveUp = true @@ -804,33 +932,33 @@ private final class UpstreamPump: NSObject, URLSessionDataDelegate, @unchecked S } condition.lock() head = http - let tooLarge = maximumBytes.map { http.expectedContentLength > Int64($0) } ?? false - if tooLarge { failOverLimit() } condition.broadcast() condition.unlock() - completionHandler(tooLarge ? .cancel : .allow) + completionHandler(.allow) } func urlSession(_ session: URLSession, dataTask: URLSessionDataTask, didReceive data: Data) { condition.lock() - while pending.count >= Self.highWaterBytes && !consumerGaveUp { condition.wait() } - if let maximumBytes, data.count > maximumBytes - receivedBytes { failOverLimit() } - let abandoned = consumerGaveUp - if !abandoned { - if maximumBytes != nil { receivedBytes += data.count } - pending.append(data) - condition.broadcast() + var stop = consumerGaveUp || capped + var newlyCapped = false + if !stop { + if pending.count + data.count > hardCapBytes { + capped = true + newlyCapped = true + stop = true + } else { + pending.append(data) + condition.broadcast() + } } condition.unlock() - if abandoned { dataTask.cancel() } - } - - /// The body passed `maximumBytes`: fail it and stop accepting bytes. Caller holds `condition`. - private func failOverLimit() { - failure = URLError(.dataLengthExceedsMaximum) - finished = true - consumerGaveUp = true - condition.broadcast() + if newlyCapped { + EngineLog.emit( + "[HLSOriginRelay] the consumer fell more than \(hardCapBytes >> 20) MiB behind one fetch; " + + "cancelling it so the player asks again", category: .hlsServer) + onCapped() + } + if stop { dataTask.cancel() } } func urlSession(_ session: URLSession, task: URLSessionTask, didCompleteWithError error: Error?) { diff --git a/Sources/AetherEngine/PlaybackClock.swift b/Sources/AetherEngine/PlaybackClock.swift index b40ff7b1e..dee6e843a 100644 --- a/Sources/AetherEngine/PlaybackClock.swift +++ b/Sources/AetherEngine/PlaybackClock.swift @@ -11,8 +11,22 @@ public final class PlaybackClock: ObservableObject { @Published public internal(set) var currentTime: Double = 0 /// Source PTS of the currently displayed frame. On native: rides AVPlayer's rendered position -- equals `currentTime` in steady play, but holds the on-screen frame during a seek or rebuffer, not the scrub target (issue #49). SW/audio: always equals `currentTime`. + /// + /// `nativeRemoteHLS` (AE#616): item time, less the lead over the picture the engine measured on its + /// own injected subtitle renditions (#316). The lead exists where an origin restarts a transcode at the + /// keyframe before a segment's slot, and is re-measured on every presented line, so it differs from + /// `currentTime` by that lead. With no injected rendition selected there is nothing to measure it + /// against, and this is item time, which can run ahead of the frame after a seek on such an origin. @Published public internal(set) var sourceTime: Double = 0 + /// Whether `sourceTime` is known to follow the displayed frame. True on every route but + /// `nativeRemoteHLS`. There (AE#616) it turns true when a presented line of an injected rendition + /// measured the lead, and false again at every time jump (seek, item change): until the next line, + /// `sourceTime` carries the previous lead, which is off by however far the new anchor moved. False + /// for the whole session without an injected rendition selected. A host timing its own overlay off + /// `sourceTime` can hold it while this is false instead of detecting seeks itself. + @Published public internal(set) var sourceTimeFollowsPicture: Bool = true + @Published public internal(set) var progress: Float = 0 /// Largest session-relative time reached on a live source. 0 when not live. diff --git a/Sources/AetherEngine/PlayerState.swift b/Sources/AetherEngine/PlayerState.swift index c11d05a55..8052287d3 100644 --- a/Sources/AetherEngine/PlayerState.swift +++ b/Sources/AetherEngine/PlayerState.swift @@ -103,8 +103,9 @@ public enum VideoRoute: String, Sendable, Equatable { public enum AudioDelivery: String, Sendable, Equatable, CaseIterable { /// No session: pre-load, or torn down. case none - /// The source carries no audio stream (or none was selected). Silence is the source's, not the - /// engine's, and no ladder rung can change it. + /// The source carries no audio stream. Silence is the source's, not the engine's, and no ladder + /// rung can change it. A source whose audio stream the pick passed over (its parameters left + /// empty by the probe) is `.droppedNoPipeline`, not this (AE#641). case noAudioInSource /// The source's audio bitstream is muxed into fMP4 unchanged: Atmos, DTS-HD and every other /// bitstream reach the renderer exactly as authored. @@ -118,9 +119,11 @@ public enum AudioDelivery: String, Sendable, Equatable, CaseIterable { /// libavcodec decodes the audio and the engine renders it itself (the software path and the /// software audio-only host). case decoded - /// The source HAS audio and none of it could be delivered: no libavcodec decoder for it, or the - /// bridge could not be built or could not write its header. The session plays video-only and - /// silently. This is the one value a fallback ladder acts on. + /// The source HAS audio and none of it could be delivered: no libavcodec decoder for it, the + /// bridge could not be built or could not write its header, no stream could be picked because the + /// probe left its parameters empty, or a live bridge was built and its decoder produced nothing, + /// after which the engine rebuilt the session without the track (AE#641). The session plays + /// video-only and silently. This is the one value a fallback ladder acts on. case droppedNoPipeline /// AVFoundation owns the audio: the remote-HLS bypass and the native audio-only host both hand /// the source to AVPlayer, which does its own media selection. The engine has no pipeline of its @@ -536,7 +539,7 @@ public struct LoadOptions: Sendable, Equatable { /// Lean audio-only path (FFmpeg + AVSampleBufferAudioRenderer): skips video probe, display-criteria handshake, HLS/muxer/loopback stack. Also set automatically when the probe finds no video stream. Default `false`. public var audioOnly: Bool - /// DVR rewind window in seconds; nil = live-only (seek is a no-op). Engine retains roughly this much past content disk-backed. Suggested default: 1800. Ignored when `isLive == false`. Default nil. + /// DVR rewind window in seconds; nil = live-only (seek is a no-op). Engine retains roughly this much past content disk-backed, bounded by the session disk budget (a quarter of the free space, at most 2 GiB), so a long window on a high-bitrate channel or a small volume holds less than it asks for. Suggested default: 1800. Ignored when `isLive == false`. Default nil. public var dvrWindowSeconds: Double? /// LL-HLS blocking-reload (`#EXT-X-SERVER-CONTROL:CAN-BLOCK-RELOAD`) override for live loopback sessions. @@ -554,8 +557,10 @@ public struct LoadOptions: Sendable, Equatable { /// live-edge holdback (`HOLD-BACK` >= 3 x TARGETDURATION, RFC 8216bis) the first manifest is gated on /// (AE#189) becomes >= 18s, which a strict-realtime origin can only fill in wall-clock time (10-18s of /// black on an IPTV zap). `.fastZap` cuts at every keyframe past 0.5s instead: segments quantize to the - /// source keyframe cadence, TARGETDURATION follows the real GOP length, and the holdback shrinks with - /// it. The first serve still prefers the full holdback, but after two finalized segments a + /// source keyframe cadence, TARGETDURATION follows the real GOP length with 1.5x headroom over the + /// longest GOP seen before the seal (a broadcast's GOPs are irregular and each segment is one whole + /// GOP, AE#670), and the holdback shrinks with it: 1 s GOPs serve TARGETDURATION 2, a 6 s holdback. + /// The first serve still prefers the full holdback, but after two finalized segments a /// strict-realtime source gets one observed-segment grace clamped to 0.5...2.0s, then may serve a /// shallow first window. This bounds black-screen startup but may produce one early `-16832` or a /// short rebuffer. `.standard` retains the full-holdback guarantee. A smaller TARGETDURATION also @@ -616,7 +621,17 @@ public struct LoadOptions: Sendable, Equatable { /// from wherever the source can still serve, so a host that turns this off owns the eviction case too. public var clampsLiveResumeToWindow: Bool = true - /// AVPlayer item from the remote URL directly (Jellyfin live `master.m3u8`): no demuxer probe, no loopback. AVPlayer manages live edge / reconnect. Pair with `isLive: true`. Default `false`. + /// AVPlayer item from the remote URL directly: no demuxer probe, no loopback. AVPlayer manages live + /// edge / reconnect. Built for live (`isLive: true`, Jellyfin live `master.m3u8`); a remote HLS VOD + /// URL lands here too, whatever this says, because the loopback path reroutes it (AE#154). Default + /// `false`. + /// + /// On this route the clock is AVPlayer's item time, and that is not always the media time of the + /// frame on screen (AE#616). An origin whose playlist places a segment at its slot while the segment + /// starts at the keyframe before it (a Jellyfin transcode restarted by a seek) makes item time lead + /// the picture by that gap. `clock.sourceTime` subtracts the gap while one of the renditions the + /// engine injects for `LoadOptions.externalSubtitles` (#316) is selected and presenting; without one + /// it is item time. `clock.currentTime` and `seek(to:)` stay on item time either way. public var nativeRemoteHLS: Bool /// Reroute a live `nativeRemoteHLS` session onto the loopback live-ingest path when AVPlayer reaches @@ -643,6 +658,9 @@ public struct LoadOptions: Sendable, Equatable { /// Start the native WebVTT subtitle readers eagerly at load (instead of lazily on `setNativeSubtitleSelected`), so the `/subs_N_M.vtt` segments are already populated when AVKit fetches them under a host-independent selection (e.g. an `EXT-X-MEDIA ... DEFAULT=YES` rendition that AVKit auto-selects). Equivalent to a fully-populated static VOD subtitle file. Only meaningful with `prepareNativeSubtitles`. Default `false` (Sodalite#32 probe). public var eagerNativeSubtitleReaders: Bool = false + /// Serve an I-frame rendition (`EXT-X-I-FRAME-STREAM-INF`) next to the master, so a stock `AVPlayerViewController` shows its own scrub thumbnails and can scan on I-frames, with no host code (AE#682). One keyframe per served segment, at the source's full resolution. Costs a second reader on the source for the whole session, opened shortly after load because AVKit asks for the first keyframe before anyone scrubs. Silently absent, with one log line naming the reason, when the session cannot answer every listed keyframe: live, a source without a trustworthy keyframe index (MPEG-TS), `sequentialOrigin`, `heldSourceConnection`, an origin limited to one request, a disc source, a custom reader that cannot clone, or media-playlist routing. A host with its own transport bar wants `scrubThumbnail` instead. A tuning field: correctable through `reloadAtCurrentPosition(applying:)`. Default `false`. + public var serveIFramePlaylist: Bool = false + /// Confirm E-AC-3 JOC (Dolby Atmos) on this session's audio tracks, so `audioTracks` carries an honest /// `TrackInfo.isAtmos` for a badge instead of the pre-decode guess. No container reliably declares JOC, so /// this runs the same bounded decode pass as `AetherEngine.probeDetectingAtmos` (see `AtmosDetectionOptions` @@ -731,7 +749,7 @@ public struct LoadOptions: Sendable, Equatable { /// window length outright). nil keeps the demuxer's own value. Default nil. public var declaredDurationSeconds: Double? = nil - /// Caller-bounded demux probe budget in bytes, mapped to `AVFormatContext.probesize` for the main playback open. nil keeps the engine default (50 MB). A smaller value speeds `find_stream_info` on slow remote sources whose sparse streams (PGS, mjpeg cover art) would otherwise read to the full budget. An over-tight budget fails OPEN, not closed: `find_stream_info` still returns success with a logged warning, so the session loads with late-resolving tracks silently missing rather than throwing a load error. The value is written to the context verbatim (FFmpeg's AVOption floor of 32 is bypassed), so validate track presence after load if you set this aggressively. The routing `probe(url:)` API and still extraction keep the full budget; the embedded subtitle side-demuxer caps its own probe (it only needs codec ids, not resolved sparse tracks) and tightens to this value when it is smaller (#76). Default nil (#68). + /// Caller-bounded demux probe budget in bytes, mapped to `AVFormatContext.probesize` for the main playback open. nil keeps the engine default (50 MB). A smaller value speeds `find_stream_info` on slow remote sources whose sparse streams (PGS, mjpeg cover art) would otherwise read to the full budget. An over-tight budget fails OPEN only while some stream resolves inside it: `find_stream_info` then returns success with a logged warning, and the session loads with late-resolving tracks silently missing. When NO stream resolves inside the budget it returns an error (`-1`, rendered "Operation not permitted") and the load throws. `[Demuxer] open timings` prints the connect / open_input / find_stream_info split of every open, which is the measurement to take before and after tightening it (AE#678). The value is written to the context verbatim (FFmpeg's AVOption floor of 32 is bypassed), so validate track presence after load if you set this aggressively. The routing `probe(url:)` API and still extraction keep the full budget; the embedded subtitle side-demuxer caps its own probe (it only needs codec ids, not resolved sparse tracks) and tightens to this value when it is smaller (#76). Default nil (#68). public var probesize: Int64? /// Caller-bounded demux probe budget in microseconds, mapped to `AVFormatContext.max_analyze_duration` for the main playback open. nil keeps the engine default (60 s). Pass a positive value to set an explicit cap; do NOT pass `0` expecting "no cap": FFmpeg maps `0` to a container-dependent heuristic (~5-7 s for MPEG-TS, longer elsewhere) that is SHORTER than the engine's 60 s default. Same scope and fail-open trade-off as `probesize`. Default nil (#68). @@ -842,6 +860,22 @@ public struct LoadOptions: Sendable, Equatable { /// is demuxed), so this has nothing to act on there and the engine says so in the log. public var preferredDecodePath: DecodePath = .automatic + /// Whether a native session AVPlayer refuses on its merits may be rebuilt on the software path + /// (AE#561). Default `true`. + /// + /// When AVPlayer fails the item with a verdict on the MEDIA (`CoreMediaErrorDomain`), every native + /// recovery answers the same bytes again, so the engine spends one rebuild per session on + /// `SoftwarePlaybackHost`, whose libavcodec skips the frame Apple's parser refused. `false` + /// declines that rung: the failure surfaces as `.error` with `PlaybackErrorKind.nativeItemFailed`, + /// the way it did before 7.9.0, for a host that re-plans a failing title with a ladder of its own + /// (AE#629). Either way `softwarePathEscalations` says when the rung is taken. + /// + /// A tuning field: correctable on a playing session through `reloadAtCurrentPosition(applying:)`. + public var escalatesToSoftwarePath: Bool = true + + /// Sodalite#175: `.secondary` for an engine running beside the one that owns the panel. Default `.primary`. + public var sharedOutputRole: SharedOutputRole + /// ENGINE-INTERNAL: marks this load as a live REJOIN (`reloadAtCurrentPosition`). Not settable from the public initializer. When true, the native load path skips its explicit initial seek so AVPlayer picks edge-minus-holdback (see `LiveReloadPolicy`); without it the reloaded item can wedge in `waitingToPlay` against Jellyfin's re-served backlog. Meaningful only when `isLive` is true. var isLiveRejoin: Bool = false @@ -878,6 +912,7 @@ public struct LoadOptions: Sendable, Equatable { preserveASSMarkup: Bool = false, prepareNativeSubtitles: Bool = false, eagerNativeSubtitleReaders: Bool = false, + serveIFramePlaylist: Bool = false, confirmAtmos: Bool = false, nativeSubtitlePreferredLanguages: [String] = [], sequentialOrigin: Bool = false, @@ -895,7 +930,9 @@ public struct LoadOptions: Sendable, Equatable { audioDelaySeconds: Double = 0, deinterlaceMode: DeinterlaceMode = .auto, deinterlaceFieldRate: DeinterlaceFieldRate = .field, - preferredDecodePath: DecodePath = .automatic + preferredDecodePath: DecodePath = .automatic, + escalatesToSoftwarePath: Bool = true, + sharedOutputRole: SharedOutputRole = .primary ) { self.omitCriteriaColorExtensions = omitCriteriaColorExtensions self.suppressDisplayCriteria = suppressDisplayCriteria @@ -922,6 +959,7 @@ public struct LoadOptions: Sendable, Equatable { self.preserveASSMarkup = preserveASSMarkup self.prepareNativeSubtitles = prepareNativeSubtitles self.eagerNativeSubtitleReaders = eagerNativeSubtitleReaders + self.serveIFramePlaylist = serveIFramePlaylist self.confirmAtmos = confirmAtmos self.nativeSubtitlePreferredLanguages = nativeSubtitlePreferredLanguages self.sequentialOrigin = sequentialOrigin @@ -940,6 +978,8 @@ public struct LoadOptions: Sendable, Equatable { self.deinterlaceMode = deinterlaceMode self.deinterlaceFieldRate = deinterlaceFieldRate self.preferredDecodePath = preferredDecodePath + self.escalatesToSoftwarePath = escalatesToSoftwarePath + self.sharedOutputRole = sharedOutputRole } } @@ -982,6 +1022,9 @@ public struct SourceProbe: Sendable { public let isDolbyVision: Bool /// Dolby Vision profile number (5, 7, 8, 10) read from the dvcC/dvvC configuration record; nil when not DV. public let dvProfile: Int? + /// AE#658: pixel format, bit depth, colour description and profile of the video stream; nil when + /// the source has no video. + public let videoStreamFormat: VideoStreamFormat? /// HDR10+ (ST 2094-40) dynamic metadata was SEEN in this source's video. /// /// Always `false` unless the probe was asked for `.hdr10Plus` (the container carries no such declaration, @@ -991,6 +1034,14 @@ public struct SourceProbe: Sendable { /// Separate from `videoFormat == .hdr10Plus` because a Dolby Vision source can carry an HDR10+ layer too /// (Blu-ray Profile 7 and the 8.1 remuxes of it), and that source keeps reading `.dolbyVision`. public internal(set) var carriesHDR10PlusMetadata: Bool + /// HDR Vivid (CUVA T/UWA 005.1) dynamic metadata was SEEN in this source's HEVC video (#699). + /// + /// Same contract as `carriesHDR10PlusMetadata`: always `false` unless the probe was asked for + /// `.hdrVivid`, and `false` never means "proven absent". `videoFormat` does not move: HDR Vivid rides + /// an HLG or PQ base layer, the display is switched for that base, and the label keeps saying + /// `.hlg` / `.hdr10`. Apple platforms do not apply the dynamic metadata; the flag exists so a host can + /// label the source. + public internal(set) var carriesHDRVividMetadata: Bool /// Settable inside the module so `probeDetectingAtmos` can enrich one track without rebuilding the struct field by field. public internal(set) var audioTracks: [TrackInfo] /// Includes both text and bitmap (PGS / DVB) variants. @@ -1011,11 +1062,14 @@ public struct SourceProbe: Sendable { isDolbyVision: Bool, dvProfile: Int? = nil, carriesHDR10PlusMetadata: Bool = false, + carriesHDRVividMetadata: Bool = false, audioTracks: [TrackInfo], subtitleTracks: [TrackInfo], metadata: MediaMetadata = MediaMetadata(title: nil, artist: nil, album: nil, artworkData: nil), - isLive: Bool = false + isLive: Bool = false, + videoStreamFormat: VideoStreamFormat? = nil ) { + self.videoStreamFormat = videoStreamFormat self.url = url self.durationSeconds = durationSeconds self.videoFormat = videoFormat @@ -1027,6 +1081,7 @@ public struct SourceProbe: Sendable { self.isDolbyVision = isDolbyVision self.dvProfile = dvProfile self.carriesHDR10PlusMetadata = carriesHDR10PlusMetadata + self.carriesHDRVividMetadata = carriesHDRVividMetadata self.audioTracks = audioTracks self.subtitleTracks = subtitleTracks self.metadata = metadata @@ -1054,6 +1109,10 @@ public struct SoftwareDecodeProbeResult: Sendable { /// container that withheld its PTS and had one invented from decode order produces a sawtooth, /// which is the one shape no packet-level or renderer-level counter can see. public let frameTimesSeconds: [Double] + /// AE#654: the colour tags the first picture reached the display layer with, as + /// `primaries / transfer / matrix` in CoreVideo's names, `-` for a missing one. An untagged source + /// reads `ITU_R_709_2` in all three here, the same as VideoToolbox's own output for it. + public let firstFrameColor: String? public init( codecName: String, @@ -1069,9 +1128,11 @@ public struct SoftwareDecodeProbeResult: Sendable { firstFrameWidth: Int, firstFrameHeight: Int, firstError: String?, - frameTimesSeconds: [Double] = [] + frameTimesSeconds: [Double] = [], + firstFrameColor: String? = nil ) { self.frameTimesSeconds = frameTimesSeconds + self.firstFrameColor = firstFrameColor self.codecName = codecName self.codecID = codecID self.width = width @@ -1123,7 +1184,26 @@ public struct TrackInfo: Identifiable, Sendable, Equatable { /// track. Hosts can avoid presenting overlay controls that cannot affect it. public let isNativelyRenderedSubtitle: Bool - public init(id: Int, name: String, codec: String, language: String?, channels: Int = 0, bitrate: Int64 = 0, isDefault: Bool, isForced: Bool = false, isHearingImpaired: Bool = false, isCommentary: Bool = false, isAtmos: Bool = false, assHeader: String? = nil, isExternal: Bool = false, isNativelyRenderedSubtitle: Bool = false) { + /// AE#658, audio only: sample rate in Hz, 0 when undeclared. + public let sampleRate: Int + /// AE#658, audio only: bits per sample the stream carries (`bits_per_raw_sample`), 0 where the codec + /// has no fixed depth (AAC, AC-3, E-AC-3, Opus decode to float and have none to report). + public let bitsPerSample: Int + /// AE#658, audio only: the decoder's output sample format in libav's names ("fltp", "s32p", "s16"), + /// nil when the probe had no decoder for the stream. + public let sampleFormat: String? + /// AE#658, audio only: the channel layout as libav describes it ("stereo", "5.1(side)", "7.1"). + public let channelLayout: String? + /// AE#658: codec profile as libavcodec names it ("LC", "DTS-HD MA + DTS:X", "Dolby TrueHD + Dolby Atmos"), + /// nil when undeclared. This is where DTS:X and TrueHD Atmos show up; `isAtmos` covers E-AC-3 JOC only. + public let profile: String? + + public init(id: Int, name: String, codec: String, language: String?, channels: Int = 0, bitrate: Int64 = 0, isDefault: Bool, isForced: Bool = false, isHearingImpaired: Bool = false, isCommentary: Bool = false, isAtmos: Bool = false, assHeader: String? = nil, isExternal: Bool = false, isNativelyRenderedSubtitle: Bool = false, sampleRate: Int = 0, bitsPerSample: Int = 0, sampleFormat: String? = nil, channelLayout: String? = nil, profile: String? = nil) { + self.sampleRate = sampleRate + self.bitsPerSample = bitsPerSample + self.sampleFormat = sampleFormat + self.channelLayout = channelLayout + self.profile = profile self.id = id self.name = name self.codec = codec diff --git a/Sources/AetherEngine/Recording/LiveRecordingQueue.swift b/Sources/AetherEngine/Recording/LiveRecordingQueue.swift index 655093ee7..efb635ddb 100644 --- a/Sources/AetherEngine/Recording/LiveRecordingQueue.swift +++ b/Sources/AetherEngine/Recording/LiveRecordingQueue.swift @@ -62,17 +62,19 @@ final class LiveRecordingQueue: @unchecked Sendable { } /// Drains what is queued and stops accepting. Blocks the CALLER, never the demux thread: only - /// `stopRecording` and teardown call it. + /// the writer's teardown calls it, and never from the main actor. func finish() { lock.lock() finished = true lock.unlock() - // Let an in-flight pump finish, then drain whatever it left behind, then let that finish. - queue.sync { } - pump() - queue.sync { } + // The queue is serial, so an in-flight pump finishes first and this one drains what it left. + queue.sync { pump() } } + /// Audit REC-1: takes the backlog a batch at a time. `removeFirst()` per item shifted the whole + /// array each time, quadratic on the tens of thousands of small packets the ceiling admits. The + /// bytes still leave `pendingBytes` one write at a time, so the ceiling keeps counting a batch + /// that is taken but not yet on disk. private func pump() { while true { lock.lock() @@ -81,10 +83,15 @@ final class LiveRecordingQueue: @unchecked Sendable { lock.unlock() return } - let item = pending.removeFirst() - pendingBytes -= item.bytes.count + var batch: [QueuedPacket] = [] + swap(&batch, &pending) lock.unlock() - drain(item) + for item in batch { + drain(item) + lock.lock() + pendingBytes -= item.bytes.count + lock.unlock() + } } } } diff --git a/Sources/AetherEngine/Recording/LiveRecordingWriter.swift b/Sources/AetherEngine/Recording/LiveRecordingWriter.swift index 0c443e703..0079b8204 100644 --- a/Sources/AetherEngine/Recording/LiveRecordingWriter.swift +++ b/Sources/AetherEngine/Recording/LiveRecordingWriter.swift @@ -32,6 +32,11 @@ final class LiveRecordingWriter: LiveRecordingSink, @unchecked Sendable { private var videoSourceStreamIndex: Int32? private var reportedFailure = false private var teardownScheduled = false + /// The failure a scheduled teardown is ending the recording with, kept so whoever else ends the + /// recording meanwhile (a stop, a zap) can report it instead of calling the file cleanly ended. + private var pendingFailure: RecordingFailure? + /// Entered when a teardown is scheduled, left when it has run: `finish(reason:)` joins it. + private let teardownGroup = DispatchGroup() private var queue: LiveRecordingQueue! @@ -39,6 +44,13 @@ final class LiveRecordingWriter: LiveRecordingSink, @unchecked Sendable { private let onFailure: @Sendable (RecordingFailure) -> Void var bytesWritten: Int64 { ctxLock.lock(); defer { ctxLock.unlock() }; return _bytesWritten } + var isClosed: Bool { ctxLock.lock(); defer { ctxLock.unlock() }; return closed } + + #if DEBUG + /// Test-only: runs on the drain queue before each write, so a test can hold the drain and make a + /// teardown take as long as a slow disk would. + nonisolated(unsafe) var beforeWriteForTesting: (@Sendable () -> Void)? + #endif var droppedBytes: Int64 { queue.droppedBytes } init(url: URL, @@ -170,6 +182,9 @@ final class LiveRecordingWriter: LiveRecordingSink, @unchecked Sendable { // MARK: - Writer queue private func write(_ item: LiveRecordingQueue.QueuedPacket) { + #if DEBUG + beforeWriteForTesting?() + #endif ctxLock.lock() // Deliberately NOT gated on `closed`. `finish` stops ACCEPTING first and drains second, so // gating the write here would discard everything still queued at the moment of a stop, up @@ -218,8 +233,18 @@ final class LiveRecordingWriter: LiveRecordingSink, @unchecked Sendable { // MARK: - Teardown - func finish(reason: RecordingEndReason) { + /// Closes the recording and returns the failure it ended with, if any (audit FEA-105). A stop that + /// lands while a scheduled teardown is already draining used to see `closed`, return at once, and let + /// the caller publish `.ended` for a file that was still being written; it now joins that teardown + /// and hands back the failure the teardown recorded, which its own report can no longer deliver + /// because the recording has been released by then. + @discardableResult + func finish(reason: RecordingEndReason) -> RecordingFailure? { finish(reason: reason, failure: nil) + teardownGroup.wait() + ctxLock.lock() + defer { ctxLock.unlock() } + return pendingFailure } /// Tears the writer down from somewhere that must not block: the demux thread (a refused @@ -228,10 +253,14 @@ final class LiveRecordingWriter: LiveRecordingSink, @unchecked Sendable { ctxLock.lock() guard !closed, !teardownScheduled else { ctxLock.unlock(); return } teardownScheduled = true + pendingFailure = failure + teardownGroup.enter() ctxLock.unlock() - DispatchQueue.global(qos: .utility).async { [weak self] in - self?.finish(reason: .stoppedByHost, failure: failure) + // Strong self: the group is entered, and a group released while entered is a libdispatch trap. + DispatchQueue.global(qos: .utility).async { + self.finish(reason: .stoppedByHost, failure: failure) + self.teardownGroup.leave() } } diff --git a/Sources/AetherEngine/Renderer/SampleBufferRenderer.swift b/Sources/AetherEngine/Renderer/SampleBufferRenderer.swift index bdcab9e52..79d7e0267 100644 --- a/Sources/AetherEngine/Renderer/SampleBufferRenderer.swift +++ b/Sources/AetherEngine/Renderer/SampleBufferRenderer.swift @@ -5,8 +5,8 @@ import CoreVideo /// Video renderer using AVSampleBufferDisplayLayer for optimal frame pacing. /// -/// Includes a small reorder buffer (4 frames) to handle B-frame decode -/// order from VTDecompressionSession. Frames are sorted by PTS before +/// Includes a small reorder buffer to handle B-frame decode order from VTDecompressionSession +/// (4 frames, see `reorderDepth(forHardwareDecoder:)`). Frames are sorted by PTS before /// being enqueued to the display layer in strict presentation order. final class SampleBufferRenderer: @unchecked Sendable { @@ -27,10 +27,39 @@ final class SampleBufferRenderer: @unchecked Sendable { /// window renders only this layer, the host overlay cannot reach it). let subtitleCompositor = SubtitleFrameCompositor() - /// B-frame reorder buffer (4 frames): collects decoder output, flushes to display layer in ascending PTS order. Third tuple slot carries per-frame HDR10+ T.35 SEI bytes, paired through the reorder to kCMSampleAttachmentKey_HDR10PlusPerFrameData. + /// B-frame reorder buffer: collects decoder output, flushes to display layer in ascending PTS order. Third tuple slot carries per-frame HDR10+ T.35 SEI bytes, paired through the reorder to kCMSampleAttachmentKey_HDR10PlusPerFrameData. private let reorderLock = NSLock() private var reorderBuffer: [(CVPixelBuffer, CMTime, Data?)] = [] - private let reorderDepth = 4 // handles up to 3 consecutive B-frames + + /// Audit PERF-104: VideoToolbox HEVC with temporal processing emits B-frames out of presentation + /// order (19479fc5, -12080 from the layer), so its renderer holds 4 frames, enough for 3 + /// consecutive B-frames. libavcodec and dav1d emit presentation order, and 3 held frames are 75 MB + /// of IOSurface at 4K P010 for nothing. One frame stays held either way: #407 reads each frame's + /// duration off its held successor. + static let hardwareDecoderReorderDepth = 4 + static let presentationOrderReorderDepth = 1 + static func reorderDepth(forHardwareDecoder hardware: Bool) -> Int { + hardware ? hardwareDecoderReorderDepth : presentationOrderReorderDepth + } + + /// Guarded by `reorderLock`. Starts at the hardware depth, so a renderer nobody configured keeps the + /// behaviour it always had. + private var reorderDepth = SampleBufferRenderer.hardwareDecoderReorderDepth + + /// Set once by the host when it picks the decoder; a depth below the current one takes effect on + /// the next enqueue, which hands the surplus over in order. + func setReorderDepth(_ depth: Int) { + reorderLock.lock() + reorderDepth = max(1, min(depth, Self.hardwareDecoderReorderDepth)) + reorderLock.unlock() + } + + /// Frames waiting for a smaller presentation time. Diagnostics and tests. + var heldFrameCount: Int { + reorderLock.lock() + defer { reorderLock.unlock() } + return reorderBuffer.count + } /// Drop frames before this PTS after a seek (prevents keyframe-to-target fast-forward). Cleared after the first passing frame. private var skipUntilPTS: CMTime? @@ -96,9 +125,27 @@ final class SampleBufferRenderer: @unchecked Sendable { /// an even 24 fps timeline and for one carrying a doubled or a duplicate interval alike; only the /// spacing separates them. Guarded by `reorderLock`, reset by `takeCadence()`. private var _lastHandedPtsSeconds: Double? + /// Audit PERF-104: the newest timestamp that LEFT the reorder buffer, recorded under the same lock + /// that pops it. The late-frame guard reads this, not `_lastHandedPtsSeconds`, which is written + /// after the layer enqueue: a second enqueuer (a drain beside the decode thread) landing in that + /// gap would be judged against the frame before and slip through out of order. Reset by `flush`. + /// Guarded by `reorderLock`. + private var _lastReleasedPtsSeconds: Double? private var _minHandedDeltaSeconds = Double.infinity private var _maxHandedDeltaSeconds = -Double.infinity + /// Audit PERF-104: frames that arrived after a later one had already been handed over, so they can + /// no longer be enqueued in presentation order. Dropped rather than enqueued out of order, and the + /// first one raises the depth to the hardware decoder's for the rest of this renderer's life, which + /// covers an HEVC stream whose SPS understates its reorder and PTS derived by the demuxer. + /// Guarded by `reorderLock`. + private var _outOfOrderFramesDropped = 0 + var outOfOrderFramesDropped: Int { + reorderLock.lock() + defer { reorderLock.unlock() } + return _outOfOrderFramesDropped + } + /// #298: frames refused at the enqueue gate for carrying an unschedulable PTS. Guarded by `reorderLock`. private var _untimedFramesDropped = 0 var untimedFramesDropped: Int { @@ -196,7 +243,7 @@ final class SampleBufferRenderer: @unchecked Sendable { _minHandedDeltaSeconds = .infinity _maxHandedDeltaSeconds = -.infinity return Cadence(handedOver: enqueueCount, - lostBeforeLayer: _untimedFramesDropped + _sampleBuildFailures, + lostBeforeLayer: _untimedFramesDropped + _sampleBuildFailures + _outOfOrderFramesDropped, minDeltaSeconds: minD, maxDeltaSeconds: maxD) } @@ -314,6 +361,20 @@ final class SampleBufferRenderer: @unchecked Sendable { } let ptsSeconds = CMTimeGetSeconds(pts) + if let handed = _lastReleasedPtsSeconds, ptsSeconds < handed { + _outOfOrderFramesDropped += 1 + let dropped = _outOfOrderFramesDropped + let raised = reorderDepth < Self.hardwareDecoderReorderDepth + reorderDepth = Self.hardwareDecoderReorderDepth + reorderLock.unlock() + if dropped == 1 { + EngineLog.emit( + "[Renderer] frame at \(String(format: "%.3f", ptsSeconds))s arrived after \(String(format: "%.3f", handed))s " + + "was handed over; dropped, reorder depth \(raised ? "raised to \(Self.hardwareDecoderReorderDepth)" : "already \(Self.hardwareDecoderReorderDepth)")", + category: .swPlayback) + } + return + } // #303: the frontier is the newest timestamp HELD, not the newest handed over. A B-frame run // arrives out of order, so taking the last call's timestamp would report a cushion that // shrinks and grows with the coding pattern rather than with the buffer. @@ -327,6 +388,7 @@ final class SampleBufferRenderer: @unchecked Sendable { while reorderBuffer.count > reorderDepth { let (pb, t, hdr) = reorderBuffer.removeFirst() + _lastReleasedPtsSeconds = CMTimeGetSeconds(t) // #407: the successor is already held, so its timestamp is the frame's exact duration at // no extra latency. Read before the unlock, since enqueue() runs on the decode thread. let next = reorderBuffer.first?.1 @@ -347,6 +409,7 @@ final class SampleBufferRenderer: @unchecked Sendable { // #407: the next frame handed over will not follow the last one, so the gap between them is // not a cadence measurement. Left standing, every seek would report one enormous interval. _lastHandedPtsSeconds = nil + _lastReleasedPtsSeconds = nil // #303: nothing is held any more, so the frontier is not a frontier. Left standing, a // backward seek would keep reporting the pre-seek timestamp and read as a cushion of // however far the seek travelled. @@ -366,6 +429,7 @@ final class SampleBufferRenderer: @unchecked Sendable { reorderLock.lock() let remaining = reorderBuffer reorderBuffer.removeAll() + if let newest = remaining.last { _lastReleasedPtsSeconds = CMTimeGetSeconds(newest.1) } reorderLock.unlock() for (i, (pb, t, hdr)) in remaining.enumerated() { diff --git a/Sources/AetherEngine/Renderer/SubtitleFrameCompositor.swift b/Sources/AetherEngine/Renderer/SubtitleFrameCompositor.swift index 086c86f80..bdeeb7ae1 100644 --- a/Sources/AetherEngine/Renderer/SubtitleFrameCompositor.swift +++ b/Sources/AetherEngine/Renderer/SubtitleFrameCompositor.swift @@ -16,9 +16,13 @@ final class SubtitleFrameCompositor: @unchecked Sendable { let maxTextWidth: CGFloat } - /// Plain window check; cue times and SW frame PTS share the source axis. - nonisolated static func activeCues(in cues: [SubtitleCue], at seconds: Double) -> [SubtitleCue] { - cues.filter { $0.startTime <= seconds && seconds < $0.endTime } + /// Cue times and frame PTS share the source axis. Shift subtitle presentation only; + /// playback rate already advances the frame clock in media seconds. + nonisolated static func activeCues(in cues: [SubtitleCue], at seconds: Double, + delaySeconds: Double = 0) -> [SubtitleCue] { + let subtitleTime = seconds - delaySeconds + guard seconds.isFinite, delaySeconds.isFinite, subtitleTime.isFinite else { return [] } + return cues.filter { $0.startTime <= subtitleTime && subtitleTime < $0.endTime } } /// Default look: readable in a small window, resolution-independent. @@ -61,6 +65,7 @@ final class SubtitleFrameCompositor: @unchecked Sendable { private let lock = NSLock() private var cues: [SubtitleCue] = [] private var enabled = false + private var delaySeconds: Double = 0 /// Cache key of the overlay currently rendered (active cue ids); nil = no overlay cached. private var cachedCueIDs: [Int]? private var cachedOverlay: CIImage? @@ -71,10 +76,11 @@ final class SubtitleFrameCompositor: @unchecked Sendable { private var poolFormat: (width: Int, height: Int, pixelFormat: OSType)? /// Any thread; called by the engine when its published cues or the PiP flag change. - func update(cues: [SubtitleCue], enabled: Bool) { + func update(cues: [SubtitleCue], enabled: Bool, delaySeconds: Double) { lock.lock() self.cues = cues self.enabled = enabled + if delaySeconds.isFinite { self.delaySeconds = delaySeconds } lock.unlock() } @@ -83,10 +89,11 @@ final class SubtitleFrameCompositor: @unchecked Sendable { lock.lock() let enabled = self.enabled let cues = self.cues + let delay = self.delaySeconds lock.unlock() guard enabled else { return buffer } - let active = Self.activeCues(in: cues, at: ptsSeconds) + let active = Self.activeCues(in: cues, at: ptsSeconds, delaySeconds: delay) guard !active.isEmpty else { lock.lock(); cachedCueIDs = nil; cachedOverlay = nil; lock.unlock() return buffer @@ -116,12 +123,35 @@ final class SubtitleFrameCompositor: @unchecked Sendable { logFailureOnce("pool exhausted") return buffer } + // Audit DEC-3: the renderer builds the format description from the delivered buffer, so + // the output has to carry the source's pixel aspect ratio and colour tags, or anamorphic + // content shows at coded size and HDR drops to SDR for as long as a cue is up. Carried + // before the render too, because CoreImage picks the YCbCr matrix from the destination. + Self.carryAttachments(from: buffer, to: output) let base = CIImage(cvPixelBuffer: buffer) let composited = overlay.composited(over: base) - ciContext.render(composited, to: output, bounds: CGRect(x: 0, y: 0, width: width, height: height), colorSpace: CGColorSpace(name: CGColorSpace.itur_709)) + ciContext.render(composited, to: output, bounds: CGRect(x: 0, y: 0, width: width, height: height), + colorSpace: Self.renderColorSpace(for: buffer)) + Self.carryAttachments(from: buffer, to: output) return output } + /// The source's own colour space, so the render encodes the picture the way it was decoded + /// (PQ and HLG included). BT.709 only when the source carries no usable tags. + nonisolated static func renderColorSpace(for buffer: CVPixelBuffer) -> CGColorSpace { + if let attachments = CVBufferCopyAttachments(buffer, .shouldPropagate), + let space = CVImageBufferCreateColorSpaceFromAttachments(attachments)?.takeRetainedValue() { + return space + } + return CGColorSpace(name: CGColorSpace.itur_709) ?? CGColorSpaceCreateDeviceRGB() + } + + /// Replaces rather than adds: a recycled pool buffer can hold a previous source's tags. + nonisolated static func carryAttachments(from source: CVPixelBuffer, to output: CVPixelBuffer) { + CVBufferRemoveAllAttachments(output) + CVBufferPropagateAttachments(source, output) + } + /// One CGImage per cue-set change: text cues bottom-up in the default look, image cues at their /// canvas-mapped rects. CG coordinate origin is bottom-left; layout values are top-left based, /// so y flips via frameHeight. @@ -317,7 +347,11 @@ final class SubtitleFrameCompositor: @unchecked Sendable { } } + /// Audit DEC-5: `reset()` clears the pool under `lock` from the actor while the decode thread can + /// be in here, so the pool reference is only read and replaced under the same lock. private func dequeueBuffer(width: Int, height: Int, pixelFormat: OSType) -> CVPixelBuffer? { + lock.lock() + defer { lock.unlock() } if poolFormat?.width != width || poolFormat?.height != height || poolFormat?.pixelFormat != pixelFormat { let attrs: [CFString: Any] = [ kCVPixelBufferWidthKey: width, diff --git a/Sources/AetherEngine/SharedOutputCoordinator.swift b/Sources/AetherEngine/SharedOutputCoordinator.swift new file mode 100644 index 000000000..9c2c624c3 --- /dev/null +++ b/Sources/AetherEngine/SharedOutputCoordinator.swift @@ -0,0 +1,114 @@ +import Foundation + +/// The outputs every `AetherEngine` in a process shares, and who is still using them (Sodalite#175). +/// +/// The audio session, its channel preference and the panel's display criteria are process-wide, and +/// each engine used to act on them as if it were alone. Measured on an Apple TV with two live tiles: +/// every stop that released the session paused the OTHER engine within 10 ms, picture and all, 4 of 4. +/// Engines join here when a load starts and leave on a final teardown; the decisions below are the +/// ones that need to know whether anybody is left. +@MainActor +final class SharedOutputCoordinator { + static let shared = SharedOutputCoordinator() + + struct Member { + var role: SharedOutputRole + var tag: String? + var sourceChannels: Int? + /// Weak, so an engine released without a final `stop()` does not stay a member forever. + weak var owner: AnyObject? + let tracksOwner: Bool + } + + enum LeaveOutcome: Equatable, Sendable { + case notMember + case othersRemain(Int) + case lastOut(releaseSession: Bool) + } + + private var members: [ObjectIdentifier: Member] = [:] + /// Set by any leaver that asked for the session to be released, paid by the last one out. Without + /// it, the engine that opted in could leave first and the one that did not would close the round. + private var releaseOwed = false + /// Criteria writers that stopped while others still played. Keyed by engine, so one that loads + /// again as `.primary` drops its own reset instead of having it wipe the criteria it has just written. + private var deferredCriteriaResets: [ObjectIdentifier: @MainActor () -> Void] = [:] + private var lastTransition: Task? + + init() {} + + var isEmpty: Bool { + pruneReleasedOwners() + return members.isEmpty + } + + var preferredSourceChannels: Int? { + pruneReleasedOwners() + return members.values.compactMap(\.sourceChannels).max() + } + + /// Drops members whose engine was deallocated without a `stop()`. Leaves the deferred resets and + /// the owed release alone: the next `leave` or read decides. + private func pruneReleasedOwners() { + let gone = members.filter { $0.value.tracksOwner && $0.value.owner == nil } + for (id, member) in gone { + members[id] = nil + deferredCriteriaResets[id] = nil + EngineLog.emit("[SharedOutput] \(member.tag ?? "engine-\(UInt(bitPattern: id.hashValue) % 10_000)") was released without stop(), dropped", category: .engine) + } + } + + func join(_ id: ObjectIdentifier, role: SharedOutputRole, tag: String?, owner: AnyObject? = nil) { + let channels = members[id]?.sourceChannels + members[id] = Member(role: role, tag: tag, sourceChannels: channels, owner: owner, tracksOwner: owner != nil) + // A secondary never writes criteria the pending reset could wipe, so it keeps the reset owed. + if role == .primary { deferredCriteriaResets[id] = nil } + } + + func noteSourceChannels(_ channels: Int?, for id: ObjectIdentifier) { + guard members[id] != nil else { return } + members[id]?.sourceChannels = channels + } + + func leave(_ id: ObjectIdentifier, releasesSession: Bool) -> LeaveOutcome { + pruneReleasedOwners() + guard members.removeValue(forKey: id) != nil else { return .notMember } + if releasesSession { releaseOwed = true } + guard members.isEmpty else { return .othersRemain(members.count) } + let release = releaseOwed + releaseOwed = false + let resets = deferredCriteriaResets.values + deferredCriteriaResets.removeAll() + resets.forEach { $0() } + return .lastOut(releaseSession: release) + } + + func othersActive(besides id: ObjectIdentifier) -> Bool { + pruneReleasedOwners() + return members.keys.contains { $0 != id } + } + + /// Defers a criteria reset to run when the last engine leaves, unless nobody else is playing now. + /// If `id` is alone or not a member, the reset runs immediately. + func deferCriteriaReset(for id: ObjectIdentifier, _ body: @escaping @MainActor () -> Void) { + guard othersActive(besides: id) else { body(); return } + deferredCriteriaResets[id] = body + } + + /// One queue for every engine's activation and release. AE#538 ordered them per engine; with two + /// engines the release of one and the activation of the other had no order between them. + func enqueueTransition(_ body: @escaping @Sendable () async -> Void) -> Task { + let previous = lastTransition + let transition = Task.detached(priority: .userInitiated) { + await previous?.value + await body() + } + lastTransition = transition + return transition + } + + /// Log label for an engine: its host-given tag, or the identifier's hash when there is none. + func label(for id: ObjectIdentifier) -> String { + members[id]?.tag ?? "engine-\(UInt(bitPattern: id.hashValue) % 10_000)" + } +} diff --git a/Sources/AetherEngine/SharedOutputRole.swift b/Sources/AetherEngine/SharedOutputRole.swift new file mode 100644 index 000000000..e7e83cfdc --- /dev/null +++ b/Sources/AetherEngine/SharedOutputRole.swift @@ -0,0 +1,9 @@ +/// Which of the process-wide outputs a load may drive (Sodalite#175). One engine is `.primary`; an +/// engine running beside it, such as a multiview tile, is `.secondary`. See `SharedOutputCoordinator`. +public enum SharedOutputRole: String, Sendable, Equatable { + /// Drives the panel (display criteria) and may own Now Playing. The default, and the only role a + /// host with one engine ever needs. + case primary + /// Never writes display criteria and never owns Now Playing, whatever the host set for either. + case secondary +} diff --git a/Sources/AetherEngine/Subtitles/ASSScriptBuilder.swift b/Sources/AetherEngine/Subtitles/ASSScriptBuilder.swift index b82d7cf1e..c364fef02 100644 --- a/Sources/AetherEngine/Subtitles/ASSScriptBuilder.swift +++ b/Sources/AetherEngine/Subtitles/ASSScriptBuilder.swift @@ -70,10 +70,10 @@ public final class ASSScriptBuilder { seen.removeAll(keepingCapacity: true) } - /// ASS timestamp `H:MM:SS.cc` (centiseconds). Negative input - /// clamps to zero. + /// ASS timestamp `H:MM:SS.cc` (centiseconds). Negative input and NaN clamp to zero, and a + /// time past 4e9 s (infinity included) pins there. public static func timestamp(_ seconds: Double) -> String { - let total = max(0, seconds) + let total = max(0, SourceTimestampBounds.clampedSeconds(seconds)) var centis = Int((total * 100).rounded()) let h = centis / 360_000 centis %= 360_000 diff --git a/Sources/AetherEngine/Subtitles/MovTextSampleBuilder.swift b/Sources/AetherEngine/Subtitles/MovTextSampleBuilder.swift index 0b1000c7b..fe73bae26 100644 --- a/Sources/AetherEngine/Subtitles/MovTextSampleBuilder.swift +++ b/Sources/AetherEngine/Subtitles/MovTextSampleBuilder.swift @@ -5,11 +5,12 @@ import Foundation enum MovTextSampleBuilder { /// Strip ASS/SSA override blocks (`{\...}`) and normalize inline escapes to plain text. + /// + /// Audit SUB-1: one forward pass. Removing each block in place shifted the whole remaining cue + /// per block, quadratic on a remote cue made of many small blocks (karaoke tracks emit one per + /// syllable). An unclosed `{` keeps itself and everything after it, as before. static func sanitize(_ assText: String) -> String { - var s = assText - while let open = s.firstIndex(of: "{"), let close = s[open...].firstIndex(of: "}") { - s.removeSubrange(open...close) - } + var s = SubtitleRectText.strippingOverrideBlocks(assText) s = s.replacingOccurrences(of: "\\N", with: "\n") s = s.replacingOccurrences(of: "\\n", with: "\n") s = s.replacingOccurrences(of: "\\h", with: " ") diff --git a/Sources/AetherEngine/Subtitles/NativeSubtitleCueStore.swift b/Sources/AetherEngine/Subtitles/NativeSubtitleCueStore.swift index 8531b9fcf..a58e1dbed 100644 --- a/Sources/AetherEngine/Subtitles/NativeSubtitleCueStore.swift +++ b/Sources/AetherEngine/Subtitles/NativeSubtitleCueStore.swift @@ -31,7 +31,8 @@ final class NativeSubtitleCueStore: @unchecked Sendable { init() {} private static func key(start: Double, end: Double, text: String) -> String { - "\(Int((start * 1000).rounded()))|\(Int((end * 1000).rounded()))|\(text)" + let start = SourceTimestampBounds.clampedSeconds(start), end = SourceTimestampBounds.clampedSeconds(end) + return "\(Int((start * 1000).rounded()))|\(Int((end * 1000).rounded()))|\(text)" } func setShiftSeconds(_ s: Double) { lock.lock(); shiftSeconds = s; lock.unlock() } diff --git a/Sources/AetherEngine/Subtitles/SubtitlePacketStore.swift b/Sources/AetherEngine/Subtitles/SubtitlePacketStore.swift index e16602cd2..379cb5921 100644 --- a/Sources/AetherEngine/Subtitles/SubtitlePacketStore.swift +++ b/Sources/AetherEngine/Subtitles/SubtitlePacketStore.swift @@ -58,7 +58,9 @@ final class SubtitlePacketStore: @unchecked Sendable { /// Ceiling for one in-assembly PGS display set (a 4K set stays far below this); a pending /// buffer past it is malformed or mis-parsed and gets dropped rather than grown unbounded. - static let maxPendingDisplaySetBytes: Int = 16 * 1024 * 1024 + /// Audit SUB-111: pending sets also count toward the aggregate cap, so 16 MiB per set was + /// room for 16 MiB x 2 writers x every PID of a crafted transport stream. + static let maxPendingDisplaySetBytes: Int = 4 * 1024 * 1024 /// #151: which reader is writing. The pump and the forward prefetcher can both feed the same /// stream; a completed entry re-harvested by the other collapses on a byte-identical payload in @@ -86,14 +88,22 @@ final class SubtitlePacketStore: @unchecked Sendable { private var entriesByStream: [Int32: [StoredSubtitlePacket]] = [:] private var bytesByStream: [Int32: Int] = [:] private var pendingSetByStream: [PendingKey: PendingDisplaySet] = [:] + /// Audit SUB-103: same-PTS runs longer than `linearDedupeRun` answer "was this payload already + /// stored here" from a hash set instead of comparing against every packet of the run. The set is + /// only a filter: it may hold hashes of evicted packets (a stale hit costs one scan) but never + /// lacks one of a live packet, since every append to the run inserts its hash. + private var payloadHashesByRun: [Int32: [UInt64: Set]] = [:] + private static let linearDedupeRun = 8 + private static let maxCachedRunsPerStream = 4096 /// Instance caps (default to the static ceilings). Injectable so tests can drive eviction with /// tiny payloads instead of allocating gigabytes. private let perStreamCap: Int private let aggregateCap: Int - /// #166 aggregate-budget bookkeeping. `totalBytes` mirrors the sum of `bytesByStream` (kept - /// incrementally so the per-append check is O(1)). `protectedStreams` are the active drain + /// #166 aggregate-budget bookkeeping. `totalBytes` mirrors the sum of `bytesByStream` plus the + /// display sets still in assembly (audit SUB-111), kept incrementally so the per-append check is + /// O(1). `protectedStreams` are the active drain /// targets, never evicted by aggregate pressure. `lastTouchByStream` orders non-protected /// streams coldest-first for eviction; a monotonic counter (no wall clock) drives it. private var totalBytes: Int = 0 @@ -179,7 +189,10 @@ final class SubtitlePacketStore: @unchecked Sendable { private func appendLocked(streamIndex: Int32, ptsSeconds: Double, durationSeconds: Double, flags: Int32, payload: Data, webvttSettings: String? = nil) { let before = bytesByStream[streamIndex] ?? 0 - var entries = entriesByStream[streamIndex] ?? [] + // Audit SUB-103: taken out of the dictionary before it is mutated. Left in, the dictionary + // holds the second reference and the insert below copied every retained packet per append, + // 10k appends took 0.65 s and 40k took 10.9 s. + var entries = entriesByStream.removeValue(forKey: streamIndex) ?? [] var bytes = before // #362: a re-harvest of a packet already stored takes the FRESH sequence. It is a new // observation of that position by the run doing the writing, and that is exactly what the @@ -198,24 +211,26 @@ final class SubtitlePacketStore: @unchecked Sendable { // prefetcher re-harvesting the same packet (#151), and only that collapses. Anything else // joins the end of the run, so a shared timestamp reaches the drainer in harvest order: // the drainer decodes a window in array order and later events layer over earlier ones. - var probe = Self.lowerBound(entries, ptsSeconds) - var duplicateIndex: Int? - while probe < entries.count, entries[probe].ptsSeconds == ptsSeconds { - if entries[probe].payload == payload { - duplicateIndex = probe - break - } - probe += 1 - } - if let duplicateIndex { + let run = Self.lowerBound(entries, ptsSeconds).. perStreamCap, entries.count > 1 { - bytes -= entries.removeFirst().payload.count + if bytes > perStreamCap { + // Audit SUB-103: evict to a low-water mark in one move. Dropping just enough made every + // append at the cap shift the whole array for one packet. + let lowWater = perStreamCap - perStreamCap / 10 + var dropped = 0 + while bytes > lowWater, dropped < entries.count - 1 { + bytes -= entries[dropped].payload.count + dropped += 1 + } + entries.removeFirst(dropped) + payloadHashesByRun[streamIndex] = nil } entriesByStream[streamIndex] = entries bytesByStream[streamIndex] = bytes @@ -225,6 +240,25 @@ final class SubtitlePacketStore: @unchecked Sendable { enforceAggregateCapLocked(justTouched: streamIndex) } + /// Index within `run` of the packet `payload` duplicates, nil when it is new. A long run is + /// screened through `payloadHashesByRun` first; see there. + private func duplicateIndexLocked(in entries: [StoredSubtitlePacket], run: Range, payload: Data, + streamIndex: Int32, ptsSeconds: Double) -> Int? { + guard !run.isEmpty else { return nil } + func scan() -> Int? { + run.first { entries[$0].payload.count == payload.count && entries[$0].payload == payload } + } + guard run.count > Self.linearDedupeRun else { return scan() } + let key = (ptsSeconds + 0.0).bitPattern + var runs = payloadHashesByRun.removeValue(forKey: streamIndex) ?? [:] + var hashes = runs.removeValue(forKey: key) ?? Set(run.map { entries[$0].payload.hashValue }) + let seenBefore = !hashes.insert(payload.hashValue).inserted + if runs.count >= Self.maxCachedRunsPerStream { runs.removeAll() } + runs[key] = hashes + payloadHashesByRun[streamIndex] = runs + return seenBefore ? scan() : nil + } + /// First index at or past `ptsSeconds` in a PTS-sorted run. Harvest is near-monotonic, but the /// forward prefetcher (#151) backfills far behind the frontier, so the position is searched /// rather than assumed. Searched in log time rather than scanned from the front: the scan made @@ -245,6 +279,21 @@ final class SubtitlePacketStore: @unchecked Sendable { return low } + /// First index whose PTS is greater than `ptsSeconds`; `lowerBound` and this bracket a same-PTS run. + static func upperBound(_ entries: [StoredSubtitlePacket], _ ptsSeconds: Double) -> Int { + var low = 0 + var high = entries.count + while low < high { + let mid = low + (high - low) / 2 + if entries[mid].ptsSeconds <= ptsSeconds { + low = mid + 1 + } else { + high = mid + } + } + return low + } + /// #166: bound retained bytes across ALL streams. Evict oldest entries from the coldest /// (least-recently-touched) NON-protected stream first, then the next coldest, until the total /// is back under `aggregateCap` or only protected streams remain. Protected streams (the active @@ -252,20 +301,35 @@ final class SubtitlePacketStore: @unchecked Sendable { /// cold stream is dropped and re-harvested from the pump/prefetcher if it is selected later. private func enforceAggregateCapLocked(justTouched: Int32) { guard totalBytes > aggregateCap else { return } - let candidates = bytesByStream.keys + // Audit SUB-103: down to a low-water mark, so the pass does not rerun on the next append, and + // each victim loses its leading packets in one `removeFirst(k)`. + let lowWater = aggregateCap - aggregateCap / 10 + var streams = Set(bytesByStream.keys) + for key in pendingSetByStream.keys { streams.insert(key.streamIndex) } + let candidates = streams .filter { !protectedStreams.contains($0) && $0 != justTouched } .sorted { (lastTouchByStream[$0] ?? 0) < (lastTouchByStream[$1] ?? 0) } for idx in candidates { - guard totalBytes > aggregateCap else { break } - guard var entries = entriesByStream[idx] else { continue } + guard totalBytes > lowWater else { break } + // Audit SUB-111: an unfinished display set of a cold stream goes before anything complete. + for writer in [Writer.pump, Writer.prefetch] { + _ = takePendingLocked(PendingKey(streamIndex: idx, writer: writer)) + } + guard totalBytes > lowWater, var entries = entriesByStream.removeValue(forKey: idx) else { + if entriesByStream[idx] == nil { lastTouchByStream[idx] = nil } + continue + } var bytes = bytesByStream[idx] ?? 0 - while totalBytes > aggregateCap, !entries.isEmpty { - let removed = entries.removeFirst().payload.count + var dropped = 0 + while totalBytes > lowWater, dropped < entries.count { + let removed = entries[dropped].payload.count bytes -= removed totalBytes -= removed + dropped += 1 } + entries.removeFirst(dropped) + payloadHashesByRun[idx] = nil if entries.isEmpty { - entriesByStream[idx] = nil bytesByStream[idx] = nil lastTouchByStream[idx] = nil } else { @@ -275,6 +339,21 @@ final class SubtitlePacketStore: @unchecked Sendable { } } + /// A display set in assembly leaves the dictionary while it is worked on, so appending to it is in + /// place, and its bytes leave the aggregate count with it. `putPendingLocked` is the way back. + private func takePendingLocked(_ key: PendingKey) -> PendingDisplaySet? { + guard let pending = pendingSetByStream.removeValue(forKey: key) else { return nil } + totalBytes -= pending.payload.count + return pending + } + + private func putPendingLocked(_ pending: PendingDisplaySet, _ key: PendingKey) { + pendingSetByStream[key] = pending + totalBytes += pending.payload.count + touchCounter &+= 1 + lastTouchByStream[key.streamIndex] = touchCounter + } + /// Shared pump-side harvest for both hosts: convert a raw AVPacket into a stored entry on /// the source PTS axis (raw pts x time_base, matching what EmbeddedSubtitleDecoder computes /// for tap packets; no start_time subtraction) and append it. Copies synchronously; the @@ -305,6 +384,12 @@ final class SubtitlePacketStore: @unchecked Sendable { func harvestChunk(streamIndex: Int32, ptsSeconds: Double?, durationSeconds: Double, flags: Int32, payload: Data, assembleSplitDisplaySets: Bool, writer: Writer = .pump, webvttSettings: String? = nil) { + // Audit FEA-101: the pump, the software-host tap and the prefetcher all store through here, + // and the drain scales these seconds back into Int64 milliseconds. Same rule as the + // demuxer's bound: an implausible time is unset, an implausible duration is 0. + let bound = SourceTimestampBounds.maxPlausibleSeconds + let ptsSeconds = ptsSeconds.flatMap { $0.isFinite && abs($0) < bound ? $0 : nil } + let durationSeconds = durationSeconds.isFinite && durationSeconds < bound ? Swift.max(0, durationSeconds) : 0 lock.lock(); defer { lock.unlock() } // #416: a harvested packet is a position its writer demonstrably read, so the run reaches // it. This is what gives the pump the forward lookahead its playhead-based note cannot @@ -326,7 +411,7 @@ final class SubtitlePacketStore: @unchecked Sendable { } let key = PendingKey(streamIndex: streamIndex, writer: writer) while !chunk.isEmpty { - var pending = pendingSetByStream[key] + var pending = takePendingLocked(key) // A backward pts jump under an open set means the pump re-anchored mid-set; // the stale partial buffer must not swallow the fresh set's segments. if let pts = ptsSeconds, let open = pending, pts < open.ptsSeconds - 1.0 { @@ -338,7 +423,6 @@ final class SubtitlePacketStore: @unchecked Sendable { // restart overlap above) is undecodable on its own and gets dropped. pending = nil guard let pts = ptsSeconds else { - pendingSetByStream[key] = nil return // No anchor for this set; skip its chunks until the next PCS. } pending = PendingDisplaySet(ptsSeconds: pts, durationSeconds: durationSeconds, @@ -346,9 +430,9 @@ final class SubtitlePacketStore: @unchecked Sendable { } guard var open = pending else { // Mid-set start (backfill landed between PCS and END): not decodable, drop. - pendingSetByStream[key] = nil return } + pending = nil let endBoundary = Self.pgsEndBoundary(in: chunk) let consumed: Data if let endBoundary { @@ -361,16 +445,15 @@ final class SubtitlePacketStore: @unchecked Sendable { open.payload.append(consumed) open.flags |= flags if open.payload.count > Self.maxPendingDisplaySetBytes { - pendingSetByStream[key] = nil return } if endBoundary != nil { appendLocked(streamIndex: streamIndex, ptsSeconds: open.ptsSeconds, durationSeconds: open.durationSeconds, flags: open.flags, payload: open.payload) - pendingSetByStream[key] = nil } else { - pendingSetByStream[key] = open + putPendingLocked(open, key) + enforceAggregateCapLocked(justTouched: streamIndex) } } } @@ -403,7 +486,8 @@ final class SubtitlePacketStore: @unchecked Sendable { func entries(streamIndex: Int32, from: Double, through: Double) -> [StoredSubtitlePacket] { lock.lock(); defer { lock.unlock() } guard let entries = entriesByStream[streamIndex] else { return [] } - return entries.filter { $0.ptsSeconds >= from && $0.ptsSeconds <= through } + // The run is PTS-sorted, so the window is found by search instead of a scan of the session. + return Array(entries[Self.lowerBound(entries, from)...].prefix { $0.ptsSeconds <= through }) } /// #362: PTS of the first stored packet on `streamIndex` strictly after `ptsSeconds`. @@ -416,8 +500,7 @@ final class SubtitlePacketStore: @unchecked Sendable { func firstPTS(streamIndex: Int32, after ptsSeconds: Double) -> Double? { lock.lock(); defer { lock.unlock() } guard let entries = entriesByStream[streamIndex] else { return nil } - var index = Self.lowerBound(entries, ptsSeconds) - while index < entries.count, entries[index].ptsSeconds <= ptsSeconds { index += 1 } + let index = Self.upperBound(entries, ptsSeconds) guard index < entries.count else { return nil } // Deliberately answered even across a harvest hole, where the next stored packet belongs to // an older run and the real successor is still on its way. The answer is then too late, but @@ -451,6 +534,7 @@ final class SubtitlePacketStore: @unchecked Sendable { totalBytes += newBytes - (bytesByStream[idx] ?? 0) entriesByStream[idx] = Array(kept) bytesByStream[idx] = newBytes + payloadHashesByRun[idx] = nil } } } @@ -460,6 +544,7 @@ final class SubtitlePacketStore: @unchecked Sendable { entriesByStream.removeAll() bytesByStream.removeAll() pendingSetByStream.removeAll() + payloadHashesByRun.removeAll() lastTouchByStream.removeAll() protectedStreams.removeAll() totalBytes = 0 diff --git a/Sources/AetherEngine/Subtitles/SubtitleRectText.swift b/Sources/AetherEngine/Subtitles/SubtitleRectText.swift index 6147be571..13cb60bed 100644 --- a/Sources/AetherEngine/Subtitles/SubtitleRectText.swift +++ b/Sources/AetherEngine/Subtitles/SubtitleRectText.swift @@ -14,25 +14,49 @@ extension SubtitleTextRun { /// Plain-text extraction from FFmpeg subtitle rects, shared by `SubtitleDecoder` (sidecar) and `EmbeddedSubtitleDecoder` (in-container) so ASS parsing fixes live in one place. enum SubtitleRectText { + /// Audit SUB-101: the most text of one cue any pass below looks at. A real cue is a few hundred + /// bytes; a hostile track can hand over megabytes, and every per-cue pass here runs on the demux + /// pump thread under the subtitle tap lock. + static let maxCueBytes = 64 * 1024 + + /// `text` cut to `maxCueBytes` at a scalar boundary; the same string, untouched, when it fits. + static func capped(_ text: String) -> String { + guard text.utf8.count > maxCueBytes else { return text } + var end = text.unicodeScalars.startIndex + var used = 0 + for scalar in text.unicodeScalars { + let next = used + UTF8.width(scalar) + if next > maxCueBytes { break } + used = next + end = text.unicodeScalars.index(after: end) + } + return String(text[..) -> String? { if let textPtr = rect.pointee.text { - let s = String(cString: textPtr) + let s = boundedString(cString: textPtr) let trimmed = s.trimmingCharacters(in: .whitespacesAndNewlines) if !trimmed.isEmpty { return trimmed } } if let assPtr = rect.pointee.ass { - return plainText(fromASSEventLine: String(cString: assPtr)) + return plainText(fromASSEventLine: boundedString(cString: assPtr)) } return nil } + private static func boundedString(cString pointer: UnsafePointer) -> String { + let length = strnlen(pointer, maxCueBytes) + return String(decoding: UnsafeRawBufferPointer(start: pointer, count: length), as: UTF8.self) + } + /// Plain text from a raw ASS event line (`ReadOrder,Layer,Style,...,Text`), for surfaces that need /// plain text out of markup-preserving cues (the WebVTT rendition over tap-harvested stores, /// Sodalite#32). Guarded on the first field being the integer ReadOrder so a plain, comma-heavy /// line is never misparsed as an event; non-event lines just get tag/escape cleaning. static func plainText(fromASSEventLine line: String) -> String? { - var l = line + var l = capped(line) if l.hasPrefix("Dialogue: ") { l.removeFirst("Dialogue: ".count) } @@ -54,19 +78,44 @@ enum SubtitleRectText { /// Strip ASS escapes (`\\N` newline, `\\h` hard space) and /// `{...}` override tags; nil when nothing displayable remains. static func cleanASSBody(_ raw: String) -> String? { - var s = raw + var s = capped(raw) s = s.replacingOccurrences(of: "\\N", with: "\n") s = s.replacingOccurrences(of: "\\n", with: "\n") s = s.replacingOccurrences(of: "\\h", with: " ") - s = s.replacingOccurrences( - of: "\\{[^}]*\\}", - with: "", - options: .regularExpression - ) + s = strippingOverrideBlocks(s) let trimmed = s.trimmingCharacters(in: .whitespacesAndNewlines) return trimmed.isEmpty ? nil : trimmed } + /// Drop every `{...}` span: from a `{` to the first `}` after it, so `{a{b}c}` leaves `c}`. A `{` + /// with no `}` behind it keeps itself and the rest verbatim. + /// + /// Audit SUB-101: this was `\\{[^}]*\\}` through `NSRegularExpression`, which rescans to the end + /// of the text at every unclosed `{` (20k braces took 2.6 s, doubling cost 4x). One forward pass + /// instead: once a `{` finds no `}`, no later one can either. Scanned as UTF-8, so the braces + /// match exactly where the regex's UTF-16 units did. + static func strippingOverrideBlocks(_ text: String) -> String { + guard text.utf8.contains(0x7B) else { return text } + let bytes = Array(text.utf8) + var out: [UInt8] = [] + out.reserveCapacity(bytes.count) + var index = 0 + while index < bytes.count { + let byte = bytes[index] + guard byte == 0x7B else { + out.append(byte) + index += 1 + continue + } + guard let close = bytes[(index + 1)...].firstIndex(of: 0x7D) else { + out.append(contentsOf: bytes[index...]) + break + } + index = close + 1 + } + return String(decoding: out, as: UTF8.self) + } + /// Default ASS play resolution (`ASS_DEFAULT_PLAYRESX/Y`, libavcodec/ass.h). Every event line /// libavcodec synthesises for SRT, WebVTT and teletext positions against this space, so it is /// the right frame of reference unless a real ASS header declares its own. @@ -88,7 +137,9 @@ enum SubtitleRectText { } return nil } - guard let x = value("PlayResX"), let y = value("PlayResY"), x > 0, y > 0 else { return nil } + // `Double(_:)` accepts `inf` and `nan` (audit SUB-112). + guard let x = value("PlayResX"), let y = value("PlayResY"), x > 0, y > 0, + x.isFinite, y.isFinite else { return nil } return CGSize(width: x, height: y) } @@ -128,7 +179,7 @@ enum SubtitleRectText { static func styledRuns(fromASSEventLine line: String, playRes: CGSize = SubtitleRectText.defaultASSPlayRes) -> (runs: [SubtitleTextRun], placement: SubtitleTextPlacement?, firstTextRow: Int)? { - var body = line + var body = capped(line) if body.hasPrefix("Dialogue: ") { body.removeFirst("Dialogue: ".count) } let parts = body.split(separator: ",", maxSplits: 8, omittingEmptySubsequences: false) let text: String = (parts.count == 9 && Int(parts[0]) != nil) ? String(parts[8]) : body @@ -248,14 +299,18 @@ enum SubtitleRectText { Double($0.trimmingCharacters(in: .whitespaces)) } guard parts.count == 2, let x = parts[0], let y = parts[1] else { return nil } - return CGPoint(x: x / playRes.width, y: y / playRes.height) + // Audit SUB-112: a NaN or infinite tag (`Double(_:)` accepts both) breaks the [0, 1] + // contract, and a host laying out with it raises; the cue falls back to its alignment. + let normalized = CGPoint(x: x / playRes.width, y: y / playRes.height) + guard normalized.x.isFinite, normalized.y.isFinite else { return nil } + return normalized } /// Trim whitespace and newlines across the edges of a run sequence, so a styled cue matches /// what the plain path produces. libzvbi teletext ass can prefix a row-positioning newline that /// would otherwise render as a blank line ONLY on styled cues (#107). Interior blank lines are /// NOT folded here: that is teletext-specific and lives in `teletextBody`. - private static func edgeTrimmed(_ runs: [SubtitleTextRun]) -> [SubtitleTextRun]? { + static func edgeTrimmed(_ runs: [SubtitleTextRun]) -> [SubtitleTextRun]? { var cleaned = runs.filter { !$0.text.isEmpty } // Edge-trim leading/trailing whitespace and newlines across the run sequence so a coloured // cue matches the plain path (teletextBody flattens + trims the .text case). libzvbi @@ -265,11 +320,11 @@ enum SubtitleRectText { // Predicate matches the plain path's `.whitespacesAndNewlines` (Unicode Zs plus tab plus // the newline characters). A literal-space/tab/newline test let U+00A0 survive on a styled // cue that an unstyled one trimmed, so the two paths disagreed on the same payload. - while let first = cleaned.first { - let d = String(first.text.drop(while: \.isWhitespace)) - if d.isEmpty { cleaned.removeFirst(); continue } - cleaned[0] = first.withText(d) - break + // Audit SUB-110: one `removeFirst(k)`, not one shift of the whole array per blank run. + let leadingBlank = cleaned.firstIndex { !$0.text.allSatisfy(\.isWhitespace) } ?? cleaned.count + cleaned.removeFirst(leadingBlank) + if let first = cleaned.first { + cleaned[0] = first.withText(String(first.text.drop(while: \.isWhitespace))) } while let last = cleaned.last { var s = last.text diff --git a/Sources/AetherEngine/Subtitles/WebVTTBuilder.swift b/Sources/AetherEngine/Subtitles/WebVTTBuilder.swift index fa106b36f..af4938802 100644 --- a/Sources/AetherEngine/Subtitles/WebVTTBuilder.swift +++ b/Sources/AetherEngine/Subtitles/WebVTTBuilder.swift @@ -41,7 +41,7 @@ enum WebVTTBuilder { } private static func timestamp(_ seconds: Double) -> String { - let total = max(0, seconds) + let total = max(0, SourceTimestampBounds.clampedSeconds(seconds)) let whole = Int(total) let h = whole / 3600 let m = (whole % 3600) / 60 diff --git a/Sources/AetherEngine/Subtitles/WebVTTSegmentParser.swift b/Sources/AetherEngine/Subtitles/WebVTTSegmentParser.swift index 0ee8e61a2..d98c44b2d 100644 --- a/Sources/AetherEngine/Subtitles/WebVTTSegmentParser.swift +++ b/Sources/AetherEngine/Subtitles/WebVTTSegmentParser.swift @@ -29,6 +29,9 @@ enum WebVTTSegmentParser { /// are cut on frame times, so the clipped halves rarely meet exactly. private static let joinTolerance = 0.25 + /// Audit SUB-106: a real broadcaster segment carries 0 to 5 cues. Cues past this count are not read. + static let maxCuesPerSegment = 4096 + static func parse(_ text: String) -> WebVTTSegment? { let lines = text.replacingOccurrences(of: "\r\n", with: "\n").components(separatedBy: "\n") guard lines.first(where: { !$0.trimmingCharacters(in: .whitespaces).isEmpty })? @@ -61,6 +64,7 @@ enum WebVTTSegmentParser { let joined = body.joined(separator: "\n").trimmingCharacters(in: .whitespacesAndNewlines) if !joined.isEmpty { cues.append(WebVTTSegment.Cue(start: start, end: end, text: joined)) + if cues.count >= maxCuesPerSegment { break } } } return WebVTTSegment(anchorLocalSeconds: anchor.local, anchorMPEGTS90k: anchor.mpegts, cues: cues) @@ -101,10 +105,17 @@ enum WebVTTSegmentParser { static func merged(into existing: [SubtitleCue], adding: [SubtitleCue], nextID: inout Int) -> [SubtitleCue] { var result = existing + // Audit SUB-106: only a cue with identical text can join, so the candidates come from a + // per-text index instead of a scan of everything published (40k cues took 3.9 s). The + // indices stay ascending, so the first match is the one the scan found. + var indicesByText: [String: [Int]] = [:] + for (index, candidate) in result.enumerated() { + if case .text(let text) = candidate.body { indicesByText[text, default: []].append(index) } + } for cue in adding { guard case .text(let text) = cue.body else { continue } - let match = result.firstIndex { candidate in - guard case .text(let candidateText) = candidate.body, candidateText == text else { return false } + let match = indicesByText[text]?.first { index in + let candidate = result[index] return cue.startTime <= candidate.endTime + joinTolerance && cue.endTime >= candidate.startTime - joinTolerance } @@ -116,6 +127,7 @@ enum WebVTTSegmentParser { body: old.body, placement: old.placement) } else { + indicesByText[text, default: []].append(result.count) result.append(cue) } } @@ -142,14 +154,17 @@ enum WebVTTSegmentParser { } /// `HH:MM:SS.mmm` or `MM:SS.mmm`; hours are unbounded (providers run the LOCAL clock for days). + /// Audit SUB-106: `Double(_:)` accepts `inf`, `nan` and `1e999`, and a cue ending at `+inf` was + /// never pruned, so a value that is not a finite non-negative number is refused. private static func parseTimestamp(_ text: String) -> Double? { let parts = text.components(separatedBy: ":") guard parts.count == 2 || parts.count == 3 else { return nil } let values = parts.compactMap { Double($0.replacingOccurrences(of: ",", with: ".")) } - guard values.count == parts.count else { return nil } - return parts.count == 3 + guard values.count == parts.count, values.allSatisfy({ $0.isFinite && $0.sign == .plus }) else { return nil } + let seconds = parts.count == 3 ? values[0] * 3600 + values[1] * 60 + values[2] : values[0] * 60 + values[1] + return seconds.isFinite ? seconds : nil } diff --git a/Sources/AetherEngine/Video/AnnexBSampleConverter.swift b/Sources/AetherEngine/Video/AnnexBSampleConverter.swift new file mode 100644 index 000000000..b78667459 --- /dev/null +++ b/Sources/AetherEngine/Video/AnnexBSampleConverter.swift @@ -0,0 +1,88 @@ +import Foundation +import AetherLibavcodec +import AetherLibavutil + +/// Rewrites an Annex-B video sample to 4-byte length-prefixed NALs, keeping every NAL. +/// +/// movenc does this conversion itself whenever the extradata is Annex B, but for an `hvc1` sample +/// entry it runs `ff_hevc_annexb2mp4` with `filter_ps` set and drops every in-band VPS, SPS and PPS. +/// The decoder is then left with the parameter sets in `hvcC` alone. A Blu-ray or broadcast HEVC +/// stream that sends a new PPS mid-title (seen at 118 s and 345.7 s on one UHD disc) has every later +/// slice decoded against the stale one: macOS reports "Cannot Decode", a device plays sound over a +/// black picture. Converting here, and handing movenc a length-prefixed record so it does not convert +/// again, keeps the parameter sets in the samples. That is the shape a Matroska remux of the same +/// stream already reaches the muxer in (its samples carry their in-band parameter sets and movenc +/// copies them as they are). +/// +/// Diagnosed and first implemented by yipengfei329 for MovieClaw (AetherEngine PR #703, patch P38). +enum AnnexBSampleConverter { + /// The NAL payload ranges of an Annex-B buffer, 3-byte and 4-byte start codes alike, with the + /// zero bytes in front of the next start code trimmed off, as `ff_nal_parse_units` does. Empty when + /// the buffer holds no start code. + static func nalRanges(_ bytes: UnsafeRawBufferPointer) -> [Range] { + guard let base = bytes.baseAddress?.assumingMemoryBound(to: UInt8.self), bytes.count >= 4 else { + return [] + } + let n = bytes.count + var starts: [Int] = [] + var i = 0 + while i + 2 < n { + // The third byte of a start code is 1 and the two before it are 0, so any byte above 1 at + // i + 2 rules out a start code beginning at i, i + 1 or i + 2. + if base[i + 2] > 1 { i += 3; continue } + if base[i] == 0, base[i + 1] == 0, base[i + 2] == 1 { + starts.append(i + 3) + i += 3 + } else { + i += 1 + } + } + var ranges: [Range] = [] + ranges.reserveCapacity(starts.count) + for (k, start) in starts.enumerated() { + var end = k + 1 < starts.count ? starts[k + 1] - 3 : n + while end > start, base[end - 1] == 0 { end -= 1 } + if end > start { ranges.append(start..) -> Bool { + guard let data = packet.pointee.data, packet.pointee.size > 0 else { return false } + let source = UnsafeRawBufferPointer(start: data, count: Int(packet.pointee.size)) + let ranges = nalRanges(source) + guard !ranges.isEmpty else { return false } + let total = ranges.reduce(0) { $0 + 4 + $1.count } + guard total <= Int(Int32.max) else { return false } + + let pad = Int(AV_INPUT_BUFFER_PADDING_SIZE) + guard let newRef = av_buffer_alloc(total + pad) else { return false } + guard let dst = newRef.pointee.data else { + var ref: UnsafeMutablePointer? = newRef + av_buffer_unref(&ref) + return false + } + var w = 0 + for range in ranges { + let len = range.count + dst[w + 0] = UInt8((len >> 24) & 0xFF) + dst[w + 1] = UInt8((len >> 16) & 0xFF) + dst[w + 2] = UInt8((len >> 8) & 0xFF) + dst[w + 3] = UInt8(len & 0xFF) + w += 4 + memcpy(dst + w, data + range.lowerBound, len) + w += len + } + memset(dst + total, 0, pad) + av_buffer_unref(&packet.pointee.buf) + packet.pointee.buf = newRef + packet.pointee.data = newRef.pointee.data + packet.pointee.size = Int32(total) + return true + } +} diff --git a/Sources/AetherEngine/Video/CodecRoutePolicy.swift b/Sources/AetherEngine/Video/CodecRoutePolicy.swift index 31fdb6c35..70a04f3df 100644 --- a/Sources/AetherEngine/Video/CodecRoutePolicy.swift +++ b/Sources/AetherEngine/Video/CodecRoutePolicy.swift @@ -140,6 +140,14 @@ extension HLSVideoEngine { /// (media-direct, no CODECS) hid it, but once HEVC is signaled through a master (below) a strict /// Apple TV rejects the Main10 declaration against the Main hvcC in the init; macOS / the Simulator /// tolerate the mismatch. Deriving the string from the actual hvcC keeps master and init consistent. + /// AE#674: the highest `general_level_idc` a master's CODECS declares (5.1, which covers 2160p60). + /// Encoders stamp 5.2 (156) on 1080p24 streams, and AVPlayer screens a master variant's declared + /// level against the device: an Apple TV 4K refuses 5.2 with -11848 / CoreMedia -15517 on every + /// attempt, DV and reduced master alike, while the same bytes decode media-direct, where nothing + /// is declared. The init's hvcC keeps the stated level; only the attribute is capped. A stream + /// that genuinely needs more would end on the media playlist anyway, so capping costs it nothing. + static let maxDeclaredHEVCLevel = 153 + static func hevcCodecsString( fromConfigRecord hvcC: [UInt8], sampleEntry: String = "hvc1" @@ -174,13 +182,30 @@ extension HLSVideoEngine { var trimmedConstraints = constraintBytes while let last = trimmedConstraints.last, last == 0 { trimmedConstraints.removeLast() } let tier = tierFlag == 1 ? "H" : "L" - var s = "\(sampleEntry).\(spacePrefix)\(profileIDC).\(compatHex).\(tier)\(levelIDC)" + let declaredLevel = min(Int(levelIDC), maxDeclaredHEVCLevel) + var s = "\(sampleEntry).\(spacePrefix)\(profileIDC).\(compatHex).\(tier)\(declaredLevel)" if !trimmedConstraints.isEmpty { s += "." + trimmedConstraints.map { String(format: "%02x", $0) }.joined(separator: ".") } return s } + /// Same string read off the first SPS of Annex-B extradata (MPEG-TS, Annex-B Matroska), which + /// carries no hvcC. The SPS's profile_tier_level general part is byte for byte the hvcC header + /// bytes 1..12, and it is what the mp4 muxer builds the init's hvcC from, so the two agree. + /// Audit HLS-3: without this an 8-bit Main TS source fell back to the Main10 declaration. + static func hevcCodecsString( + fromAnnexBExtradata extradata: [UInt8], + sampleEntry: String = "hvc1" + ) -> String? { + guard let sps = VideoConfigRecord.splitAnnexBNALs(extradata) + .first(where: { $0.count > 2 && (($0[0] >> 1) & 0x3F) == 33 }) else { return nil } + // Past the 2-byte NAL header: one byte of vps_id / max_sub_layers / nesting, then the PTL. + let rbsp = H264SPS.unescape(Array(sps.dropFirst(2))) + guard rbsp.count >= 13 else { return nil } + return hevcCodecsString(fromConfigRecord: [1] + Array(rbsp[1...12]), sampleEntry: sampleEntry) + } + /// RFC 6381 `avc1.PPCCLL` read straight off the avcC configuration record, which states all three /// bytes outright: AVCProfileIndication, profile_compatibility (the constraint_set flags) and /// AVCLevelIndication are bytes 1..3. Same reasoning as `hevcCodecsString`: the record is what the @@ -239,9 +264,9 @@ extension HLSVideoEngine { profile: codecpar.pointee.profile, level: codecpar.pointee.level) } - /// Derive the plain-HEVC CODECS string from the source hvcC when parseable, else fall back to the - /// legacy Main10 form. Used only by the non-DV `.none` / `.profile82` branch; DV variants keep their - /// deliberate `hvc1.2.4` (Main10 PQ base) declaration. + /// Derive the HEVC CODECS string from the source hvcC (or the SPS of Annex-B extradata) when + /// parseable, else fall back to the Main10 form. Every hvc1 route uses it, the DV ones included + /// (AE#674): their base layer is the hvcC in the init, so the master declares what that states. private func plainHEVCCodecs( codecpar: UnsafePointer, fallbackLevel hevcLevel: Int @@ -249,7 +274,8 @@ extension HLSVideoEngine { if let ed = codecpar.pointee.extradata, codecpar.pointee.extradata_size > 0 { let bytes = Array(UnsafeBufferPointer( start: ed, count: Int(codecpar.pointee.extradata_size))) - if let derived = Self.hevcCodecsString(fromConfigRecord: bytes) { + if let derived = Self.hevcCodecsString(fromConfigRecord: bytes) + ?? Self.hevcCodecsString(fromAnnexBExtradata: bytes) { return derived } } @@ -459,7 +485,7 @@ extension HLSVideoEngine { let dvLevelRaw = Int(dvRecord?.dv_level ?? 0) let dvLevel = dvLevelRaw > 0 ? dvLevelRaw : 6 let hevcLevelRaw = Int(codecpar.pointee.level) - let hevcLevel = hevcLevelRaw > 0 ? hevcLevelRaw : 150 + let hevcLevel = hevcLevelRaw > 0 ? min(hevcLevelRaw, Self.maxDeclaredHEVCLevel) : 150 let dvLevelStr = String(format: "%02d", dvLevel) if let r = dvRecord { @@ -596,7 +622,7 @@ extension HLSVideoEngine { return CodecRoute( codecTagOverride: "hvc1", videoRange: .pq, - primaryCodecs: "hvc1.2.4.L\(hevcLevel)", + primaryCodecs: plainHEVCCodecs(codecpar: codecpar, fallbackLevel: hevcLevel), supplementalCodecs: supplemental, doviConfig: doviConfig, convertP7ToProfile81: false, @@ -618,7 +644,7 @@ extension HLSVideoEngine { return CodecRoute( codecTagOverride: "hvc1", videoRange: .hlg, - primaryCodecs: "hvc1.2.4.L\(hevcLevel)", + primaryCodecs: plainHEVCCodecs(codecpar: codecpar, fallbackLevel: hevcLevel), supplementalCodecs: supplemental, doviConfig: doviConfig, convertP7ToProfile81: false, @@ -640,7 +666,7 @@ extension HLSVideoEngine { return CodecRoute( codecTagOverride: "hvc1", videoRange: .pq, - primaryCodecs: "hvc1.2.4.L\(hevcLevel)", + primaryCodecs: plainHEVCCodecs(codecpar: codecpar, fallbackLevel: hevcLevel), supplementalCodecs: supplemental, doviConfig: doviConfig, convertP7ToProfile81: effectiveDvMode, diff --git a/Sources/AetherEngine/Video/DolbyVisionRecordAudit.swift b/Sources/AetherEngine/Video/DolbyVisionRecordAudit.swift index 807b46a47..bbfbabc50 100644 --- a/Sources/AetherEngine/Video/DolbyVisionRecordAudit.swift +++ b/Sources/AetherEngine/Video/DolbyVisionRecordAudit.swift @@ -191,7 +191,14 @@ public enum DolbyVisionRecordAudit { } catch { return nil } + return rpuProfile(walking: demuxer, packetBudget: packetBudget) + } + /// The walk over an opened demuxer, split out so a test can hand it a counting reader. + static func rpuProfile( + walking demuxer: Demuxer, packetBudget: Int = auditPacketBudget, + byteBudget: Int64 = Int64(walkByteBudget) + ) -> Int? { let videoIdx = demuxer.videoStreamIndex guard videoIdx >= 0, let stream = demuxer.stream(at: videoIdx) else { return nil } let codecpar = stream.pointee.codecpar @@ -199,13 +206,24 @@ public enum DolbyVisionRecordAudit { codec: .hevc, extradata: codecpar?.pointee.extradata, size: Int(codecpar?.pointee.extradata_size ?? 0)) + // Audit BIT-103: Matroska resyncs byte by byte through junk inside one av_read_frame, where + // neither the packet fuse nor the packet byte count below can see the bytes go by. + demuxer.beginInputByteBudget(byteBudget) + defer { demuxer.endInputByteBudget() } + var walked = 0 + var packetsRead = 0 + var bytesRead = 0 while walked < packetBudget { + guard !walkExhausted(packetsRead: packetsRead, bytesRead: bytesRead, packetBudget: packetBudget) + else { return nil } guard let packet = (try? demuxer.readPacket()) ?? nil else { return nil } defer { av_packet_unref(packet) av_packet_free_safe(packet) } + packetsRead += 1 + bytesRead += Int(max(packet.pointee.size, 0)) guard packet.pointee.stream_index == videoIdx else { continue } walked += 1 if let profile = rpuProfile(packet, framing: framing) { return profile } @@ -213,6 +231,18 @@ public enum DolbyVisionRecordAudit { return nil } + /// Audit BIT-2: the walk counted only video packets, so a source that stops delivering video after + /// its head made every open read the rest of the file. Every packet counts against these instead. + /// No AVDISCARD_ALL on the other streams: the demuxer would then skip them inside one read, where + /// neither ceiling can see the bytes go by. + static let foreignPacketFuseMultiplier = 16 + static let walkByteBudget = 64 * 1024 * 1024 + + static func walkExhausted(packetsRead: Int, bytesRead: Int, packetBudget: Int) -> Bool { + let (fuse, overflow) = packetBudget.multipliedReportingOverflow(by: foreignPacketFuseMultiplier) + return packetsRead >= (overflow ? Int.max : fuse) || bytesRead >= walkByteBudget + } + /// The verdict for a source in one call, gate included: opens it, reads the record and the VUI, and /// walks packets only when the record is contradicted. nil means the record stands. /// diff --git a/Sources/AetherEngine/Video/DoviRpuConverter+Probe.swift b/Sources/AetherEngine/Video/DoviRpuConverter+Probe.swift index fa4da7a19..c724255af 100644 --- a/Sources/AetherEngine/Video/DoviRpuConverter+Probe.swift +++ b/Sources/AetherEngine/Video/DoviRpuConverter+Probe.swift @@ -40,7 +40,38 @@ extension AetherEngine { // MARK: - Dovi convert probe (aetherctl dovitest) - /// Walk every HEVC video packet, run `convertPacketToProfile81`, and write Annex-B output (AVCC length prefix replaced by `00 00 00 01`) for validation with `dovi_tool extract-rpu`. False returns are counted as failures but still emitted. + private nonisolated static let doviProbeStartCode: [UInt8] = [0x00, 0x00, 0x00, 0x01] + + /// The head of the probe's output: the source's VPS/SPS/PPS, so `dovi_tool`'s parser can walk the stream. hvcC keeps them out of band and they are re-emitted start-coded; Annex-B extradata already is start-coded and is written as it is. + nonisolated static func doviProbeParameterSets( + extradata: UnsafePointer?, size: Int, framing: VideoNALFraming + ) -> Data { + guard let extradata, size > 0 else { return Data() } + switch framing { + case .annexB: + return Data(bytes: extradata, count: size) + case .lengthPrefixed: + var out = Data() + for nal in parseHVCCParameterSets(extradata, size) { + out.append(contentsOf: doviProbeStartCode) + out.append(contentsOf: nal) + } + return out + } + } + + /// One packet as Annex B: every NAL behind a four-byte start code, whichever framing the packet arrived in. + nonisolated static func doviProbeAnnexB(_ packet: UnsafePointer, framing: VideoNALFraming) -> Data { + guard let data = packet.pointee.data, packet.pointee.size > 0 else { return Data() } + var out = Data() + A53SEIParser.forEachNAL(data, Int(packet.pointee.size), framing) { nal, len in + out.append(contentsOf: doviProbeStartCode) + out.append(nal, count: len) + } + return out + } + + /// Walk every HEVC video packet, run `convertPacketToProfile81`, and write the result as Annex B for validation with `dovi_tool extract-rpu`. The source's own NAL framing is resolved once from its extradata and used for the converter, the enhancement-layer probe and the writer alike, so an Annex-B source (a disc remux) is walked as what it is. False returns are counted as failures but still emitted. A read or write error ends the run by throwing rather than reporting the part that was done. public nonisolated static func doviConvertProbe( url: URL, outputPath: String, @@ -59,65 +90,43 @@ extension AetherEngine { ) } - FileManager.default.createFile(atPath: outputPath, contents: nil) - guard let handle = FileHandle(forWritingAtPath: outputPath) else { - throw AetherEngineError.noVideoStream - } + // O_NOFOLLOW: the output name may sit in a directory other users can write to. + let fd = open(outputPath, O_WRONLY | O_CREAT | O_TRUNC | O_NOFOLLOW, 0o600) + guard fd >= 0 else { throw POSIXError(POSIXErrorCode(rawValue: errno) ?? .EIO) } + let handle = FileHandle(fileDescriptor: fd, closeOnDealloc: true) defer { try? handle.close() } - let startCode: [UInt8] = [0x00, 0x00, 0x00, 0x01] - - // MP4 hvcC keeps VPS/SPS/PPS out-of-band; emit them once as Annex-B so dovi_tool's parser can walk the stream. - if let cp = stream.pointee.codecpar, - let ed = cp.pointee.extradata, cp.pointee.extradata_size > 0 { - let edSize = Int(cp.pointee.extradata_size) - for nal in parseHVCCParameterSets(ed, edSize) { - handle.write(Data(startCode)) - handle.write(Data(nal)) - } + var extradata: UnsafePointer? + var extradataSize = 0 + if let cp = stream.pointee.codecpar, let ed = cp.pointee.extradata, cp.pointee.extradata_size > 0 { + extradata = UnsafePointer(ed) + extradataSize = Int(cp.pointee.extradata_size) } + let framing = A53SEIParser.nalFraming(codec: .hevc, extradata: extradata, size: extradataSize) + try handle.write(contentsOf: doviProbeParameterSets( + extradata: extradata, size: extradataSize, framing: framing)) var packetsProcessed = 0 var conversions = 0 var failures = 0 var elType: String? = nil - while true { - let maybePacket: UnsafeMutablePointer? - do { - maybePacket = try demuxer.readPacket() - } catch { - break + while let packet = try demuxer.readPacket() { + defer { + av_packet_unref(packet) + av_packet_free_safe(packet) } - guard let packet = maybePacket else { break } // EOF - - if packet.pointee.stream_index == videoIdx { - packetsProcessed += 1 - if elType == nil { - elType = DoviRpuConverter.enhancementLayerType(packet) - } - if DoviRpuConverter.convertPacketToProfile81(packet) { - conversions += 1 - } else { - failures += 1 - } - if let data = packet.pointee.data, packet.pointee.size > 4 { - let size = Int(packet.pointee.size) - var off = 0 - while off + 4 <= size { - var len = 0 - for i in 0..<4 { len = (len << 8) | Int(data[off + i]) } - let nalStart = off + 4 - if len == 0 || nalStart + len > size { break } - handle.write(Data(startCode)) - handle.write(Data(bytes: data + nalStart, count: len)) - off = nalStart + len - } - } + guard packet.pointee.stream_index == videoIdx else { continue } + packetsProcessed += 1 + if elType == nil { + elType = DoviRpuConverter.enhancementLayerType(packet, framing: framing) } - - av_packet_unref(packet) - av_packet_free_safe(packet) + if DoviRpuConverter.convertPacketToProfile81(packet, framing: framing) { + conversions += 1 + } else { + failures += 1 + } + try handle.write(contentsOf: doviProbeAnnexB(packet, framing: framing)) } return DoviConvertProbeResult( diff --git a/Sources/AetherEngine/Video/DoviRpuConverter.swift b/Sources/AetherEngine/Video/DoviRpuConverter.swift index 951b0484a..2dd68515e 100644 --- a/Sources/AetherEngine/Video/DoviRpuConverter.swift +++ b/Sources/AetherEngine/Video/DoviRpuConverter.swift @@ -39,7 +39,18 @@ public enum DoviRpuConverter { guard Self.canRebuild(framing) else { return true } let size = Int(packet.pointee.size) - var outputNALs: [[UInt8]] = [] + // Audit PERF-111: what survives is described, not copied. Each piece points into the source + // packet (passed through) or into a libdovi buffer (a rewritten RPU), and the one rebuild below + // is the only copy. The source stays alive until that copy is made, and the libdovi buffers + // until this function returns. + struct Piece { + let base: UnsafePointer + let count: Int + } + var pieces: [Piece] = [] + pieces.reserveCapacity(16) + var rewritten: [UnsafePointer] = [] + defer { for buffer in rewritten { dovi_data_free(buffer) } } var converted = false var droppedEL = false var degraded = false @@ -56,34 +67,26 @@ public enum DoviRpuConverter { degraded = true return } - let rc = dovi_convert_rpu_with_mode(rpu, 2) - if rc != 0 { - dovi_rpu_free(rpu) + defer { dovi_rpu_free(rpu) } + guard dovi_convert_rpu_with_mode(rpu, 2) == 0, + let out = dovi_write_unspec62_nalu(rpu) else { degraded = true return } - guard let out = dovi_write_unspec62_nalu(rpu) else { - dovi_rpu_free(rpu) - degraded = true - return - } - let outLen = out.pointee.len - guard let outData = out.pointee.data, outLen > 0 else { + guard let outData = out.pointee.data, out.pointee.len > 0 else { dovi_data_free(out) - dovi_rpu_free(rpu) degraded = true return } - outputNALs.append([UInt8](UnsafeBufferPointer(start: outData, count: outLen))) - dovi_data_free(out) - dovi_rpu_free(rpu) + rewritten.append(out) + pieces.append(Piece(base: outData, count: out.pointee.len)) converted = true case nalTypeEL: droppedEL = true default: - outputNALs.append([UInt8](UnsafeBufferPointer(start: nal, count: len))) + pieces.append(Piece(base: nal, count: len)) } } @@ -92,8 +95,8 @@ public enum DoviRpuConverter { } var total = 0 - for nal in outputNALs { - total += lengthPrefixSize + nal.count + for piece in pieces { + total += lengthPrefixSize + piece.count } // Degenerate: all NALs were RPU/EL. Leave the packet untouched rather than emit a zero-length video packet. guard total > 0 else { return true } @@ -110,8 +113,8 @@ public enum DoviRpuConverter { let emitsStartCodes = framing == .annexB var w = 0 - for nal in outputNALs { - let n = nal.count + for piece in pieces { + let n = piece.count if emitsStartCodes { dst[w + 0] = 0; dst[w + 1] = 0; dst[w + 2] = 0; dst[w + 3] = 1 } else { @@ -121,10 +124,8 @@ public enum DoviRpuConverter { dst[w + 3] = UInt8(n & 0xFF) } w += lengthPrefixSize - nal.withUnsafeBufferPointer { src in - if let base = src.baseAddress, n > 0 { - memcpy(dst + w, base, n) - } + if n > 0 { + memcpy(dst + w, piece.base, n) } w += n } diff --git a/Sources/AetherEngine/Video/H264CompositionOffsetRepair.swift b/Sources/AetherEngine/Video/H264CompositionOffsetRepair.swift index 780ef5080..f718bd9d2 100644 --- a/Sources/AetherEngine/Video/H264CompositionOffsetRepair.swift +++ b/Sources/AetherEngine/Video/H264CompositionOffsetRepair.swift @@ -195,7 +195,8 @@ enum H264CompositionOffsetRepair { decodeLead: Int64 ) -> Int64 { guard streamStartTime != Int64.min, ladderStart != Int64.min, decodeLead > 0 else { return 0 } - let raw = streamStartTime - ladderStart + let (raw, overflow) = streamStartTime.subtractingReportingOverflow(ladderStart) + guard !overflow else { return 0 } return min(max(raw, 0), decodeLead) } @@ -290,8 +291,13 @@ enum H264CompositionOffsetRepair { // are then spread over twice the ladder while still being distinct. Ranks have to FILL the // window they came from: the span may exceed the sample only by the pictures still in flight // at its ragged edge, which is the reorder delay. + // A hierarchical mini-GOP longer than the reorder delay sends its anchor further ahead than + // that allowance (#699's HEVC stream: POC 15 decoded while 11 to 14 are still to come, at + // delay 2), so the window may instead prove itself by a decode-order prefix of at least + // `minimumSamples` pictures that fills its display range exactly, which is the stricter test. guard let maxIndex = displayIndices.max(), let minIndex = displayIndices.min(), - maxIndex - minIndex + 1 <= Int64(samples.count + videoDelay + 1) else { + maxIndex - minIndex + 1 <= Int64(samples.count + videoDelay + 1) + || hasContiguousPrefix(samples.map { $0.pictureOrderCount / pocStep }) else { return .inconclusive("display indices do not fill the sampled window") } @@ -353,6 +359,19 @@ enum H264CompositionOffsetRepair { ) } + /// True when some decode-order prefix of at least `minimumSamples` display indices covers its + /// own range without a gap, i.e. a whole number of mini-GOPs closed inside the window. + static func hasContiguousPrefix(_ indices: [Int64]) -> Bool { + guard var low = indices.first else { return false } + var high = low + for (offset, index) in indices.enumerated() { + low = min(low, index) + high = max(high, index) + if offset + 1 >= minimumSamples, high - low == Int64(offset) { return true } + } + return false + } + /// Which phase of the cadence the sampled ladder starts on, and where that puts lattice ordinal /// zero. Every sampled picture has to land exactly, and the answer has to be the only phase in /// the period that does, or the ladder is not evidence of a lattice at all. This is what lets a @@ -425,6 +444,10 @@ enum H264CompositionOffsetRepair { private(set) var sequenceBaseOrdinal: Int64? /// A seek leaves the parser and the sequence anchor behind; the next keyframe re-anchors. private var awaitingReanchor = true + /// Pictures that follow the landing keyframe in decode order but precede it in display + /// order (an open GOP's leading pictures). The landing is displayed that many slots after + /// the one it is decoded in, so the re-anchor has to know the count. + private var landingLeadingPictures: Int64 = 0 /// Pictures emitted untouched because no anchor was available or the arithmetic did not /// close. Surfaced so a stream this repair does not describe is visible as a number. private(set) var unrepairedPictures = 0 @@ -436,6 +459,11 @@ enum H264CompositionOffsetRepair { awaitingReanchor = true sequenceAnchorDTS = nil sequenceBaseOrdinal = nil + landingLeadingPictures = 0 + } + + mutating func noteLanding(leadingPictures: Int64) { + landingLeadingPictures = max(0, leadingPictures) } /// nil when the picture cannot be placed; the caller then emits it untouched. @@ -463,10 +491,21 @@ enum H264CompositionOffsetRepair { return nil } let landingIndex = pictureOrderCount / plan.pocStep - sequenceAnchorDTS = dts - landingIndex * plan.step + // The landing is decoded in slot k and displayed in slot k + leading, so the + // sequence origin sits `landingIndex - leading` slots before its decode time. + let (anchorSlots, slotsOverflow) = landingIndex.subtractingReportingOverflow(landingLeadingPictures) + // Audit BIT-101: a 64-bit `tfdt` step and a landing order near -2^31 leave Int64. + let (landingTicks, productOverflow) = slotsOverflow + ? (0, true) : anchorSlots.multipliedReportingOverflow(by: plan.step) + let (anchor, anchorOverflow) = dts.subtractingReportingOverflow(landingTicks) + guard !productOverflow, !anchorOverflow else { + unrepairedPictures += 1 + return nil + } + sequenceAnchorDTS = anchor sequenceBaseOrdinal = plan.presentationOrdinal(ladderTimestamp: dts) .flatMap { ordinal -> Int64? in - let (base, overflow) = ordinal.subtractingReportingOverflow(landingIndex) + let (base, overflow) = ordinal.subtractingReportingOverflow(anchorSlots) return overflow ? nil : base } awaitingReanchor = false @@ -501,22 +540,25 @@ enum H264CompositionOffsetRepair { } } -/// Reads a picture order count per packet with libavcodec's H.264 parser. No decoder is opened and -/// no picture is reconstructed: the parser walks the slice header, which is where display order -/// lives. It takes MP4's length-prefixed AVCC payload directly as long as the codec context carries -/// the `avcC` extradata, so no Annex-B conversion sits in the packet path. +/// Reads a picture order count per packet with libavcodec's H.264 or HEVC parser. No decoder is +/// opened and no picture is reconstructed: the parser walks the slice header, which is where display +/// order lives. It takes MP4's length-prefixed AVCC / HVCC payload directly as long as the codec +/// context carries the `avcC` / `hvcC` extradata, so no Annex-B conversion sits in the packet path. final class H264PictureOrderReader { private var parser: UnsafeMutablePointer? private var context: UnsafeMutablePointer? + private let codecID: AVCodecID var isFramePicture: Bool { parser?.pointee.picture_structure == AV_PICTURE_STRUCTURE_FRAME } init?(codecParameters: UnsafePointer, timeBase: AVRational) { - guard let codec = avcodec_find_decoder(AV_CODEC_ID_H264), + codecID = codecParameters.pointee.codec_id + guard codecID == AV_CODEC_ID_H264 || codecID == AV_CODEC_ID_HEVC, + let codec = avcodec_find_decoder(codecID), let context = avcodec_alloc_context3(codec) else { return nil } self.context = context guard avcodec_parameters_to_context(context, codecParameters) >= 0 else { return nil } context.pointee.pkt_timebase = timeBase - guard let parser = av_parser_init(Int32(AV_CODEC_ID_H264.rawValue)) else { return nil } + guard let parser = av_parser_init(Int32(codecID.rawValue)) else { return nil } self.parser = parser // MP4 samples are whole access units; without this the parser hunts for start codes that // length-prefixed payloads do not contain. @@ -535,7 +577,7 @@ final class H264PictureOrderReader { func reset() { guard let parser else { return } av_parser_close(parser) - self.parser = av_parser_init(Int32(AV_CODEC_ID_H264.rawValue)) + self.parser = av_parser_init(Int32(codecID.rawValue)) self.parser?.pointee.flags |= Int32(PARSER_FLAG_COMPLETE_FRAMES) } @@ -574,6 +616,7 @@ final class H264CompositionOffsetRepairSession { private let videoDelay: Int private let streamStartTime: Int64 private let ladderStart: Int64 + private let codecName: String private var reader: H264PictureOrderReader? private var rewriter: H264CompositionOffsetRepair.Rewriter? private let partialRepairCandidate: H264PartialCompositionRepairSession? @@ -582,9 +625,15 @@ final class H264CompositionOffsetRepairSession { private var held: [(packet: UnsafeMutablePointer, pictureOrderCount: Int64?)] = [] private var heldBytes = 0 private var verdictDescription = "sampling" - - /// nil unless this stream is the exact shape the defect needs: ISO-BMFF, H.264, and a bitstream - /// that declares reordered pictures. Everything else never sees a parser or a held packet. + /// After a seek on a repaired stream: nil until the landing keyframe arrives, then its picture + /// order while the packets behind it are held to count its leading pictures. + private var relanding = false + private var landingPictureOrder: Int64? + + /// nil unless this stream is the exact shape the defect needs: ISO-BMFF, H.264 or HEVC, and a + /// bitstream that declares reordered pictures. Everything else never sees a parser or a held + /// packet. HEVC carries the same loss (#699: a CUVA conformance stream with B pictures and no + /// `ctts`), and its slice header carries the same picture order count. init?( containerFormatName: String?, stream: UnsafeMutablePointer, @@ -594,7 +643,7 @@ final class H264CompositionOffsetRepairSession { let containers = containerFormatName?.split(separator: ",") ?? [] guard containers.contains("mov") || containers.contains("mp4") else { return nil } guard let codecpar = stream.pointee.codecpar, - codecpar.pointee.codec_id == AV_CODEC_ID_H264, + codecpar.pointee.codec_id == AV_CODEC_ID_H264 || codecpar.pointee.codec_id == AV_CODEC_ID_HEVC, codecpar.pointee.video_delay > 0, let reader = H264PictureOrderReader( codecParameters: codecpar, timeBase: stream.pointee.time_base) @@ -605,7 +654,11 @@ final class H264CompositionOffsetRepairSession { self.ladderStart = ladderStart self.reader = reader let (lead, overflow) = stream.pointee.start_time.subtractingReportingOverflow(ladderStart) - partialRepairCandidate = !overflow && stream.pointee.start_time != Int64.min && ladderStart != Int64.min + // The partial-region repair reads H.264 NAL types to find its IDRs, so HEVC gets the + // all-missing repair only. + self.codecName = codecpar.pointee.codec_id == AV_CODEC_ID_HEVC ? "HEVC" : "H.264" + partialRepairCandidate = codecpar.pointee.codec_id == AV_CODEC_ID_H264 + && !overflow && stream.pointee.start_time != Int64.min && ladderStart != Int64.min ? H264PartialCompositionRepairSession(stream: stream, streamIndex: streamIndex, presentationLead: lead) : nil } @@ -640,6 +693,7 @@ final class H264CompositionOffsetRepairSession { case .off: return monitorsPartialOffsets ? partialRepairCandidate?.ingest(packet) ?? false : false case .repairing: + if relanding { return ingestAfterSeek(packet) } guard packet.pointee.stream_index == streamIndex else { return false } applyRepair(to: packet, pictureOrderCount: reader?.pictureOrderCount(for: packet)) return false @@ -668,6 +722,58 @@ final class H264CompositionOffsetRepairSession { } } + /// A seek lands on a keyframe that may be an open GOP's entry point, displayed after the leading + /// pictures decoded behind it. Their number is only known at the first picture that follows the + /// landing in display order (leading pictures all precede the trailing ones in decode order), so + /// everything from the landing up to that picture is held, every stream, and then repaired with + /// the count. Without it the post-seek axis sat that many slots early, which put B pictures + /// below their own decode time and left them in decode order (#699, resume onto a CRA). + private func ingestAfterSeek(_ packet: UnsafeMutablePointer) -> Bool { + var pictureOrderCount: Int64? + if packet.pointee.stream_index == streamIndex { + pictureOrderCount = reader?.pictureOrderCount(for: packet) + if landingPictureOrder == nil { + // Pictures before the landing keyframe cannot be placed; the rewriter passes them + // through untouched, as it did before. + guard (packet.pointee.flags & AV_PKT_FLAG_KEY) != 0, let pictureOrderCount else { + applyRepair(to: packet, pictureOrderCount: pictureOrderCount) + return false + } + landingPictureOrder = pictureOrderCount + } else if let landing = landingPictureOrder, let pictureOrderCount, pictureOrderCount > landing { + held.append((packet, pictureOrderCount)) + finishLanding() + return true + } + } else if landingPictureOrder == nil { + return false + } + held.append((packet, pictureOrderCount)) + heldBytes += Int(max(0, packet.pointee.size)) + if heldBytes >= H264CompositionOffsetRepair.sampleByteBudget + || held.count >= H264CompositionOffsetRepair.heldPacketCeiling { + finishLanding() + } + return true + } + + private func finishLanding() { + relanding = false + guard var rewriter else { return } + if let landing = landingPictureOrder { + let leading = held.filter { + $0.packet.pointee.stream_index == streamIndex + && ($0.pictureOrderCount.map { $0 < landing } ?? false) + }.count + rewriter.noteLanding(leadingPictures: Int64(leading)) + } + for entry in held where entry.packet.pointee.stream_index == streamIndex { + applyRepair(to: entry.packet, pictureOrderCount: entry.pictureOrderCount, using: &rewriter) + } + self.rewriter = rewriter + landingPictureOrder = nil + } + /// A healthy file usually proves itself on its first packet, and paying twelve packets of hold /// for it would tax every well-formed MP4 with B-frames on the platform. private var earlyHealthyVerdict: Bool { @@ -678,6 +784,7 @@ final class H264CompositionOffsetRepairSession { /// EOF during sampling. Decides on what is there, so the held packets are still delivered. func endOfStream() { if phase == .sampling { decide() } + if relanding { finishLanding() } if monitorsPartialOffsets { partialRepairCandidate?.endOfStream() } } @@ -697,6 +804,8 @@ final class H264CompositionOffsetRepairSession { case .repairing: rewriter?.noteSeek() reader?.reset() + relanding = true + landingPictureOrder = nil case .off: break } @@ -718,7 +827,7 @@ final class H264CompositionOffsetRepairSession { } func dequeue() -> UnsafeMutablePointer? { - guard phase != .sampling else { return nil } + guard phase != .sampling, !relanding else { return nil } if !held.isEmpty { return held.removeFirst().packet } return monitorsPartialOffsets ? partialRepairCandidate?.dequeue() : nil } @@ -759,7 +868,7 @@ final class H264CompositionOffsetRepairSession { } self.rewriter = rewriter EngineLog.emit( - "[Demuxer] #409 missing H.264 composition offsets confirmed on stream \(streamIndex): " + "[Demuxer] #409 missing \(codecName) composition offsets confirmed on stream \(streamIndex): " + "\(verdictDescription) samples=\(samples.count)", category: .demux ) diff --git a/Sources/AetherEngine/Video/H264MatroskaSlotPermutation.swift b/Sources/AetherEngine/Video/H264MatroskaSlotPermutation.swift index 92c0e4060..53720e956 100644 --- a/Sources/AetherEngine/Video/H264MatroskaSlotPermutation.swift +++ b/Sources/AetherEngine/Video/H264MatroskaSlotPermutation.swift @@ -159,6 +159,9 @@ enum H264MatroskaSlotPermutation { /// Presentation slots in the order they arrived, which for this defect is decode order. The /// slot a picture owns is the one at its DISPLAY rank, so this is the whole lookup table. private(set) var slots: [Int64] = [] + /// Slot positions already spoken for in this sequence: a rank handed out, or the coding + /// position of a picture that goes out on its own slot. + private var claimed: Set = [] private(set) var repairedPictures = 0 private(set) var unrepairedPictures = 0 /// A seek leaves the sequence behind; only an IDR starts a new one. A landing that is not one @@ -176,9 +179,17 @@ enum H264MatroskaSlotPermutation { mutating func noteSeek() { slots.removeAll(keepingCapacity: true) + claimed.removeAll(keepingCapacity: true) awaitingSequenceStart = true } + /// Audit BIT-3: the slots of this sequence no picture has claimed, in display order. When an + /// IDR closes the sequence these are the only slots the pictures still waiting on it can own; + /// the next sequence's slots belong to its own pictures. + func unclaimedSlots() -> [Int64] { + slots.indices.filter { !claimed.contains(Int64($0)) }.map { slots[$0] } + } + /// Records a video picture and returns the display rank whose slot it should carry, or nil /// when it has to go out exactly as it arrived. The slot is recorded either way: it belongs /// to whichever picture owns this coding position, and that picture may still be coming. @@ -196,6 +207,7 @@ enum H264MatroskaSlotPermutation { } if isKeyframe, pictureOrderCount == 0 { slots = [slot] + claimed = [0] awaitingSequenceStart = false repairedPictures += 1 return 0 @@ -209,21 +221,24 @@ enum H264MatroskaSlotPermutation { return nil } slots.append(slot) + let codingIndex = Int64(slots.count - 1) guard let pictureOrderCount, pictureOrderCount >= 0, pictureOrderCount % pocStep == 0 else { unrepairedPictures += 1 + claimed.insert(codingIndex) return nil } let rank = pictureOrderCount / pocStep - let codingIndex = Int64(slots.count - 1) // Behind its own slot by more than the reorder delay would put the picture before its // stated decode time; further ahead than the ceiling is not a mini-GOP any more. guard codingIndex - rank <= Int64(videoDelay), rank - codingIndex <= H264MatroskaSlotPermutation.lookaheadCeiling else { unrepairedPictures += 1 + claimed.insert(codingIndex) return nil } repairedPictures += 1 + claimed.insert(rank) return rank } @@ -342,10 +357,35 @@ final class H264MatroskaSlotPermutationSession: H264TimestampRepairSession { _ packet: UnsafeMutablePointer, pictureOrderCount: Int64? ) -> Int64? { - sequence?.admit( - slot: packet.pointee.pts, - pictureOrderCount: packet.pointee.pts == Int64.min ? nil : pictureOrderCount, - isKeyframe: isKeyframe(packet)) + let pictureOrderCount = packet.pointee.pts == Int64.min ? nil : pictureOrderCount + let keyframe = isKeyframe(packet) + if keyframe, pictureOrderCount == 0 { settleRanksOfClosingSequence() } + return sequence?.admit(slot: packet.pointee.pts, pictureOrderCount: pictureOrderCount, isKeyframe: keyframe) + } + + /// Audit BIT-3: a rank means something only on the ladder it was counted against, and an IDR is + /// about to replace that ladder. Every held picture still carrying a rank gets its slot from the + /// closing sequence now: the one its rank owns when the container stated it, else one of the + /// slots no picture claimed, in display order. Left pending, the lookup answered from the NEW + /// sequence and two pictures shared one presentation time. + private func settleRanksOfClosingSequence() { + guard let sequence, sequence.brokenReason == nil else { return } + var waiting: [(index: Int, rank: Int64)] = [] + for index in pending.indices { + guard let rank = pending[index].rank else { continue } + if let slot = sequence.slot(forRank: rank) { + pending[index].packet.pointee.pts = slot + } else { + waiting.append((index, rank)) + } + pending[index].rank = nil + } + guard !waiting.isEmpty else { return } + var free = sequence.unclaimedSlots()[...] + for entry in waiting.sorted(by: { $0.rank < $1.rank }) { + guard let slot = free.popFirst() else { break } + pending[entry.index].packet.pointee.pts = slot + } } /// Hands out everything at the head of the queue whose slot is known. A picture still waiting for @@ -418,8 +458,9 @@ final class H264MatroskaSlotPermutationSession: H264TimestampRepairSession { // is stateful and must see each picture exactly once. for index in pending.indices where pending[index].isPicture { let entry = pending[index] - pending[index].rank = admit( - entry.packet, pictureOrderCount: entry.pictureOrderCount) + // Bound first: admitting an IDR settles the ranks already held in `pending`. + let rank = admit(entry.packet, pictureOrderCount: entry.pictureOrderCount) + pending[index].rank = rank } drain() case .healthy: diff --git a/Sources/AetherEngine/Video/H264SPS.swift b/Sources/AetherEngine/Video/H264SPS.swift index d01e7660f..8020b12f9 100644 --- a/Sources/AetherEngine/Video/H264SPS.swift +++ b/Sources/AetherEngine/Video/H264SPS.swift @@ -133,10 +133,16 @@ enum H264SPS { return nil } - /// Scan Annex-B for a coded IDR slice NAL (type 5). #133: a mid-stream live join must open on a true - /// IDR, not an open-GOP recovery-point I-slice (type 1) whose references precede the join point; starting - /// there renders garbage (green frames) until the next real IDR. Cheap NAL-header walk, no RBSP parse. - static func containsIDR(fromAnnexB data: UnsafeBufferPointer) -> Bool { + /// AE#627: whether a decoder can start at this access unit. An IDR slice qualifies, and so does a + /// picture coded only in I/SI slices behind a recovery point SEI with `recovery_frame_cnt == 0`: + /// every picture from it on in output order decodes exactly (H.264 D.2.8), and the leading + /// pictures that do not are dropped behind the gate. Some broadcast encoders never send an IDR, + /// so on those feeds a gate that required one never opened. A recovery point with a later + /// recovery frame (gradual refresh) is not an entry point and stays refused. + static func isRandomAccessEntry(fromAnnexB data: UnsafeBufferPointer) -> Bool { + var recoversHere = false + var sawSlice = false + var allIntra = true var i = 0 let n = data.count func startCode(at p: Int) -> Int { @@ -149,10 +155,49 @@ enum H264SPS { if sc == 0 { i += 1; continue } let start = i + sc guard start < n else { break } - if (data[start] & 0x1f) == 5 { return true } - i = start + var j = start + while j < n, startCode(at: j) == 0 { j += 1 } + switch data[start] & 0x1f { + case 5: + return true + case 1: + sawSlice = true + // first_mb_in_slice and slice_type sit in the first few bytes. + let head = unescape(Array(data[(start + 1).. Int? { + var p = 0 + func sei() -> Int? { + var v = 0 + while p < rbsp.count, rbsp[p] == 0xff { v += 255; p += 1 } + guard p < rbsp.count else { return nil } + v += Int(rbsp[p]); p += 1 + return v + } + while p < rbsp.count, rbsp[p] != 0x80 { + guard let type = sei(), let size = sei(), p + size <= rbsp.count else { return nil } + if type == 6 { + var r = BitReader(Array(rbsp[p..<(p + size)])) + return r.ue() + } + p += size + } + return nil } /// Build Annex-B extradata the mov muxer accepts directly; ff_isom_write_avcc sniffs the start code and packs avcC. @@ -161,7 +206,7 @@ enum H264SPS { return sc + sps + sc + pps } - private static func unescape(_ b: [UInt8]) -> [UInt8] { // remove emulation-prevention bytes (00 00 03 -> 00 00) + static func unescape(_ b: [UInt8]) -> [UInt8] { // remove emulation-prevention bytes (00 00 03 -> 00 00) var out = [UInt8](); out.reserveCapacity(b.count) var zeros = 0 var i = 0 diff --git a/Sources/AetherEngine/Video/HDR10PlusDetectionProbe.swift b/Sources/AetherEngine/Video/HDR10PlusDetectionProbe.swift index 1845ddb4e..29f2e810c 100644 --- a/Sources/AetherEngine/Video/HDR10PlusDetectionProbe.swift +++ b/Sources/AetherEngine/Video/HDR10PlusDetectionProbe.swift @@ -3,8 +3,9 @@ import AetherLibavformat import AetherLibavcodec import AetherLibavutil -/// Bounds for the pre-playback HDR10+ (ST 2094-40) carriage scan, `AetherEngine.probe(url:detecting:)` with -/// `.hdr10Plus`. +/// Bounds for the pre-playback dynamic HDR metadata scan, `AetherEngine.probe(url:detecting:)` with +/// `.hdr10Plus` (ST 2094-40) and/or `.hdrVivid` (CUVA). Both run in one pass over the same packets and +/// share these bounds. /// /// Separate from `LoadOptions` and from the lightweight `probe(url:)` path, exactly like /// `AtmosDetectionOptions`: a host opts into reading real video packets without the default probe changing @@ -25,6 +26,10 @@ public struct HDR10PlusDetectionOptions: Sendable, Equatable { /// This is the cap that actually binds on the content the feature targets: one UHD HEVC keyframe runs to /// several MB, so a handful of packets can exhaust it long before `maxPackets` does. A packet larger /// than the remaining budget stops the pass BEFORE inspection, even if it carries HDR10+. + /// + /// It also sets what the pass may read from the source in total, video or not: four times this + /// value, and never less than 4 MiB. That bound sits below the demuxer, so the blocks of the other + /// streams, which the demuxer reads and drops without ever handing them back as packets, count too. public var maxBytes: Int64 /// Soft wall-clock budget, checked before and after reads and after inspection. NOT preemptive: @@ -65,11 +70,22 @@ struct HDR10PlusDetectionOutcome: Sendable, Equatable { let stopReason: StopReason let packetsRead: Int let bytesRead: Int64 + /// What the pass confirmed. `.found` means every requested format was; a cap can still end a pass + /// that confirmed one of two, and that confirmation stands. Defaults to the single-target reading. + let found: ProbeDetail + + init(stopReason: StopReason, packetsRead: Int, bytesRead: Int64, found: ProbeDetail? = nil) { + self.stopReason = stopReason + self.packetsRead = packetsRead + self.bytesRead = bytesRead + self.found = found ?? (stopReason == .found ? .hdr10Plus : []) + } - /// A negative is never authoritative and is never published as one: only `.found` sets the flag, and the - /// flag is only ever set, never cleared. Every other reason means "not seen inside this budget", which - /// for a cap is genuinely inconclusive and for EOF is only as conclusive as the source is short. - var carriesHDR10Plus: Bool { stopReason == .found } + /// A negative is never authoritative and is never published as one: only a validated payload sets a + /// flag, and a flag is only ever set, never cleared. Not found means "not seen inside this budget", + /// which for a cap is genuinely inconclusive and for EOF is only as conclusive as the source is short. + var carriesHDR10Plus: Bool { found.contains(.hdr10Plus) } + var carriesHDRVivid: Bool { found.contains(.hdrVivid) } } /// Which extra, strictly more expensive detail a probe should resolve on top of the container metadata. @@ -86,6 +102,10 @@ public struct ProbeDetail: OptionSet, Sendable { /// HDR10+ (ST 2094-40) carriage via a bounded packet scan. See `HDR10PlusDetectionOptions`. public static let hdr10Plus = ProbeDetail(rawValue: 1 << 1) + + /// HDR Vivid (CUVA T/UWA 005.1) carriage in HEVC via the same bounded packet scan as `.hdr10Plus`, + /// sharing its `HDR10PlusDetectionOptions` budget. See `SourceProbe.carriesHDRVividMetadata`. + public static let hdrVivid = ProbeDetail(rawValue: 1 << 2) } extension AetherEngine { @@ -104,6 +124,15 @@ extension AetherEngine { return nil } + /// Source bytes the scan may consume, whatever stream they belong to. `maxBytes` alone bounds only + /// the video packets the scan sees; with the other streams at AVDISCARD_ALL the demuxer reads and + /// drops their blocks inside one `av_read_frame`, so a file with one video packet followed by + /// gigabytes of another stream would otherwise be read to its end. Saturating: `maxBytes` is public. + nonisolated static func hdr10PlusInputByteBudget(maxBytes: Int64) -> Int64 { + let (product, overflowed) = max(0, maxBytes).multipliedReportingOverflow(by: 4) + return overflowed ? .max : max(product, 4 * 1024 * 1024) + } + /// Packet ceiling for the AVDISCARD_ALL fuse, saturating rather than trapping: `maxPackets` is public and /// `Int.max` is a plausible "no limit" value to pass. nonisolated static func hdr10PlusForeignPacketFuse(maxPackets: Int) -> Int { @@ -122,8 +151,9 @@ extension AetherEngine { detected == .hdr10 ? .hdr10Plus : detected } - /// Bounded scan for HDR10+ carriage on `videoIndex`. Opens no decoder: it reads demuxed packets and asks - /// `HDR10PlusMetadataScan` about each one, stopping at the first hit or the first cap. + /// Bounded scan for HDR10+ and/or HDR Vivid carriage on `videoIndex` (`targets`). Opens no decoder: it + /// reads demuxed packets and asks `HDR10PlusMetadataScan` / `HDRVividMetadataScan` about each one, + /// stopping once every target is confirmed or at the first cap. /// /// Deliberately runs BEFORE any queue-flushing seek, unlike `detectAtmos`. `avformat_find_stream_info` /// leaves the packets it read queued, `av_read_frame` hands those back first, and at the head of a @@ -137,8 +167,10 @@ extension AetherEngine { demuxer: Demuxer, videoIndex: Int32, options: HDR10PlusDetectionOptions, + targets: ProbeDetail = .hdr10Plus, now: () -> UInt64 = { DispatchTime.now().uptimeNanoseconds } ) -> HDR10PlusDetectionOutcome { + let targets = targets.intersection([.hdr10Plus, .hdrVivid]) guard videoIndex >= 0, let stream = demuxer.stream(at: videoIndex), let codecpar = stream.pointee.codecpar, codecpar.pointee.codec_type == AVMEDIA_TYPE_VIDEO else { @@ -148,6 +180,12 @@ extension AetherEngine { // Matroska's BlockAdditional carriage is attached by the demuxer to the packet, so the scan needs the // packets themselves either way; dropping the other streams keeps the byte budget spent on video. demuxer.discardAllStreamsExcept([videoIndex]) + demuxer.beginInputByteBudget(Self.hdr10PlusInputByteBudget(maxBytes: options.maxBytes)) + defer { demuxer.endInputByteBudget() } + // A read the input budget refused surfaces as an error or as EOF, depending on the container. + func readEnded(_ reason: HDR10PlusDetectionOutcome.StopReason) -> HDR10PlusDetectionOutcome.StopReason { + demuxer.inputByteBudgetExhausted ? .byteCap : reason + } let start = now() func elapsed() -> TimeInterval { @@ -156,13 +194,15 @@ extension AetherEngine { var packetsRead = 0 var bytesRead: Int64 = 0 var packetsSeen = 0 + var found: ProbeDetail = [] let fuse = Self.hdr10PlusForeignPacketFuse(maxPackets: options.maxPackets) while true { if let cap = Self.hdr10PlusScanCapReached( packetsRead: packetsRead, bytesRead: bytesRead, elapsed: elapsed(), options: options ) { - return HDR10PlusDetectionOutcome(stopReason: cap, packetsRead: packetsRead, bytesRead: bytesRead) + return HDR10PlusDetectionOutcome( + stopReason: cap, packetsRead: packetsRead, bytesRead: bytesRead, found: found) } let packet: UnsafeMutablePointer? @@ -170,7 +210,7 @@ extension AetherEngine { packet = try demuxer.readPacket() } catch { return HDR10PlusDetectionOutcome( - stopReason: .demuxError, packetsRead: packetsRead, bytesRead: bytesRead) + stopReason: readEnded(.demuxError), packetsRead: packetsRead, bytesRead: bytesRead, found: found) } defer { if let packet { @@ -180,46 +220,52 @@ extension AetherEngine { } guard elapsed() < options.timeBudget else { return HDR10PlusDetectionOutcome( - stopReason: .timeCap, packetsRead: packetsRead, bytesRead: bytesRead) + stopReason: .timeCap, packetsRead: packetsRead, bytesRead: bytesRead, found: found) } guard let pkt = packet else { return HDR10PlusDetectionOutcome( - stopReason: .demuxEOF, packetsRead: packetsRead, bytesRead: bytesRead) + stopReason: readEnded(.demuxEOF), packetsRead: packetsRead, bytesRead: bytesRead, found: found) } - var found = false packetsSeen += 1 if pkt.pointee.stream_index == videoIndex { let packetBytes = Int64(pkt.pointee.size) guard packetBytes >= 0 else { return HDR10PlusDetectionOutcome( - stopReason: .demuxError, packetsRead: packetsRead, bytesRead: bytesRead) + stopReason: .demuxError, packetsRead: packetsRead, bytesRead: bytesRead, found: found) } guard packetBytes <= options.maxBytes - bytesRead else { return HDR10PlusDetectionOutcome( - stopReason: .byteCap, packetsRead: packetsRead, bytesRead: bytesRead) + stopReason: .byteCap, packetsRead: packetsRead, bytesRead: bytesRead, found: found) } packetsRead += 1 bytesRead += packetBytes - found = HDR10PlusMetadataScan.packetCarriesHDR10Plus(pkt, codecParameters: codecpar) + if targets.contains(.hdr10Plus), !found.contains(.hdr10Plus), + HDR10PlusMetadataScan.packetCarriesHDR10Plus(pkt, codecParameters: codecpar) { + found.insert(.hdr10Plus) + } + if targets.contains(.hdrVivid), !found.contains(.hdrVivid), + HDRVividMetadataScan.packetCarriesHDRVivid(pkt, codecParameters: codecpar) { + found.insert(.hdrVivid) + } } // Evidence in hand outranks the soft budget. The caps exist to bound what this pass SPENDS, // and a confirmation is already paid for; dropping it would turn an overrun into a false // negative on a slow origin, which the caller cannot tell apart from "this source has none". - if found { + if !targets.isEmpty, found.isSuperset(of: targets) { return HDR10PlusDetectionOutcome( - stopReason: .found, packetsRead: packetsRead, bytesRead: bytesRead) + stopReason: .found, packetsRead: packetsRead, bytesRead: bytesRead, found: found) } guard elapsed() < options.timeBudget else { return HDR10PlusDetectionOutcome( - stopReason: .timeCap, packetsRead: packetsRead, bytesRead: bytesRead) + stopReason: .timeCap, packetsRead: packetsRead, bytesRead: bytesRead, found: found) } // AVDISCARD_ALL is advisory, so a container that keeps handing back foreign packets still ends. if packetsSeen >= fuse { return HDR10PlusDetectionOutcome( - stopReason: .packetCap, packetsRead: packetsRead, bytesRead: bytesRead) + stopReason: .packetCap, packetsRead: packetsRead, bytesRead: bytesRead, found: found) } } } diff --git a/Sources/AetherEngine/Video/HDR10PlusMetadataScan.swift b/Sources/AetherEngine/Video/HDR10PlusMetadataScan.swift index 47755ce2c..4dd233c18 100644 --- a/Sources/AetherEngine/Video/HDR10PlusMetadataScan.swift +++ b/Sources/AetherEngine/Video/HDR10PlusMetadataScan.swift @@ -45,11 +45,11 @@ enum HDR10PlusMetadataScan { static func bytesCarryHDR10Plus( _ data: UnsafePointer?, size: Int, codecID: AVCodecID, framing: VideoNALFraming = .annexB ) -> Bool { - guard let data, size > 0 else { return false } + guard let data, size > 0, mayContainT35Header(data, size: size) else { return false } let bytes = UnsafeBufferPointer(start: data, count: size) switch codecID { case AV_CODEC_ID_H264, AV_CODEC_ID_HEVC: - return scanNALs(bytes, codecID: codecID, framing: framing) + return scanNALs(bytes, codecID: codecID, framing: framing, payload: validT35) case AV_CODEC_ID_AV1: return scanOBUs(bytes) default: @@ -57,14 +57,25 @@ enum HDR10PlusMetadataScan { } } + /// Audit BIT-4: exact prefilter. The playback scan runs on every video packet until its first hit, + /// so a source without HDR10+ paid a byte-wise NAL walk for the whole session. The T.35 header + /// holds no `00 00` pair, so no emulation-prevention byte can land inside it and AV1 does not + /// escape at all: a payload that carries HDR10+ in-band carries these six bytes verbatim. + static func mayContainT35Header(_ data: UnsafePointer, size: Int) -> Bool { + guard size >= t35Header.count else { return false } + return t35Header.withUnsafeBufferPointer { memmem(data, size, $0.baseAddress, $0.count) != nil } + } + // MARK: - H.264 / HEVC /// A walk that runs into malformed framing stops and reports what it had already VALIDATED, rather /// than discarding it. Only `validT35` ever sets the answer, so an aborted walk cannot invent one; /// abandoning a confirmed payload because a later NAL in the same packet is malformed would be a - /// false negative on the one carriage this scan exists to find. - private static func scanNALs( - _ bytes: UnsafeBufferPointer, codecID: AVCodecID, framing: VideoNALFraming + /// false negative on the one carriage this scan exists to find. `payload` validates one registered + /// T.35 message (its bytes start at the country code), so HDR Vivid shares this walk. + static func scanNALs( + _ bytes: UnsafeBufferPointer, codecID: AVCodecID, framing: VideoNALFraming, + payload: (UnsafePointer, Int) -> Bool ) -> Bool { var found = false func visit(_ start: Int, _ end: Int) -> Bool { @@ -81,7 +92,7 @@ enum HDR10PlusMetadataScan { } guard isSEI else { return true } guard let rbsp = unescape(bytes, start: start + headerSize, end: end), - let carriesMetadata = scanSEI(rbsp) else { return false } + let carriesMetadata = scanSEI(rbsp, payload: payload) else { return false } found = found || carriesMetadata return true } @@ -146,7 +157,7 @@ enum HDR10PlusMetadataScan { /// Nil denotes malformed SEI framing; false is a well-formed SEI without HDR10+. A message this /// walk has already VALIDATED outranks framing it cannot finish reading, so nil is only ever the /// answer when nothing was confirmed. - private static func scanSEI(_ bytes: [UInt8]) -> Bool? { + private static func scanSEI(_ bytes: [UInt8], payload: (UnsafePointer, Int) -> Bool) -> Bool? { var offset = 0 var found = false func extendedValue() -> Int? { @@ -167,7 +178,7 @@ enum HDR10PlusMetadataScan { size <= bytes.count - offset else { return found ? true : nil } if type == 4 { let valid = bytes.withUnsafeBufferPointer { - validT35($0.baseAddress! + offset, size: size) + payload($0.baseAddress! + offset, size) } found = found || valid } @@ -208,7 +219,7 @@ enum HDR10PlusMetadataScan { var trailing = end while trailing > offset, bytes[trailing - 1] == 0 { trailing -= 1 } guard trailing > offset, bytes[trailing - 1] == 0x80 else { return found } - found = validT35(bytes.baseAddress! + offset, size: trailing - offset - 1) || found + found = validT35(bytes.baseAddress! + offset, trailing - offset - 1) || found } } offset = end @@ -235,7 +246,7 @@ enum HDR10PlusMetadataScan { // MARK: - Complete ST 2094-40 validation - private static func validT35(_ bytes: UnsafePointer, size: Int) -> Bool { + private static func validT35(_ bytes: UnsafePointer, _ size: Int) -> Bool { guard size > t35Header.count, size - t35Header.count <= Int(AV_HDR_PLUS_MAX_PAYLOAD_SIZE), t35Header.indices.allSatisfy({ bytes[$0] == t35Header[$0] }), let metadata = av_dynamic_hdr_plus_alloc(nil) else { return false } diff --git a/Sources/AetherEngine/Video/HDRVividMetadataScan.swift b/Sources/AetherEngine/Video/HDRVividMetadataScan.swift new file mode 100644 index 000000000..77c78af1f --- /dev/null +++ b/Sources/AetherEngine/Video/HDRVividMetadataScan.swift @@ -0,0 +1,112 @@ +import Foundation +import AetherLibavcodec +import AetherLibavutil + +/// Decoder-free HDR Vivid (CUVA T/UWA 005.1) confirmation for the detail probe (#699). +/// +/// HDR Vivid's dynamic metadata rides a registered ITU-T T.35 SEI in HEVC: country code 0x26 (China), +/// terminal provider code 0x0004, provider oriented code 0x0005, then the CUVA body. No container +/// declares it and no demuxer lifts it into packet side data, so like HDR10+ it is only visible in the +/// packet payload. FFmpeg's own body parser is internal (`ff_parse_itu_t_t35_to_dynamic_hdr_vivid`), +/// so the body is walked here with the same field widths, and a message only counts when it is +/// complete and every bit after its last field is zero. A marker alone is not proof. +/// +/// HEVC only, matching libavcodec, which ignores this payload on every other codec. +enum HDRVividMetadataScan { + private static let t35Header: [UInt8] = [0x26, 0x00, 0x04, 0x00, 0x05] + + static func packetCarriesHDRVivid( + _ packet: UnsafePointer, + codecParameters: UnsafePointer + ) -> Bool { + let parameters = codecParameters.pointee + guard parameters.codec_id == AV_CODEC_ID_HEVC else { return false } + let framing: VideoNALFraming = NALUnitChain.lengthPrefixSize( + codecID: parameters.codec_id, + extradata: parameters.extradata, + extradataSize: Int(parameters.extradata_size) + ).map { .lengthPrefixed(size: $0) } ?? .annexB + return bytesCarryHDRVivid(packet.pointee.data, size: Int(packet.pointee.size), framing: framing) + } + + static func bytesCarryHDRVivid( + _ data: UnsafePointer?, size: Int, framing: VideoNALFraming = .annexB + ) -> Bool { + guard let data, size >= t35Header.count, + t35Header.withUnsafeBufferPointer({ memmem(data, size, $0.baseAddress, $0.count) != nil }) + else { return false } + return HDR10PlusMetadataScan.scanNALs( + UnsafeBufferPointer(start: data, count: size), + codecID: AV_CODEC_ID_HEVC, framing: framing, payload: validT35) + } + + private static func validT35(_ bytes: UnsafePointer, _ size: Int) -> Bool { + guard size > t35Header.count, + t35Header.indices.allSatisfy({ bytes[$0] == t35Header[$0] }) else { return false } + return bodyIsComplete(UnsafeBufferPointer(start: bytes + t35Header.count, count: size - t35Header.count)) + } + + /// The CUVA body, field for field as libavcodec's `dynamic_hdr_vivid.c` reads it. Only + /// `system_start_code` 1 to 7 has a defined syntax (T/UWA 005.1-2022, table 11), so any other value + /// is not evidence of this format. + static func bodyIsComplete(_ body: UnsafeBufferPointer) -> Bool { + var reader = BitReader(body) + guard let startCode = reader.read(8), (1...7).contains(startCode), + reader.skip(4 * 12), + let toneMapping = reader.read(1) else { return false } + if toneMapping == 1 { + guard let extraParams = reader.read(1) else { return false } + for _ in 0...extraParams { + guard reader.skip(12), let baseEnable = reader.read(1) else { return false } + if baseEnable == 1, !reader.skip(14 + 6 + 10 + 10 + 6 + 2 + 2 + 4 + 3 + 7) { return false } + guard let splineEnable = reader.read(1) else { return false } + if splineEnable == 1 { + guard let extraSplines = reader.read(1) else { return false } + for _ in 0...extraSplines { + guard let mode = reader.read(2) else { return false } + if mode == 0 || mode == 2, !reader.skip(8) { return false } + guard reader.skip(12 + 10 + 10 + 8) else { return false } + } + } + } + } + guard let saturationMapping = reader.read(1) else { return false } + if saturationMapping == 1 { + guard let gains = reader.read(3), reader.skip(gains * 8) else { return false } + } + return reader.remainderIsZero + } + + private struct BitReader { + let bytes: UnsafeBufferPointer + private(set) var position = 0 + + init(_ bytes: UnsafeBufferPointer) { self.bytes = bytes } + + mutating func read(_ count: Int) -> Int? { + guard count <= bytes.count * 8 - position else { return nil } + var value = 0 + for _ in 0..> 3] >> (7 - UInt8(position & 7))) & 1 + value = (value << 1) | Int(bit) + position += 1 + } + return value + } + + mutating func skip(_ count: Int) -> Bool { + guard count <= bytes.count * 8 - position else { return false } + position += count + return true + } + + var remainderIsZero: Bool { + var index = position + while index < bytes.count * 8 { + if (bytes[index >> 3] >> (7 - UInt8(index & 7))) & 1 != 0 { return false } + index += 1 + } + return true + } + } +} diff --git a/Sources/AetherEngine/Video/HLSSegmentProducer.swift b/Sources/AetherEngine/Video/HLSSegmentProducer.swift index a7bdb1482..47b790ac8 100644 --- a/Sources/AetherEngine/Video/HLSSegmentProducer.swift +++ b/Sources/AetherEngine/Video/HLSSegmentProducer.swift @@ -50,6 +50,12 @@ final class HLSSegmentProducer: @unchecked Sendable { /// which is only correct while the two agree; on a source where they do not, every walker /// downstream (A53 captions, the DV P7 RPU rewrite) reads the packet at the wrong offsets. let nalFramingOverride: VideoNALFraming? + /// Forwarded to `MP4SegmentMuxer.VideoConfig.convertsAnnexBSamples`. The packets stay Annex B + /// up to the muxer, so `nalFramingOverride` keeps describing them. + let convertsAnnexBSamples: Bool + /// The session's BIT-1 framing verdict, handed to every muxer this producer builds for the + /// program's own track (audit BIT-104). + let nalFramingLatch: NALFramingLatch? init( codecpar: UnsafePointer, @@ -59,7 +65,9 @@ final class HLSSegmentProducer: @unchecked Sendable { convertP7ToProfile81: Bool = false, colorOverride: MP4SegmentMuxer.ColorOverride? = nil, extradataOverride: [UInt8]? = nil, - nalFramingOverride: VideoNALFraming? = nil + nalFramingOverride: VideoNALFraming? = nil, + convertsAnnexBSamples: Bool = false, + nalFramingLatch: NALFramingLatch? = nil ) { self.codecpar = codecpar self.timeBase = timeBase @@ -69,6 +77,8 @@ final class HLSSegmentProducer: @unchecked Sendable { self.colorOverride = colorOverride self.extradataOverride = extradataOverride self.nalFramingOverride = nalFramingOverride + self.convertsAnnexBSamples = convertsAnnexBSamples + self.nalFramingLatch = nalFramingLatch } } @@ -203,6 +213,13 @@ final class HLSSegmentProducer: @unchecked Sendable { /// Fires synchronously on the pump thread per finalized live segment (index, duration, startSeconds, discontinuous). var onLiveSegmentFinalized: (@Sendable (Int, Double, Double, Bool) -> Void)? + /// AE#684: the sound a finalized live segment carries (index, first and last audio packet on the + /// output axis, seconds). Fired just before `onLiveSegmentFinalized` for the same index. + var onLiveSegmentSound: (@Sendable (Int, Double, Double) -> Void)? + /// Keyed by segment index; a nil value is a segment cut with no audio packet in it. + private var liveSegmentSoundByIndex: [Int: (first: Double, last: Double)?] = [:] + /// AE#684: set for the length of one merged read, so a second reader thread trips the assertion there. + private var mergedReadInFlight = false /// AE#443: the resident segment count the live runaway park may use, from the session that owns the /// window (`VideoSegmentProvider.liveResidentParkCap`). Unset leaves the static floor, which is the @@ -245,7 +262,10 @@ final class HLSSegmentProducer: @unchecked Sendable { /// Order-preserving funnel for sequential finalize reports. private func emitSequentialReport(index: Int, duration: Double) { - let base = seqNextReportIndex ?? index + // Anchored on the pump's first segment, not on the first report: a hole a long first GOP + // skipped is known before that segment's capture lands, and anchoring on the hole would + // leave every later report waiting for an index below it. + let base = seqNextReportIndex ?? min(index, baseIndex) seqNextReportIndex = base seqReadyReports[index] = duration var next = base @@ -282,9 +302,9 @@ final class HLSSegmentProducer: @unchecked Sendable { oldShift: Int64, fallbackDurationPts: Int64 ) -> (newShift: Int64, continuationDts: Int64) { - let lastOutputDts = lastSrcDts - oldShift - let continuationDts = lastOutputDts + max(fallbackDurationPts, 1) - return (srcDts - continuationDts, continuationDts) + let lastOutputDts = SourceTimestampBounds.difference(lastSrcDts, oldShift) + let continuationDts = SourceTimestampBounds.sum(lastOutputDts, max(fallbackDurationPts, 1)) + return (SourceTimestampBounds.difference(srcDts, continuationDts), continuationDts) } /// Derive audio shift so boundary packet lands exactly on the video seam regardless of source base. @@ -293,7 +313,7 @@ final class HLSSegmentProducer: @unchecked Sendable { audioBoundarySrcDts: Int64, seamOutAudioTb: Int64 ) -> Int64 { - audioBoundarySrcDts - seamOutAudioTb + SourceTimestampBounds.difference(audioBoundarySrcDts, seamOutAudioTb) } /// Raw source PTS of the previous video packet (before shift); used for live discontinuity detection. @@ -450,6 +470,14 @@ final class HLSSegmentProducer: @unchecked Sendable { private var firstSyncItemPtsBySegment: [Int: Int64] = [:] private var pendingAudioSegIndex: Int = 0 + /// A sequential origin's playlist advertises a segment from the VIDEO cut ledger (its EXTINF is + /// the distance between two keyframe-gated opens, and a plan index no keyframe opened is a hole + /// with no URI). Audio routed by time against the plan would open that hole on its own wherever + /// the source's GOP is longer than the stride, and the video after it would land in a file the + /// playlist never lists: a remote MKV with an 11 s first GOP lost 4 to 11 s that way and stalled + /// AVPlayer at the end of seg0. So there, as on live, audio follows the video cutter. + private var audioFollowsVideoCut: Bool { !isLive && onSequentialSegmentFinalized != nil } + /// VOD keyframe-gated cutter: opens each segment at the IRAP that reaches its plan boundary (#92). private var vodCutter: VODSegmentCutter @@ -499,6 +527,11 @@ final class HLSSegmentProducer: @unchecked Sendable { /// before `start()`; nil for hosts that drive the engine without one (`aetherctl`, tests). var sideReaderLinkGate: SideReaderLinkGate? + /// AE#514: the session's ledger of played-stream packet bytes, which the bitrate fields read at the + /// playhead. Set once before `start()`, like the gate above; nil for hosts that drive the engine + /// without a session (`aetherctl` probes, tests). + var playedMediaLedger: PlayedMediaLedger? + /// Forward-only producer restart counter; surfaced in live telemetry. Written on pump thread, read under packetCounterLock. var restartCount: Int { packetCounterLock.lock() @@ -534,6 +567,9 @@ final class HLSSegmentProducer: @unchecked Sendable { /// Pre-gate drop counters; surface the "lädt unendlich" failure mode when the gate never opens. private var pregateVideoDropCount: Int = 0 + /// AE#627: dropped packets that did carry the key flag. The wait line describes only the packet it + /// is printed on, so without this a join gate refusing every keyframe read as a feed with none. + private var pregateKeyDropCount: Int = 0 private var pregateWaitStart: Date? private static let liveKeyframeGateTimeoutSeconds: TimeInterval = 15 @@ -550,6 +586,21 @@ final class HLSSegmentProducer: @unchecked Sendable { private var pregateAudioBuffer: [(UnsafeMutablePointer, PacketOrigin)] = [] private var pregateAudioBufferBytes: Int = 0 private var pregateAudioReplaySorted = false + /// Audit SEG-6: replay cursor into `pregateAudioBuffer`. Entries before it were handed to the pump + /// (which owns and frees them); draining with removeFirst() was quadratic at TrueHD packet rates. + private var pregateAudioReplayIndex = 0 + + private var hasPregateAudioToReplay: Bool { pregateAudioReplayIndex < pregateAudioBuffer.count } + + private func freeUnreplayedPregateAudio() { + for entry in pregateAudioBuffer[pregateAudioReplayIndex...] { + var pkt: UnsafeMutablePointer? = entry.0 + trackedPacketFree(&pkt) + } + pregateAudioBuffer.removeAll() + pregateAudioReplayIndex = 0 + pregateAudioBufferBytes = 0 + } private var pregateAudioOverflowLogged = false private static let maxPregateAudioBufferBytes = 8 * 1024 * 1024 @@ -675,15 +726,27 @@ final class HLSSegmentProducer: @unchecked Sendable { /// Cascade, most specific first: the demuxer's own claim for this packet, then the last genuine /// delta this stream showed, then the frame duration the producer already carries for the last /// trun sample, then the historical single tick for a stream that has never carried a usable - /// timestamp at all. + /// timestamp at all. Each arm is capped at `maxStrideTicks` (audit SEG-101: a crafted + /// BlockDuration near the source bound walked nine packets to Int64.max), the same second + /// `updatedStrideTicks` already applies to a learned stride. static func repairStrideTicks(packetDuration: Int64, observedStride: Int64, - fallbackDuration: Int64) -> Int64 { - if packetDuration > 0 { return packetDuration } - if observedStride > 0 { return observedStride } - if fallbackDuration > 0 { return fallbackDuration } + fallbackDuration: Int64, maxStrideTicks: Int64 = .max) -> Int64 { + let cap = Swift.max(maxStrideTicks, 1) + if packetDuration > 0 { return Swift.min(packetDuration, cap) } + if observedStride > 0 { return Swift.min(observedStride, cap) } + if fallbackDuration > 0 { return Swift.min(fallbackDuration, cap) } return 1 } + /// Audit SEG-101: the dts a packet with neither timestamp gets, or nil when the walk would leave + /// the plausible source range. The packet is then dropped, and the next genuine timestamp + /// re-anchors the stream. + static func synthesizedDts(anchor: Int64, stride: Int64) -> Int64? { + let (dts, overflow) = anchor.addingReportingOverflow(stride) + guard !overflow, SourceTimestampBounds.plausible(dts) == dts else { return nil } + return dts + } + /// Adopts `dts - previousDts` as the stream's stride when it is forward and inside /// `maxStrideTicks`. A backward, standing or multi-second delta is a boundary or a replay, so /// the stride learned before it survives. @@ -750,7 +813,8 @@ final class HLSSegmentProducer: @unchecked Sendable { /// #65 watchdog: break a VOD backpressure park once the consumer fetch target has been frozen this long. /// Set above the log threshold so the diag tuple surfaces first. The host then re-anchors the producer on - /// AVPlayer's real position; a slow-but-advancing consumer never trips the detector (see BackpressureWedgeDetector). + /// AVPlayer's real position; a slow-but-advancing consumer never trips the detector, and seconds in which + /// the rendered clock advanced do not count toward it (AE#649; see BackpressureWedgeDetector). private static let backpressureWedgeBreakThresholdSeconds = 24 /// #207: a disk park is normal steady state for an opt-in prefetch, so it stays quiet until it has @@ -853,6 +917,14 @@ final class HLSSegmentProducer: @unchecked Sendable { defer { packetCounterLock.unlock() } return _lastPregateDroppedKeyframePts } + /// AE#627: video packets the gate dropped before a keyframe starvation ended the pump. Non-zero + /// means the source delivered video for the whole wait and none of it could open a segment. + private var _starvedVideoDrops = 0 + var starvedVideoDrops: Int { + packetCounterLock.lock() + defer { packetCounterLock.unlock() } + return _starvedVideoDrops + } var hasRestartTarget: Bool { restartTargetVideoPts != Int64.min } private func markVideoGateOpened() { packetCounterLock.lock() @@ -1091,15 +1163,11 @@ final class HLSSegmentProducer: @unchecked Sendable { /// audio belongs to the abandoned position, and the source-dts anchors would read the backward /// seek as a timeline discontinuity. private func discardPregateScanState() { - for entry in pregateAudioBuffer { - var pkt: UnsafeMutablePointer? = entry.0 - trackedPacketFree(&pkt) - } - pregateAudioBuffer.removeAll() - pregateAudioBufferBytes = 0 + freeUnreplayedPregateAudio() pregateAudioReplaySorted = false pregateAudioOverflowLogged = false pregateVideoDropCount = 0 + pregateKeyDropCount = 0 pregateAudioDropCount = 0 lastPregateVideoLog = 0 lastPregateAudioLog = 0 @@ -1261,8 +1329,9 @@ final class HLSSegmentProducer: @unchecked Sendable { private let a53CodecKind: A53SEIParser.CodecKind? private let a53NALFraming: A53SEIParser.NALFraming - /// #133: latched at init. When true, the video gate withholds until a decodable IDR access unit - /// (in-band SPS+PPS+IDR) arrives, rather than opening on any AV_PKT_FLAG_KEY packet. + /// #133: latched at init. When true, the video gate withholds until a decodable access unit + /// (in-band SPS+PPS and an IDR or immediate intra recovery point, AE#627) arrives, rather than + /// opening on any AV_PKT_FLAG_KEY packet. private let liveH264AnnexBJoin: Bool /// Sodalite#32: text-subtitle tap, generalizing the #77 CC tap. Streams in this set are kept by the @@ -1324,7 +1393,7 @@ final class HLSSegmentProducer: @unchecked Sendable { guard firstSeenDts != Int64.min, tbSeconds > 0 else { return nil } let windowTicks = Int64(Self.sourceReplayStartWindowSeconds / tbSeconds) if jumpTicks < 0 { - return newDts <= firstSeenDts + windowTicks ? .rewind : nil + return newDts <= SourceTimestampBounds.sum(firstSeenDts, windowTicks) ? .rewind : nil } guard isLive, newDts >= Self.mpegTSWrapTicks else { return nil } return newDts % Self.mpegTSWrapTicks <= windowTicks ? .axisReset : nil @@ -1666,6 +1735,7 @@ final class HLSSegmentProducer: @unchecked Sendable { } private func rotateMuxerForProgramSwitch(to newIdx: Int) -> MP4SegmentMuxer? { + if checkShouldStop() { return nil } let finishedIdx = currentMuxerSegmentIndex finalizeSessionMuxerAndAdopt() // adopts finishedIdx, nils currentMuxer pendingVideoProgramSwitch = false @@ -1722,14 +1792,15 @@ final class HLSSegmentProducer: @unchecked Sendable { return active != incoming } - /// #133 join gate: a decodable H.264 access unit at a mid-stream join needs in-band SPS + PPS and a true - /// IDR slice (not an open-GOP recovery point). Returns the reconstructed (width, height, Annex-B extradata) - /// so a zero-dimension probe codecpar can be backfilled into the first muxer. nil until such an AU arrives. + /// #133 join gate: a decodable H.264 access unit at a mid-stream join needs in-band SPS + PPS and an + /// entry point, an IDR or an immediate intra recovery point (AE#627, see `isRandomAccessEntry`). Returns + /// the reconstructed (width, height, Annex-B extradata) so a zero-dimension probe codecpar can be + /// backfilled into the first muxer. nil until such an AU arrives. private func extractJoinVideoConfig(_ packet: UnsafeMutablePointer) -> (width: Int32, height: Int32, extradata: [UInt8])? { guard let data = packet.pointee.data, packet.pointee.size > 0 else { return nil } let buf = UnsafeBufferPointer(start: data, count: Int(packet.pointee.size)) guard let (sps, pps) = H264SPS.extractSPSandPPS(fromAnnexB: buf), - H264SPS.containsIDR(fromAnnexB: buf), + H264SPS.isRandomAccessEntry(fromAnnexB: buf), let dim = H264SPS.dimensions(fromNAL: sps) else { return nil } return (Int32(dim.width), Int32(dim.height), H264SPS.annexBExtradata(sps: sps, pps: pps)) @@ -1800,12 +1871,16 @@ final class HLSSegmentProducer: @unchecked Sendable { // asking for segments, and stuck= is that number: 0 is a viewer scrubbing through // resident content, a climbing one is a consumer that went quiet. : wedgeDetector.secondsSinceTargetMoved == 0 ? "(consumer still fetching)" + // AE#649: quiet but rendering is a consumer playing out its forward buffer, + // and idle= (what the breaker counts) stays put while it does. + : wedgeDetector.lastPollRendered ? "(consumer quiet, still playing)" : "(no playback progress)" EngineLog.emit( "[HLSSegmentProducer] #65 backpressure PARK (\(context)) head=\(head) " + "target=\(target) cacheTarget=\(cacheTarget) " + "highStored=\(cache.highestStoredIndex) cached=\(cache.count) parked=\(parked)s " + "stuck=\(wedgeDetector.secondsSinceTargetMoved)s " + + "idle=\(wedgeDetector.secondsWithoutProgress)s " + suspendReason, category: .session ) @@ -2042,7 +2117,9 @@ final class HLSSegmentProducer: @unchecked Sendable { // parameter-set change is still the same program, so it keeps them (isAdCreative false). doviConfig: isAdCreative ? .keep : videoConfig.doviConfig, colorOverride: isAdCreative ? nil : videoConfig.colorOverride, - extradataOverride: isAdCreative ? nil : videoConfig.extradataOverride + extradataOverride: isAdCreative ? nil : videoConfig.extradataOverride, + convertsAnnexBSamples: isAdCreative ? false : videoConfig.convertsAnnexBSamples, + nalFramingLatch: isAdCreative ? nil : videoConfig.nalFramingLatch ) let muxerAudio: MP4SegmentMuxer.AudioConfig? = audioConfig.map { a in MP4SegmentMuxer.AudioConfig(codecpar: a.codecpar, timeBase: a.inputTimeBase, @@ -2287,6 +2364,10 @@ final class HLSSegmentProducer: @unchecked Sendable { private func advanceMuxer(to newIdx: Int) -> MP4SegmentMuxer? { guard let muxer = currentMuxer else { return nil } + // Audit SEG-5: an abandoned pump whose blocked read returns after stop() gets a packet from + // the NEW producer's position. Cutting on it would adopt this pump's partial segment under + // its full index; the teardown rule decides what happens to the in-flight segment instead. + if checkShouldStop() { return nil } switch muxer.cutFragmentForNextSegment(newIdx) { case .completed(let path, let bytesWritten): @@ -2302,6 +2383,11 @@ final class HLSSegmentProducer: @unchecked Sendable { // credit this pump with the previous epoch's production. pumpEpochHighestStored = max(pumpEpochHighestStored, currentMuxerSegmentIndex) if isLive { + let tb = muxer.muxerAudioTimeBase + let tick = tb.den > 0 ? Double(tb.num) / Double(tb.den) : 0 + liveSegmentSoundByIndex.updateValue( + muxer.takeSegmentSoundSpan().map { (Double($0.first) * tick, Double($0.last) * tick) }, + forKey: currentMuxerSegmentIndex) reportLiveSegmentFinalized(index: currentMuxerSegmentIndex, nextIndex: newIdx) } else if onSequentialSegmentFinalized != nil { @@ -2450,6 +2536,7 @@ final class HLSSegmentProducer: @unchecked Sendable { private func reportLiveSegmentFinalized(index: Int, nextIndex: Int?) { guard let startSeconds = liveSegmentStartByIndex[index] else { + liveSegmentSoundByIndex.removeValue(forKey: index) EngineLog.emit( "[HLSSegmentProducer] live finalize: no recorded start for seg-\(index); skipping append", category: .session @@ -2464,15 +2551,18 @@ final class HLSSegmentProducer: @unchecked Sendable { duration = targetSegmentDurationSeconds } let discontinuous = liveSegmentDiscontinuousByIndex[index] ?? false + let sound = liveSegmentSoundByIndex.removeValue(forKey: index) liveSegmentStartByIndex.removeValue(forKey: index) liveSegmentDiscontinuousByIndex.removeValue(forKey: index) stampLiveSegmentFinalize() EngineLog.emit( "[HLSSegmentProducer] live seg-\(index) finalized: start=\(String(format: "%.3f", startSeconds))s " + "dur=\(String(format: "%.3f", duration))s" + + (sound.map { $0.map { String(format: " sound=%.3f..%.3fs", $0.first, $0.last) } ?? " sound=none" } ?? "") + (discontinuous ? " [DISCONTINUITY]" : ""), category: .session ) + if let span = sound ?? nil { onLiveSegmentSound?(index, span.first, span.last) } onLiveSegmentFinalized?(index, duration, startSeconds, discontinuous) } @@ -2568,6 +2658,8 @@ final class HLSSegmentProducer: @unchecked Sendable { // MARK: - Public API func start() { + assert(!isLive || onLiveSegmentFinalized != nil, + "a live producer started without its provider reports cuts segments no playlist lists (audit HLS-101)") stateLock.lock() guard !pumpStarted else { stateLock.unlock(); return } pumpStarted = true @@ -2675,21 +2767,54 @@ final class HLSSegmentProducer: @unchecked Sendable { /// while playback listens to the bridged rendition. private func readNextSourcePacketMergedTapped() throws -> (packet: UnsafeMutablePointer, origin: PacketOrigin)? { let read = try readNextSourcePacketMerged() - if let read { tapForRecording(read.packet) } + if let read { + tapForRecording(read.packet) + recordPlayedMedia(read.packet, origin: read.origin) + } return read } + /// AE#514: the video stream and the audio stream this session plays, on the source axis the engine + /// folds AVPlayer's clock back onto (`source_pts - playlistShiftSeconds`). + private func recordPlayedMedia(_ packet: UnsafeMutablePointer, origin: PacketOrigin) { + guard let ledger = playedMediaLedger, packet.pointee.size > 0 else { return } + let index = packet.pointee.stream_index + let track: PlayedMediaLedger.Track + let timeBase: AVRational + if origin == .main, index == videoStreamIndex { + track = .video + timeBase = sourceVideoTimeBase + } else if let audio = audioConfig, index == audio.sourceStreamIndex, + (origin == .side) == (sideAudioDemuxer != nil) { + track = .audio + timeBase = audio.sourceTimeBase + } else { + return + } + let ticks = packet.pointee.pts != Int64.min ? packet.pointee.pts : packet.pointee.dts + guard ticks != Int64.min, timeBase.num > 0, timeBase.den > 0 else { return } + ledger.record(track, pts: Double(ticks) * Double(timeBase.num) / Double(timeBase.den), + bytes: Int(packet.pointee.size)) + } + private func readNextSourcePacketMerged() throws -> (packet: UnsafeMutablePointer, origin: PacketOrigin)? { + // AE#684: `HLSLiveIngestReader.pumpJoinIsSpent` reads "parked on EITHER reader" as "nothing + // more is cut", which holds only while one thread reads both of them in turn. + assert(!mergedReadInFlight, "the main and side readers are read by one thread, one at a time") + mergedReadInFlight = true + defer { mergedReadInFlight = false } guard let side = sideAudioDemuxer else { guard let packet = try demuxer.readPacket() else { return nil } + boundSourceTimestamps(packet) return (packet, .main) } if mergeMainLookahead == nil, !mergeMainEOF { mergeMainLookahead = try demuxer.readPacket() - if mergeMainLookahead == nil { mergeMainEOF = true } + if let pkt = mergeMainLookahead { boundSourceTimestamps(pkt) } else { mergeMainEOF = true } } if mergeSideLookahead == nil, !mergeSideEOF { mergeSideLookahead = try side.readPacket() + if let pkt = mergeSideLookahead { boundSourceTimestamps(pkt) } if mergeSideLookahead == nil { mergeSideEOF = true } else if packedSideAudioClock != nil, let pkt = mergeSideLookahead { @@ -2716,6 +2841,20 @@ final class HLSSegmentProducer: @unchecked Sendable { return (main, .main) } + /// Pump-thread-only: an out-of-range source timestamp was logged once for this producer. + private var implausibleTimestampLogged = false + + private func boundSourceTimestamps(_ packet: UnsafeMutablePointer) { + let pts = packet.pointee.pts, dts = packet.pointee.dts + guard SourceTimestampBounds.sanitize(packet), !implausibleTimestampLogged else { return } + implausibleTimestampLogged = true + EngineLog.emit( + "[HLSSegmentProducer] source packet on stream \(packet.pointee.stream_index) carries an " + + "out-of-range timestamp (pts=\(pts) dts=\(dts)), treated as unset", + category: .session + ) + } + /// Ordering key: dts when valid, else pts; AV_NOPTS_VALUE (Int64.min) yields immediately (NOPTS repair handles it downstream). private static func mergeOrderingTicks(_ packet: UnsafeMutablePointer) -> Int64 { if packet.pointee.dts != Int64.min { return packet.pointee.dts } @@ -2738,12 +2877,20 @@ final class HLSSegmentProducer: @unchecked Sendable { isLive: Bool, bufferedBytes: Int, packetSize: Int, - capBytes: Int + capBytes: Int, + bufferedCount: Int = 0, + maxEntries: Int = HLSSegmentProducer.maxPregateAudioBufferEntries ) -> Bool { guard isAudioPkt, audioWaitForVideo, isHeadOfStream || !isLive else { return false } - return bufferedBytes + max(packetSize, 0) <= capBytes + return bufferedCount < maxEntries && bufferedBytes + max(packetSize, 0) <= capBytes } + /// Audit SEG-102: the byte cap charges payload only, and each entry costs an `AVPacket`, a buffer + /// ref and an array slot besides, so 1-byte laced packets pinned about 1.4 GB under the 8 MiB cap + /// while the VOD gate waited unbounded. Above what the byte cap admits for AAC or AC-3, and still + /// about 22 minutes of 20 ms Opus frames. + static let maxPregateAudioBufferEntries = 65_536 + /// Overwrite packed side-audio timestamps with the synthesized program clock. /// KNOWN LIMITATION: free-running clock does NOT follow a live video rebase; A/V sync is lost from that boundary on. private func stampPackedSideAudio(_ packet: UnsafeMutablePointer) { @@ -2897,7 +3044,7 @@ final class HLSSegmentProducer: @unchecked Sendable { let packet: UnsafeMutablePointer let origin: PacketOrigin - if !audioWaitForVideo, !pregateAudioBuffer.isEmpty { + if !audioWaitForVideo, hasPregateAudioToReplay { // #74: once the video gate opens, drain the buffered head-of-stream audio in DTS // order before reading further source packets. These were already counted in // packetsRead when first read, so do not re-count them here. @@ -2905,7 +3052,12 @@ final class HLSSegmentProducer: @unchecked Sendable { pregateAudioBuffer.sort { Self.mergeOrderingTicks($0.0) < Self.mergeOrderingTicks($1.0) } pregateAudioReplaySorted = true } - let entry = pregateAudioBuffer.removeFirst() + let entry = pregateAudioBuffer[pregateAudioReplayIndex] + pregateAudioReplayIndex += 1 + if pregateAudioReplayIndex == pregateAudioBuffer.count { + pregateAudioBuffer.removeAll() + pregateAudioReplayIndex = 0 + } packet = entry.0 origin = entry.1 pregateAudioBufferBytes -= Int(packet.pointee.size) @@ -2913,6 +3065,14 @@ final class HLSSegmentProducer: @unchecked Sendable { guard let read = try readNextSourcePacket() else { break readLoop } + // Audit SEG-104: a stop that landed while this read was parked means the session + // has already replaced this pump, so what the read returned is not its to act on. + if checkShouldStop() { + var stale: UnsafeMutablePointer? = read.packet + trackedPacketFree(&stale) + exitReason = .stopRequested + break readLoop + } packet = read.packet origin = read.origin packetsRead += 1 @@ -3060,11 +3220,16 @@ final class HLSSegmentProducer: @unchecked Sendable { } else { // Neither timestamp survived. One tick per packet is not a presentation time, // it is a frozen clock; advance by this stream's frame interval instead. + let strideTb = isVideoPkt ? sourceVideoTbSeconds : audioSourceTbSeconds let stride = Self.repairStrideTicks( packetDuration: packet.pointee.duration, observedStride: isVideoPkt ? videoSourceStrideTicks : audioSourceStrideTicks, - fallbackDuration: isVideoPkt ? videoFallbackDurationPts : audioFallbackDurationPts) - packet.pointee.dts = anchor &+ stride + fallbackDuration: isVideoPkt ? videoFallbackDurationPts : audioFallbackDurationPts, + maxStrideTicks: strideTb > 0 ? Int64(Self.maxSynthesizedStrideSeconds / strideTb) : .max) + guard let synthesized = Self.synthesizedDts(anchor: anchor, stride: stride) else { + continue + } + packet.pointee.dts = synthesized packet.pointee.pts = packet.pointee.dts timestampsSynthesized = true noteSynthesizedTimestamp(strideTicks: stride, isVideo: isVideoPkt) @@ -3086,7 +3251,8 @@ final class HLSSegmentProducer: @unchecked Sendable { isLive: isLive, bufferedBytes: pregateAudioBufferBytes, packetSize: Int(packet.pointee.size), - capBytes: Self.maxPregateAudioBufferBytes + capBytes: Self.maxPregateAudioBufferBytes, + bufferedCount: pregateAudioBuffer.count ) { pregateAudioBuffer.append((packet, origin)) pregateAudioBufferBytes += Int(packet.pointee.size) @@ -3098,7 +3264,8 @@ final class HLSSegmentProducer: @unchecked Sendable { pregateAudioOverflowLogged = true EngineLog.emit( "[HLSSegmentProducer] pre-gate audio buffer hit the " - + "\(Self.maxPregateAudioBufferBytes)-byte cap; dropping further leading audio " + + "\(Self.maxPregateAudioBufferBytes)-byte / \(Self.maxPregateAudioBufferEntries)-entry " + + "cap; dropping further leading audio " + "(wide interleave beyond cap)", category: .session ) @@ -3110,7 +3277,7 @@ final class HLSSegmentProducer: @unchecked Sendable { // adds #EXT-X-DISCONTINUITY at the seam. if rebasesTimelineOnDiscontinuity, isVideoPkt, lastVideoSourceDts != Int64.min, videoShiftPts != Int64.min, packet.pointee.dts != Int64.min { - let jumpTicks = packet.pointee.dts - lastVideoSourceDts + let jumpTicks = SourceTimestampBounds.difference(packet.pointee.dts, lastVideoSourceDts) let thresholdSeconds = jumpTicks < 0 ? Self.discontinuityBackwardThresholdSeconds : Self.discontinuityThresholdSeconds @@ -3190,7 +3357,7 @@ final class HLSSegmentProducer: @unchecked Sendable { if rebasesTimelineOnDiscontinuity, isAudioPkt, lastAudioSourceDts != Int64.min, audioShiftPts != Int64.min, packet.pointee.dts != Int64.min, let audio = audioConfig { - let jumpTicks = packet.pointee.dts - lastAudioSourceDts + let jumpTicks = SourceTimestampBounds.difference(packet.pointee.dts, lastAudioSourceDts) let tb = audio.sourceTimeBase let thresholdSeconds = jumpTicks < 0 ? Self.discontinuityBackwardThresholdSeconds @@ -3215,10 +3382,10 @@ final class HLSSegmentProducer: @unchecked Sendable { ) pendingAudioShiftOverride = nil } - let lastOutputDts = lastAudioSourceDts - audioShiftPts + let lastOutputDts = SourceTimestampBounds.difference(lastAudioSourceDts, audioShiftPts) // Independent measurement (audio-first boundary): used directly only when no video-derived shift available. - let measuredShift = packet.pointee.dts - - (lastOutputDts + max(audioFallbackDurationPts, 1)) + let measuredShift = SourceTimestampBounds.difference( + packet.pointee.dts, SourceTimestampBounds.sum(lastOutputDts, max(audioFallbackDurationPts, 1))) var newShift = measuredShift var inherited = false if let p = pendingAudioInheritSeamOut, @@ -3230,7 +3397,7 @@ final class HLSSegmentProducer: @unchecked Sendable { ) if let bridge = audio.bridge { // Bridge: free-running encoder restamps continuously; jump its timeline by the residual gap. - let driftTicks = measuredShift - candidate + let driftTicks = SourceTimestampBounds.difference(measuredShift, candidate) let tbSec = tb.den > 0 ? Double(tb.num) / Double(tb.den) : 0 bridge.noteTimelineJump( @@ -3241,14 +3408,14 @@ final class HLSSegmentProducer: @unchecked Sendable { // Stream-copy: apply candidate verbatim (absolute, not clamped to lastOutputDts). // Delta handoff accumulated A/V drift across SSAI pod creatives (device symptom: seconds late by content return). // Sub-frame overlap at the seam left to OutputTimestampSanitizer; > 0.5 s re-anchors. - let firstOutputDts = packet.pointee.dts - candidate + let firstOutputDts = SourceTimestampBounds.difference(packet.pointee.dts, candidate) let overlapTicks = lastOutputDts - firstOutputDts let maxOverlapTicks = audio.sourceTimeBase.num > 0 ? Int64(0.5 * Double(audio.sourceTimeBase.den) / Double(audio.sourceTimeBase.num)) : Int64.max if overlapTicks > maxOverlapTicks { - newShift = packet.pointee.dts - lastOutputDts - 1 + newShift = SourceTimestampBounds.difference(SourceTimestampBounds.difference(packet.pointee.dts, lastOutputDts), 1) EngineLog.emit( "[HLSSegmentProducer] audio rebase inherit re-anchored: " + "candidate=\(candidate) overlap=\(overlapTicks) ticks " @@ -3285,7 +3452,7 @@ final class HLSSegmentProducer: @unchecked Sendable { if Date().timeIntervalSince(override_.at) < Self.rebasePairingWindowSeconds { if let bridge = audio.bridge { // Bridge: residual between applied and video-derived shift becomes an encoder-timeline jump. - let driftTicks = audioShiftPts - derivedShift + let driftTicks = SourceTimestampBounds.difference(audioShiftPts, derivedShift) let tbSec = tb.den > 0 ? Double(tb.num) / Double(tb.den) : 0 bridge.noteTimelineJump( @@ -3298,14 +3465,14 @@ final class HLSSegmentProducer: @unchecked Sendable { } else { // Stream-copy: apply seam-derived shift; sub-frame overlap left to OutputTimestampSanitizer; // only > 0.5 s overlap re-anchors. - let lastOutputDts = lastAudioSourceDts - audioShiftPts - let firstOutputDts = override_.boundarySrcDts - derivedShift + let lastOutputDts = SourceTimestampBounds.difference(lastAudioSourceDts, audioShiftPts) + let firstOutputDts = SourceTimestampBounds.difference(override_.boundarySrcDts, derivedShift) let overlapTicks = lastOutputDts - firstOutputDts let maxOverlapTicks = tb.num > 0 ? Int64(0.5 * Double(tb.den) / Double(tb.num)) : Int64.max let applied = overlapTicks > maxOverlapTicks - ? packet.pointee.dts - lastOutputDts - 1 + ? SourceTimestampBounds.difference(SourceTimestampBounds.difference(packet.pointee.dts, lastOutputDts), 1) : derivedShift EngineLog.emit( "[HLSSegmentProducer] audio rebase corrected to video-derived shift: " @@ -3333,9 +3500,9 @@ final class HLSSegmentProducer: @unchecked Sendable { if isVideoPkt, lastVideoSourceDts != Int64.min, packet.pointee.dts != Int64.min, packet.pointee.dts <= lastVideoSourceDts, - lastVideoSourceDts - packet.pointee.dts <= monoGlitchVideoTicks { + SourceTimestampBounds.difference(lastVideoSourceDts, packet.pointee.dts) <= monoGlitchVideoTicks { let original = packet.pointee.dts - let bumped = lastVideoSourceDts + 1 + let bumped = SourceTimestampBounds.sum(lastVideoSourceDts, 1) let ptsValid = packet.pointee.pts != Int64.min if !ptsValid || bumped <= packet.pointee.pts { packet.pointee.dts = bumped @@ -3371,12 +3538,12 @@ final class HLSSegmentProducer: @unchecked Sendable { if isAudioPkt, lastAudioSourceDts != Int64.min, packet.pointee.dts != Int64.min, packet.pointee.dts <= lastAudioSourceDts, - lastAudioSourceDts - packet.pointee.dts <= monoGlitchAudioTicks { + SourceTimestampBounds.difference(lastAudioSourceDts, packet.pointee.dts) <= monoGlitchAudioTicks { // Same logic for audio. Audio doesn't have B-frame // pts/dts skew so dts <= pts isn't a useful gate; // just bump. let original = packet.pointee.dts - packet.pointee.dts = lastAudioSourceDts + 1 + packet.pointee.dts = SourceTimestampBounds.sum(lastAudioSourceDts, 1) if !loggedFirstAudioDtsBump { loggedFirstAudioDtsBump = true EngineLog.emit( @@ -3424,9 +3591,9 @@ final class HLSSegmentProducer: @unchecked Sendable { pts: packet.pointee.pts, dts: packet.pointee.dts, targetPts: effectiveGateTargetPts) // #133: on a live H.264 Annex-B mid-stream join, opening on a bare keyframe flag is not - // enough. A join packet must carry a decodable IDR access unit (in-band SPS+PPS+IDR); - // otherwise the decoder renders references it never received (green frames) or, when the - // probe joined before any SPS and left codecpar at 0x0, the first muxer alloc gets 0x0 + // enough. A join packet must carry a decodable access unit (in-band SPS+PPS and an + // IDR or immediate intra recovery point, AE#627); otherwise the decoder renders references + // it never received (green frames) or, when the probe joined before any SPS and left codecpar at 0x0, the first muxer alloc gets 0x0 // dimensions and avformat_write_header fails -22, dead-ending the channel. The bounded // live timeout below covers the miss (keyframeStarvation -> reopen), unlike muxerFailed. let joinConfig = (liveH264AnnexBJoin && isKey && targetSatisfied) @@ -3437,6 +3604,7 @@ final class HLSSegmentProducer: @unchecked Sendable { let ts = packet.pointee.pts != Int64.min ? packet.pointee.pts : packet.pointee.dts if ts != Int64.min { notePregateDroppedKeyframe(pts: ts) } + pregateKeyDropCount += 1 } // AE#408: the scan has reached ground already known to carry no sync // sample (the boundary itself on the first pass, the abandoned aim after @@ -3462,10 +3630,10 @@ final class HLSSegmentProducer: @unchecked Sendable { if pregateVideoDropCount - lastPregateVideoLog >= Self.pregateLogInterval { lastPregateVideoLog = pregateVideoDropCount let awaiting = (isKey && targetSatisfied && liveH264AnnexBJoin) - ? "SPS/PPS/IDR access unit" : "video keyframe" + ? "SPS/PPS + entry-point access unit" : "video keyframe" EngineLog.emit( "[HLSSegmentProducer] still waiting for \(awaiting): " - + "dropped=\(pregateVideoDropCount) " + + "dropped=\(pregateVideoDropCount) (keyframes=\(pregateKeyDropCount)) " + "lastDts=\(packet.pointee.dts) lastPts=\(packet.pointee.pts) " + "isKey=\(isKey) " + "target=\(effectiveGateTargetPts)" @@ -3481,9 +3649,13 @@ final class HLSSegmentProducer: @unchecked Sendable { EngineLog.emit( "[HLSSegmentProducer] live keyframe gate timed out after " + "\(Int(Self.liveKeyframeGateTimeoutSeconds))s " - + "(dropped=\(pregateVideoDropCount)); exiting pump for reopen", + + "(dropped=\(pregateVideoDropCount), keyframes=\(pregateKeyDropCount)); " + + "exiting pump for reopen", category: .session ) + packetCounterLock.lock() + _starvedVideoDrops = pregateVideoDropCount + packetCounterLock.unlock() exitReason = .keyframeStarvation break readLoop } @@ -3540,7 +3712,7 @@ final class HLSSegmentProducer: @unchecked Sendable { actualFirstDts: firstActualVideoDts, desiredTfdtPts: desiredFirstVideoTfdtPts, planAnchorPts: planAnchorVideoPts) - videoShiftPts = firstActualVideoDts - pinnedTfdtPts + videoShiftPts = SourceTimestampBounds.difference(firstActualVideoDts, pinnedTfdtPts) if audioWaitForVideo, let audio = audioConfig { // Rescale into SOURCE audio TB (not encoder TB): FLAC bridge exposes this mismatch; // using inputTimeBase landed the target 48x too far for bridged DTS sources. @@ -3727,7 +3899,7 @@ final class HLSSegmentProducer: @unchecked Sendable { } else { gapInAudioTb = restartTargetAudioDts == Int64.min ? 0 - : firstActualAudioDts - restartTargetAudioDts + : SourceTimestampBounds.difference(firstActualAudioDts, restartTargetAudioDts) } let gapMs = audioTb.den > 0 ? Double(gapInAudioTb) * Double(audioTb.num) * 1000.0 / Double(audioTb.den) @@ -3761,7 +3933,7 @@ final class HLSSegmentProducer: @unchecked Sendable { packet.pointee.pts != Int64.min { let rawPts = packet.pointee.pts if lastRawVideoPts != Int64.min { - let deltaTicks = rawPts - lastRawVideoPts + let deltaTicks = SourceTimestampBounds.difference(rawPts, lastRawVideoPts) let deltaSeconds = Double(deltaTicks) * sourceVideoTbSeconds if abs(deltaSeconds) >= Self.discontinuityThresholdSeconds { pendingDiscontinuityFlag = true @@ -3785,10 +3957,10 @@ final class HLSSegmentProducer: @unchecked Sendable { let activeShift: Int64 = isVideoPkt ? videoShiftPts : audioShiftPts if activeShift != Int64.min && activeShift != 0 { if packet.pointee.dts != Int64.min { - packet.pointee.dts -= activeShift + packet.pointee.dts = SourceTimestampBounds.difference(packet.pointee.dts, activeShift) } if packet.pointee.pts != Int64.min { - packet.pointee.pts -= activeShift + packet.pointee.pts = SourceTimestampBounds.difference(packet.pointee.pts, activeShift) } } @@ -3981,18 +4153,7 @@ final class HLSSegmentProducer: @unchecked Sendable { trackedPacketFree(&fpVar) continue } - // Rescale FLAC pts to source video TB for segment lookup; live audio follows video cutter. - let fpSeg: Int - if isLive { - fpSeg = liveCurrentSegmentIndex - } else { - let fpPtsInVideoTb = av_rescale_q( - fp.pointee.pts, - audio.inputTimeBase, - sourceVideoTimeBase - ) - fpSeg = segmentIndex(forSourcePts: fpPtsInVideoTb) - } + let fpSeg = bridgedAudioSegmentIndex(fp, audio: audio) guard let muxer = ensureMuxer(forSegmentIndex: fpSeg) else { trackedPacketFree(&fpVar) bridgedMuxerGone = true @@ -4013,10 +4174,12 @@ final class HLSSegmentProducer: @unchecked Sendable { continue } if audio.stripAacAdts { Self.stripADTSHeader(packet) } - let thisAudioSeg: Int = isLive ? liveCurrentSegmentIndex : 0 + let thisAudioSeg: Int = isLive + ? liveCurrentSegmentIndex + : (audioFollowsVideoCut ? vodCutter.current : 0) if let prev = pendingAudioPkt { let prevSeg: Int - if isLive { + if isLive || audioFollowsVideoCut { prevSeg = pendingAudioSegIndex } else { let prevPtsInVideoTb = av_rescale_q( @@ -4037,7 +4200,7 @@ final class HLSSegmentProducer: @unchecked Sendable { } } pendingAudioPkt = packet - if isLive { pendingAudioSegIndex = thisAudioSeg } + if isLive || audioFollowsVideoCut { pendingAudioSegIndex = thisAudioSeg } pktPtr = nil continue } @@ -4050,6 +4213,11 @@ final class HLSSegmentProducer: @unchecked Sendable { lastError = -1 exitReason = .readError(code: -1) } + // Audit HLS-1: a stop aborts a parked read through markClosed, which surfaces here. + stateLock.lock() + let stopped = shouldStop + stateLock.unlock() + if stopped { exitReason = .stopRequested } EngineLog.emit( "[HLSSegmentProducer] demuxer.readPacket threw: \(error)", category: .session @@ -4064,6 +4232,11 @@ final class HLSSegmentProducer: @unchecked Sendable { if case .stopRequested = exitReason {} else { exitReason = .segmentStall } } + // Audit SEG-104: the abort of a parked read surfaces as end of file, which is not the source's. + if case .eof = exitReason, checkShouldStop() { + exitReason = .stopRequested + } + // muxerFailed from a backpressure break is a wedge (host re-anchors) or a stop (teardown), not a real failure. if case .muxerFailed = exitReason { stateLock.lock() @@ -4093,12 +4266,7 @@ final class HLSSegmentProducer: @unchecked Sendable { // #74: free any head-of-stream audio still buffered (e.g. the video gate never opened on a // corrupt or aborted source); replayed entries were already drained at the loop top. - for entry in pregateAudioBuffer { - var pkt: UnsafeMutablePointer? = entry.0 - trackedPacketFree(&pkt) - } - pregateAudioBuffer.removeAll() - pregateAudioBufferBytes = 0 + freeUnreplayedPregateAudio() // Flush look-behind; fallback duration produces tail-correct trun for the final fragment. if let prev = pendingVideoPkt { @@ -4114,7 +4282,7 @@ final class HLSSegmentProducer: @unchecked Sendable { } if let prev = pendingAudioPkt, let audio = audioConfig { let prevSeg: Int - if isLive { + if isLive || audioFollowsVideoCut { prevSeg = pendingAudioSegIndex } else { let prevPtsInVideoTb = av_rescale_q( @@ -4136,17 +4304,7 @@ final class HLSSegmentProducer: @unchecked Sendable { // EOF tail flush for bridge audio: drains ~100-200 ms remainder (per-feed only emits full frames). if case .eof = exitReason, let audio = audioConfig, let bridge = audio.bridge { for fp in bridge.flush() { - let fpSeg: Int - if isLive { - fpSeg = liveCurrentSegmentIndex - } else { - let fpPtsInVideoTb = av_rescale_q( - fp.pointee.pts, - audio.inputTimeBase, - sourceVideoTimeBase - ) - fpSeg = segmentIndex(forSourcePts: fpPtsInVideoTb) - } + let fpSeg = bridgedAudioSegmentIndex(fp, audio: audio) if let muxer = ensureMuxer(forSegmentIndex: fpSeg) { fp.pointee.stream_index = muxer.audioOutputStreamIndex av_packet_rescale_ts(fp, audio.inputTimeBase, muxer.muxerAudioTimeBase) @@ -4179,6 +4337,16 @@ final class HLSSegmentProducer: @unchecked Sendable { onPumpFinished?(exitReason) } + /// The segment a bridged audio packet belongs to. Live and sequential audio follow the video + /// cutter; everything else is routed by the packet's own time against the plan. One helper for the + /// per-feed and the end-of-file flush, because the flush missing the sequential rule opened a plan + /// index the cutter never reached and cost the archive its final segment (audit SEG-103). + private func bridgedAudioSegmentIndex(_ packet: UnsafeMutablePointer, audio: AudioConfig) -> Int { + if isLive { return liveCurrentSegmentIndex } + if audioFollowsVideoCut { return vodCutter.current } + return segmentIndex(forSourcePts: av_rescale_q(packet.pointee.pts, audio.inputTimeBase, sourceVideoTimeBase)) + } + // MARK: - Look-behind finalize helpers /// Sample duration (source video TB) written for a look-behind video packet. Always telescope to @@ -4207,7 +4375,7 @@ final class HLSSegmentProducer: @unchecked Sendable { capTicks: Int64 ) -> Int64 { if let next = nextDts, dts != Int64.min, next != Int64.min { - let inferred = next - dts + let inferred = SourceTimestampBounds.difference(next, dts) if inferred > 0, inferred <= capTicks { return inferred } } return existingDuration > 0 && existingDuration <= capTicks ? existingDuration : fallback @@ -4343,7 +4511,7 @@ final class HLSSegmentProducer: @unchecked Sendable { ) { if packet.pointee.duration <= 0 { if let next = nextDts { - let inferred = next - packet.pointee.dts + let inferred = SourceTimestampBounds.difference(next, packet.pointee.dts) packet.pointee.duration = inferred > 0 ? inferred : audioFallbackDurationPts } else { packet.pointee.duration = audioFallbackDurationPts diff --git a/Sources/AetherEngine/Video/HLSVideoEngine+AudioRoute.swift b/Sources/AetherEngine/Video/HLSVideoEngine+AudioRoute.swift index 00a2097d5..e64178023 100644 --- a/Sources/AetherEngine/Video/HLSVideoEngine+AudioRoute.swift +++ b/Sources/AetherEngine/Video/HLSVideoEngine+AudioRoute.swift @@ -131,6 +131,26 @@ extension HLSVideoEngine { hadSourceAudioStream ? .droppedNoPipeline : .noAudioInSource } + /// AE#641: the bridge decoded not one frame of the selected stream. A FLAC sample entry is built from + /// the encoder's extradata, so no muxer death follows on that route: a live session serves an + /// audio track that is never filled and AVPlayer waits on it, a VOD session plays silently while + /// reporting `.bridged`. Live hands it to the engine, which rebuilds video-only; VOD surfaces + /// the same verdict the E-AC-3 route reaches through its failed cut, so a host can hand the + /// position to a player that decodes the track itself. + func handleBridgeDecodedNothing(streamIndex: Int32, summary: String) { + if isLiveSession { + onLiveAudioDecodesNothing?(streamIndex, summary) + return + } + EngineLog.emit( + "[HLSVideoEngine] AE#641 the VOD audio bridge decoded nothing from stream \(streamIndex) " + + "(\(summary)); surfacing it instead of playing silently", + category: .session + ) + surfaceVODSourceFailure(FFmpegErr.einval, "Audio track could not be decoded", + kind: .audioBridgeProducedNoOutput) + } + /// Guards `audioSourceStreamIndexOverride` against stale picker selections from a previous title. static func isAudioStream(demuxer: Demuxer, index: Int32) -> Bool { guard index >= 0, let stream = demuxer.stream(at: index) else { @@ -147,6 +167,7 @@ extension HLSVideoEngine { streamCopyAudio: HLSSegmentProducer.AudioConfig?, sourceAudioStreamIndex: Int32, sourceAudioStream: UnsafeMutablePointer?, + sourceCarriesAudio: Bool, audioHLSCodecs: inout String?, audioLanguage: String? = nil ) throws -> HLSSegmentProducer { @@ -160,13 +181,25 @@ extension HLSVideoEngine { // AE#462 harness (TEST-ONLY): both attempts are skipped so the video-only tail is reachable // without a source this build has no decoder for. Loud, because a run that read as a real // classification would be worse than no harness at all. - let forcedDrop = AetherEngine.forceAudioPipelineFailureForTesting - if forcedDrop { + let forcedForTesting = AetherEngine.forceAudioPipelineFailureForTesting + if forcedForTesting { EngineLog.emit( "[HLSVideoEngine] TEST-ONLY: audio pipeline forced to fail, skipping stream-copy and bridge", category: .session ) } + // AE#641: this session's bridge already decoded nothing from this stream, and a second + // bridge would be handed the same bytes. + let knownUndecodable = Self.isKnownUndecodable( + undecodableAudioStreamIndices, sourceAudioStreamIndex: sourceAudioStreamIndex) + if knownUndecodable { + EngineLog.emit( + "[HLSVideoEngine] AE#641 audio stream \(sourceAudioStreamIndex) decoded nothing earlier " + + "in this session; skipping stream-copy and bridge", + category: .session + ) + } + let forcedDrop = forcedForTesting || knownUndecodable let sourceCodecLabel: String = { // falls back to "audio" for codecs with no libavcodec name entry if let stream = sourceAudioStream, @@ -186,7 +219,8 @@ extension HLSVideoEngine { codecTagOverride: vcfg.codecTagOverride, doviConfig: vcfg.doviConfig, colorOverride: vcfg.colorOverride, - extradataOverride: vcfg.extradataOverride + extradataOverride: vcfg.extradataOverride, + convertsAnnexBSamples: vcfg.convertsAnnexBSamples ) let probeAudio = MP4SegmentMuxer.AudioConfig( codecpar: cfg.codecpar, @@ -313,6 +347,12 @@ extension HLSVideoEngine { ) self.savedAudioConfig = cfg self.audioBridge = bridge + if sideAudioDemuxer == nil { + let streamIndex = sourceAudioStreamIndex + bridge.onDecoderProducedNothing = { [weak self] stats in + self?.handleBridgeDecodedNothing(streamIndex: streamIndex, summary: stats.summary) + } + } do { let prod = try makeProducer(baseIndex: initialProducerBaseIndex) // The label and the CODECS attribute come from the encoder the bridge ACTUALLY opened, @@ -359,8 +399,12 @@ extension HLSVideoEngine { // AE#462: the drop becomes a fact the host can read, and a line that says which of the two // silences this is. The ERROR lines above name a failure; this one names the outcome, which // is what a reader of the log is actually looking for. + // AE#641: whether the SOURCE has audio, not whether a stream was picked. `av_find_best_stream` + // passes over an audio stream whose parameters the probe left empty, and on VOD nothing falls + // back to it, so a source with an undecodable track used to read as one without audio. self.audioDelivery = Self.videoOnlyAudioDelivery( - hadSourceAudioStream: sourceAudioStream != nil && sourceAudioStreamIndex >= 0) + hadSourceAudioStream: sourceCarriesAudio + || (sourceAudioStream != nil && sourceAudioStreamIndex >= 0)) EngineLog.emit( "[HLSVideoEngine] audio delivery = \(audioDelivery.rawValue)" + (audioDelivery == .droppedNoPipeline diff --git a/Sources/AetherEngine/Video/HLSVideoEngine+LiveReopen.swift b/Sources/AetherEngine/Video/HLSVideoEngine+LiveReopen.swift index 617085ea9..3bbb10274 100644 --- a/Sources/AetherEngine/Video/HLSVideoEngine+LiveReopen.swift +++ b/Sources/AetherEngine/Video/HLSVideoEngine+LiveReopen.swift @@ -155,6 +155,27 @@ extension HLSVideoEngine { } } + /// AE#627: a keyframe starvation on the session's FIRST join (no segment ever produced) while video + /// kept arriving says the bitstream has no entry point this route accepts, not that the link + /// hiccupped. The same starvation after segments were produced is a mid-session loss and keeps its + /// reopens, and a wait that saw no video at all is a source problem a reopen can fix. + static func liveJoinFoundNoEntryPoint( + reason: HLSSegmentProducer.PumpExitReason, + segmentsProduced: Int, + starvedVideoDrops: Int + ) -> Bool { + guard case .keyframeStarvation = reason else { return false } + return segmentsProduced == 0 && starvedVideoDrops > 0 + } + + /// AE#627: the engine's answer to `onLiveJoinWithoutEntryPoint` when the software path is not on + /// offer. The reopens this skipped would have ended here too, a minute later. + func giveUpLiveJoinWithoutEntryPoint() { + escalateLiveReopenExhaustion(transport: Self.liveReopenTransport( + sourceReopenableByURL: sourceReopenableByURL, + hasCustomSourceReopenFactory: customSourceReopenFactory != nil)) + } + /// The whole escalation, so the two exhaustion sites (barren-cycle cap, reopen attempt cap) /// cannot drift apart: the decision plus BOTH of its effects. The halt is the half that is easy /// to lose, and losing it is what the -15410 zombie is made of; onLiveSourceReset alone (what @@ -167,6 +188,21 @@ extension HLSVideoEngine { func handlePumpFinished(_ prod: HLSSegmentProducer, reason: HLSSegmentProducer.PumpExitReason) { + // Audit HLS-1: a superseded pump (the #79 markClosed of a wedged read, or stop()) reports + // an aborted read as `.readError`. Acting on it spent the session-lifetime revive gate, + // doomed the replacement demuxer and queued an authoritative restart at a stale position. + restartLock.lock() + let isCurrent = producer === prod + restartLock.unlock() + guard isCurrent else { + if case .stopRequested = reason {} else { + EngineLog.emit( + "[HLSVideoEngine] superseded producer exited (reason=\(reason)); not the session's pump, ignored", + category: .session + ) + } + return + } // #65 (VOD only): a broken backpressure wedge means AVPlayer is stuck behind a parked producer. // Re-anchor the producer on AVPlayer's real position so the segments it is starved for get produced. if case .backpressureWedge = reason { @@ -246,7 +282,7 @@ extension HLSVideoEngine { packetsWritten: prod.packetsWrittenCount, cachedSegments: cache?.count ?? 0 ) { - handleVODReadErrorExit(code) + handleVODReadErrorExit(code, deadProducer: prod) } else { EngineLog.emit( "[HLSVideoEngine] VOD pump died before producing anything " @@ -344,8 +380,19 @@ extension HLSVideoEngine { return } } - restartLock.lock() let segmentsNow = provider?.liveContinuationPoint().nextIndex ?? 0 + if let onLiveJoinWithoutEntryPoint, + Self.liveJoinFoundNoEntryPoint( + reason: reason, segmentsProduced: segmentsNow, starvedVideoDrops: prod.starvedVideoDrops) { + EngineLog.emit( + "[HLSVideoEngine] AE#627 live join read \(prod.starvedVideoDrops) video packets without an " + + "entry point the native route can open; a reopen joins the same bitstream, so not reopening", + category: .session + ) + onLiveJoinWithoutEntryPoint() + return + } + restartLock.lock() let reopenDecision = Self.liveRecoveryBudgetDecision( progressIndex: segmentsNow, lastProgressIndex: lastReopenSegmentCount, @@ -382,7 +429,7 @@ extension HLSVideoEngine { /// AVPlayer's real position, authoritative so it wins the coalescer's pending slot. The /// demuxer whose read just threw is marked suspect so performRestart replaces it via the #79 /// fresh-demuxer path instead of seeking the failed connection. - func handleVODReadErrorExit(_ code: Int32) { + func handleVODReadErrorExit(_ code: Int32, deadProducer: HLSSegmentProducer? = nil) { // #377: the reader knows the difference between a source that is gone and one that is // metering us, and loses it on the way here: the give-up arm returns a bare `-1`, FFmpeg // renders that as "Operation not permitted", and this is what arrives. Ask the budget, @@ -471,11 +518,32 @@ extension HLSVideoEngine { + "\(String(format: "%.0f", spent))s of \(String(format: "%.0f", budget))s spent)", category: .session ) - Task.detached(priority: .userInitiated) { [weak self] in + let sessionEpoch = sessionEpochSnapshot() + Task.detached(priority: .userInitiated) { [weak self, weak deadProducer] in try? await Task.sleep(nanoseconds: UInt64(delay * 1_000_000_000)) guard let self else { return } - self.requestRestart(at: idx, authoritative: true) + self.fireMeteredRevive(at: idx, deadProducer: deadProducer, sessionEpoch: sessionEpoch) + } + } + + /// Audit HLS-104: the backoff runs 3 to 45 s, and in that time a segment-driven restart, a seek or + /// a stop may already have replaced the dead producer. Whichever did has the better position. + static func meteredReviveStillOwed(sessionUnchanged: Bool, deadProducerInstalled: Bool) -> Bool { + sessionUnchanged && deadProducerInstalled + } + + func fireMeteredRevive(at idx: Int, deadProducer: HLSSegmentProducer?, sessionEpoch: UInt64) { + let sessionUnchanged = isSessionEpochCurrent(sessionEpoch) + let owed = Self.meteredReviveStillOwed( + sessionUnchanged: sessionUnchanged, deadProducerInstalled: currentProducerIs(deadProducer)) + guard owed else { + EngineLog.emit( + "[HLSVideoEngine] #377 metered revive at seg\(idx) skipped: the dead producer was " + + "already replaced or the session stopped during the backoff", + category: .session) + return } + requestRestart(at: idx, authoritative: true, reviving: (deadProducer, sessionEpoch)) } /// #377: how long a refusal keeps classifying a read error as metering. The reader's give-up @@ -720,8 +788,9 @@ extension HLSVideoEngine { liveReopenOutputEndSeconds: outputEnd ) newProd.firstSegmentDiscontinuous = true + let newEpoch = newProd.epoch newProd.onVideoShiftKnown = { [weak self] shiftPts, _, _ in - self?.handleLiveTimelineRebase(shiftPts, seamOutputSeconds: outputEnd) + self?.handleLiveTimelineRebase(shiftPts, seamOutputSeconds: outputEnd, producerEpoch: newEpoch) } producer = newProd restartLock.unlock() @@ -903,7 +972,9 @@ extension HLSVideoEngine { try? await Task.sleep(nanoseconds: UInt64(delay * 1_000_000_000)) let dem = Demuxer() - registerReopenDemuxer(dem) // register before blocking open so stop() can abort via markClosed + // Audit HLS-2: a zap during the sleep above must not open the old channel; on a + // single-slot tuner that orphan open takes the slot the new channel needs. + guard registerReopenDemuxer(dem, failedProducer: failedProducer) else { return } defer { unregisterReopenDemuxer(dem) } var freshReader: IOReader? do { @@ -967,16 +1038,20 @@ extension HLSVideoEngine { } /// NSLock unavailable from async contexts; this synchronous helper wraps the check. - private func currentProducerIs(_ p: HLSSegmentProducer) -> Bool { + private func currentProducerIs(_ p: HLSSegmentProducer?) -> Bool { restartLock.lock() defer { restartLock.unlock() } return producer === p } - private func registerReopenDemuxer(_ dem: Demuxer) { + /// Registers `dem` for `stop()` to abort, atomically with the check that the session still + /// belongs to `failedProducer`. False means a stop or a newer producer already took over. + func registerReopenDemuxer(_ dem: Demuxer, failedProducer: HLSSegmentProducer) -> Bool { restartLock.lock() + defer { restartLock.unlock() } + guard producer === failedProducer else { return false } reopenDemuxer = dem - restartLock.unlock() + return true } private func unregisterReopenDemuxer(_ dem: Demuxer) { @@ -1014,8 +1089,9 @@ extension HLSVideoEngine { ) // Fresh connection joins the broadcast at "now"; source clock jumps, so the seam carries #EXT-X-DISCONTINUITY. Shift handoff deferred to seam to avoid jumping the host clock while pre-loss content is on screen. newProd.firstSegmentDiscontinuous = true + let newEpoch = newProd.epoch newProd.onVideoShiftKnown = { [weak self] shiftPts, _, _ in - self?.handleLiveTimelineRebase(shiftPts, seamOutputSeconds: outputEnd) + self?.handleLiveTimelineRebase(shiftPts, seamOutputSeconds: outputEnd, producerEpoch: newEpoch) } producer = newProd // #199: the new demuxer reads from the factory-vended reader; take ownership so stop() diff --git a/Sources/AetherEngine/Video/HLSVideoEngine+SegmentPlanning.swift b/Sources/AetherEngine/Video/HLSVideoEngine+SegmentPlanning.swift index fa3e16bb5..1218db350 100644 --- a/Sources/AetherEngine/Video/HLSVideoEngine+SegmentPlanning.swift +++ b/Sources/AetherEngine/Video/HLSVideoEngine+SegmentPlanning.swift @@ -49,7 +49,7 @@ extension HLSVideoEngine { /// interleaver before its first flush, which on a 110 min Blu-ray climbed to ~13 GB of RAM and /// swapped until the device disk filled. /// - /// Two witnesses, both required: + /// Three witnesses, all required: /// /// - **Gap (#64)**: the largest gap between consecutive keyframes. A real index never gaps more than /// a few GOPs (well under the cap); a clustered TS index gaps by thousands of seconds. @@ -61,26 +61,35 @@ extension HLSVideoEngine { /// whole-file segment, from which AVPlayer loads zero tracks. Below one segment of coverage the /// keyframe planner cannot make even the first cut, so such an index is rejected here. /// - /// Coverage is the span between keyframes, never reaching to EOF, so a dense index that stops early - /// (the trailing-gap-not-counted case) is unaffected: its span already exceeds one segment. - /// An index failing either witness is routed to the uniform-stride fallback. + /// - **Tail (PR #703)**: the span must also reach to within `maxTrailingGapSeconds` of the source + /// duration. An index that stops minutes short is not dense-but-short, it is a partial scan: an + /// MKV whose Cues are missing or point past EOF (an incomplete download) leaves only what the + /// capped prewarm walked, and the keyframe planner then cuts its last segment from the final + /// scanned keyframe to the end of the title (measured on a device: 205 s to 5933 s), which the + /// producer can never finish and AVPlayer waits on forever. The 60 s default leaves room for a + /// long final GOP and for a container duration padded by a trailing audio or subtitle track. + /// + /// An index failing any witness is routed to the uniform-stride fallback. static func keyframeIndexIsTrustworthy( keyframes: [Int64], videoTimeBase: AVRational, sourceDurationSeconds: Double, maxTrustedGapSeconds: Double = Swift.max(HLSVideoEngine.targetSegmentDuration * 4, 30), - minCoverageSeconds: Double = HLSVideoEngine.targetSegmentDuration + minCoverageSeconds: Double = HLSVideoEngine.targetSegmentDuration, + maxTrailingGapSeconds: Double = 60 ) -> Bool { guard keyframes.count >= 2, sourceDurationSeconds > 0, videoTimeBase.num > 0, videoTimeBase.den > 0 else { return false } let tb = Double(videoTimeBase.num) / Double(videoTimeBase.den) let sorted = keyframes.sorted() - let coverageSeconds = Double(sorted[sorted.count - 1] - sorted[0]) * tb + // In Double: an index spanning both Int64 extremes overflows the integer difference (audit HLS-102). + let coverageSeconds = (Double(sorted[sorted.count - 1]) - Double(sorted[0])) * tb guard coverageSeconds >= minCoverageSeconds else { return false } + guard sourceDurationSeconds - coverageSeconds <= maxTrailingGapSeconds else { return false } var largestGapSeconds = 0.0 for i in 1.. largestGapSeconds { largestGapSeconds = gapSeconds } } return largestGapSeconds <= maxTrustedGapSeconds @@ -182,6 +191,19 @@ extension HLSVideoEngine { return firstKeyframe != nil ? .singleKeyframeInSource(budget) : .unknown } + /// Most segments any plan builder emits (audit HLS-102). A week at the 4 s target is 151,200. + static let maxPlanSegments = 200_000 + + /// `base + seconds / tb` in ticks, nil when the result leaves the tick range (audit HLS-102): the + /// builders are fed container durations and manifest sums, and `Int64(_:)` traps on NaN, infinity + /// and anything past `Int64`. Truncates like the plain conversion it replaces. + static func planPts(_ base: Int64, plusSeconds seconds: Double, timeBase tb: Double) -> Int64? { + let ticks = seconds / tb + guard ticks.isFinite, abs(ticks) < Demuxer.maxPlausibleIndexTicks else { return nil } + let (pts, overflow) = base.addingReportingOverflow(Int64(ticks)) + return overflow ? nil : pts + } + /// Uniform-duration fallback plan when the keyframe index is too sparse. Source-axis boundaries are /// anchored at `startPts0` (the first keyframe PTS), exactly like the keyframe-aligned plan, so segment 0 /// begins at the content start rather than at source PTS 0. A title whose content starts late (e.g. a @@ -196,9 +218,14 @@ extension HLSVideoEngine { startPts0: Int64 = 0, strideSeconds: Double = HLSVideoEngine.targetSegmentDuration ) -> [Segment] { - guard sourceDurationSeconds > 0 else { return [] } - let stride = strideSeconds.isFinite && strideSeconds > 0 ? strideSeconds : Self.targetSegmentDuration - let count = max(1, Int(ceil(sourceDurationSeconds / stride))) + guard sourceDurationSeconds > 0, sourceDurationSeconds.isFinite else { return [] } + var stride = strideSeconds.isFinite && strideSeconds > 0 ? strideSeconds : Self.targetSegmentDuration + // Audit HLS-102: a stride this fine over a header-stated duration asked for trillions of + // segments. Widen it instead, so the plan still covers the whole duration. + if sourceDurationSeconds / stride > Double(maxPlanSegments) { + stride = sourceDurationSeconds / Double(maxPlanSegments) + } + let count = min(maxPlanSegments, max(1, Int(ceil(sourceDurationSeconds / stride)))) let tb = Double(videoTimeBase.num) / Double(videoTimeBase.den) guard tb > 0 else { return [] } @@ -206,9 +233,10 @@ extension HLSVideoEngine { plan.reserveCapacity(count) for i in 0.. Int64 { - i == 0 ? startPts0 : startPts0 + Int64((starts[i] - backoff) / tb) + func boundaryPts(_ i: Int) -> Int64? { + i == 0 ? startPts0 : planPts(startPts0, plusSeconds: starts[i] - backoff, timeBase: tb) } var plan: [Segment] = [] plan.reserveCapacity(starts.count) for i in 0..`, `CoreMediaErrorDomain -4`. Caller must seek back after this consumes packets. + /// Scan packets for in-band VPS/SPS/PPS when hvcC `numOfArrays=0` (DV P5 MP4 encoders, e.g. Wandering Earth 2 WEB-DL, issue #19). AVPlayer symptom: `item.tracks count=2`, `fourCC=`, `CoreMediaErrorDomain -4`. On a source that can rewind the scan consumes packets and the caller must seek back; on a forward-only one it only peeks (`scanSourceHead`). /// /// `rewindBeforeScan` rewinds to the head first, because in-band parameter sets are guaranteed at /// the first IRAP but only recur every GOP after it. Without the rewind the scan inherited whatever @@ -450,8 +487,6 @@ extension HLSVideoEngine { guard Self.configRecordNeedsInBandRebuild(extradata, size: extradataSize) else { return nil } let naluLengthSize = 4 // the predicate above already required it - if rewindBeforeScan { demuxer.seek(to: 0) } - var vps: [UInt8]? var sps: [UInt8]? var pps: [UInt8]? @@ -461,25 +496,11 @@ extension HLSVideoEngine { var packetsScanned = 0 // Second cap so a stream that never yields a video packet cannot walk the whole source. let readBudget = 512 - var packetsRead = 0 - while packetsScanned < packetBudget && packetsRead < readBudget { - let readResult: UnsafeMutablePointer? - do { - readResult = try demuxer.readPacket() - } catch { - break - } - guard let pkt = readResult else { break } - defer { - // trackedPacketFree not raw av_packet_free: readPacket allocs via trackedPacketAlloc; raw free leaves PacketBalanceTracker.pktAlive permanently high. - var maybePkt: UnsafeMutablePointer? = pkt - trackedPacketFree(&maybePkt) - } - packetsRead += 1 - if pkt.pointee.stream_index != videoStreamIndex { continue } + scanSourceHead(demuxer: demuxer, rewind: rewindBeforeScan, readBudget: readBudget) { pkt in + if pkt.pointee.stream_index != videoStreamIndex { return false } packetsScanned += 1 - guard let pktData = pkt.pointee.data else { continue } + guard let pktData = pkt.pointee.data else { return packetsScanned >= packetBudget } let pktSize = Int(pkt.pointee.size) var offset = 0 @@ -501,7 +522,7 @@ extension HLSVideoEngine { offset += nalLen } - if vps != nil && sps != nil && pps != nil { break } + return (vps != nil && sps != nil && pps != nil) || packetsScanned >= packetBudget } guard let vps, let sps, let pps else { @@ -531,6 +552,44 @@ extension HLSVideoEngine { return hvcC } + /// Shows the packets at the head of the source to `inspect`, which returns true once it has seen + /// enough, looking at no more than `readBudget` of them. + /// + /// A source that can rewind is rewound (when `rewind`) and the packets are CONSUMED, which is + /// what these scans always did: `start()` seeks back to 0 afterwards. A source that cannot rewind + /// has nothing to seek back with, so whatever the scan consumed is gone from the archive + /// (audit HLS-103: a sequential origin lost its opening GOP to the framing probe). Those are + /// PEEKED instead: the packets the scan looked at stay queued for the producer. + func scanSourceHead( + demuxer: Demuxer, + rewind: Bool, + readBudget: Int, + _ inspect: (UnsafeMutablePointer) -> Bool + ) { + guard demuxer.isSourceSeekable else { + try? demuxer.peekPackets(maxPackets: readBudget, inspect) + return + } + if rewind { demuxer.seek(to: 0) } + var packetsRead = 0 + while packetsRead < readBudget { + let readResult: UnsafeMutablePointer? + do { + readResult = try demuxer.readPacket() + } catch { + break + } + guard let pkt = readResult else { break } + defer { + // trackedPacketFree not raw av_packet_free: readPacket allocs via trackedPacketAlloc; raw free leaves PacketBalanceTracker.pktAlive permanently high. + var maybePkt: UnsafeMutablePointer? = pkt + trackedPacketFree(&maybePkt) + } + packetsRead += 1 + if inspect(pkt) { break } + } + } + /// What the muxer should ship for a source whose config record is Annex B (#365), and the /// framing every NAL walker in the session should use. struct VideoFramingNormalization { @@ -540,6 +599,9 @@ extension HLSVideoEngine { /// never going to reformat this track anyway). Callers fall back to deriving it from the /// extradata, which is only safe while the two agree. let measuredFraming: VideoNALFraming? + /// The packets are Annex B and `extradataOverride` is a length-prefixed record: the muxer + /// converts every sample itself and keeps its in-band parameter sets (`AnnexBSampleConverter`). + var convertsAnnexBSamples = false } /// Measure the video NAL framing on packets, then decide what config record the muxer gets. @@ -551,8 +613,10 @@ extension HLSVideoEngine { /// length happens to read as `00 00 01` and drops everything else. Measured 1080p: a 2,158,448 B /// segment came out at 61,912 B, AVPlayer reached `readyToPlay` and never produced a frame. /// - /// Consumes packets and leaves the demuxer cursor where it stopped; `start()` seeks back to 0 - /// afterwards, same contract as the in-band scan and the cue prewarm. + /// On a source that can rewind it consumes packets and leaves the demuxer cursor where it + /// stopped; `start()` seeks back to 0 afterwards, same contract as the in-band scan and the cue + /// prewarm. A forward-only source is peeked instead (`scanSourceHead`), so the producer still + /// receives the packets the probe looked at. func normalizeVideoFraming( demuxer: Demuxer, videoStreamIndex: Int32, @@ -574,8 +638,8 @@ extension HLSVideoEngine { let framing = probeVideoNALFraming(demuxer: demuxer, videoStreamIndex: videoStreamIndex) guard case .lengthPrefixed = framing else { - // The record stays Annex B here: movenc reads it to decide whether to convert the samples, - // and these samples do need converting. What it must not keep is a prefix SEI, because the + // Except for HEVC on Annex-B packets (below) the record stays Annex B: movenc reads it to + // decide whether to convert the samples, and these samples do need converting. What it must not keep is a prefix SEI, because the // hvcC movenc then builds carries it as a fourth array (`ff_isom_write_hvcc` collects five // NAL types) and Apple TV's HEVC track builder rejects such a record (AE#187). That defense // sits on the record path and cannot see this one, so the SEI goes before the muxer runs. @@ -584,6 +648,25 @@ extension HLSVideoEngine { } ?? [] let canonical = codecID == AV_CODEC_ID_HEVC ? VideoConfigRecord.canonicalizeAnnexBHEVCConfigRecord(source) : nil + // HEVC on conclusively Annex-B packets: movenc's own conversion drops every in-band + // parameter set under `hvc1`, which breaks a stream that sends a new PPS mid-title. Give + // it a length-prefixed record so it copies the samples, and convert them in the muxer + // instead, parameter sets kept. Built from the canonical record so the hvcC carries no + // SEI array (AE#187). + if codecID == AV_CODEC_ID_HEVC, case .annexB? = framing, + let record = VideoConfigRecord.fromAnnexB( + canonical ?? source, codecID: codecID, + width: codecpar.pointee.width, height: codecpar.pointee.height) { + EngineLog.emit( + "[HLSVideoEngine] #365 HEVC on Annex-B packets: the muxer gets a length-prefixed " + + "record (\(source.count) B → \(record.count) B) and converts the samples itself, " + + "keeping their in-band parameter sets", + category: .session + ) + var result = VideoFramingNormalization(extradataOverride: record, measuredFraming: framing) + result.convertsAnnexBSamples = true + return result + } EngineLog.emit( "[HLSVideoEngine] #365 the muxer will reformat this track's samples and the packets " + "are \(framing == nil ? "not conclusively framed" : "Annex B"); the muxer builds the " @@ -630,27 +713,13 @@ extension HLSVideoEngine { packetBudget: Int = 4, readBudget: Int = 512 ) -> VideoNALFraming? { - demuxer.seek(to: 0) var lengthPrefixed = 0 var annexB = 0 var inspected = 0 - var packetsRead = 0 - while inspected < packetBudget && packetsRead < readBudget { - let readResult: UnsafeMutablePointer? - do { - readResult = try demuxer.readPacket() - } catch { - break - } - guard let pkt = readResult else { break } - defer { - var maybePkt: UnsafeMutablePointer? = pkt - trackedPacketFree(&maybePkt) - } - packetsRead += 1 + scanSourceHead(demuxer: demuxer, rewind: true, readBudget: readBudget) { pkt in guard pkt.pointee.stream_index == videoStreamIndex, - let data = pkt.pointee.data, pkt.pointee.size > 4 else { continue } + let data = pkt.pointee.data, pkt.pointee.size > 4 else { return false } inspected += 1 let size = Int(pkt.pointee.size) if VideoConfigRecord.walksAsLengthPrefixed(data, size: size) { @@ -658,6 +727,7 @@ extension HLSVideoEngine { } else if VideoConfigRecord.startsWithStartCode(data, size: size) { annexB += 1 } + return inspected >= packetBudget } let framing: VideoNALFraming? diff --git a/Sources/AetherEngine/Video/HLSVideoEngine+SpatialAudio.swift b/Sources/AetherEngine/Video/HLSVideoEngine+SpatialAudio.swift index e1f83cd8d..9c9d110c8 100644 --- a/Sources/AetherEngine/Video/HLSVideoEngine+SpatialAudio.swift +++ b/Sources/AetherEngine/Video/HLSVideoEngine+SpatialAudio.swift @@ -86,6 +86,13 @@ extension HLSVideoEngine { ) savedAudioConfig = cfg audioBridge = bridge + // Same surfacing as the channel bridge: a decoder that never answers is a failure, not silence. + if sideAudioDemuxer == nil { + let streamIndex = sourceAudioStreamIndex + bridge.onDecoderProducedNothing = { [weak self] stats in + self?.handleBridgeDecodedNothing(streamIndex: streamIndex, summary: stats.summary) + } + } do { let producer = try makeProducer(baseIndex: initialProducerBaseIndex) audioHLSCodecs = bridge.hlsCodecs diff --git a/Sources/AetherEngine/Video/HLSVideoEngine.swift b/Sources/AetherEngine/Video/HLSVideoEngine.swift index 0fdbca27c..84c126870 100644 --- a/Sources/AetherEngine/Video/HLSVideoEngine.swift +++ b/Sources/AetherEngine/Video/HLSVideoEngine.swift @@ -77,13 +77,29 @@ public final class HLSVideoEngine: @unchecked Sendable { /// Caller-chosen audio stream index; nil falls back to `av_find_best_stream`. Enables /// host-driven track switching via `AetherEngine.selectAudioTrack(index:)` reload. private let audioSourceStreamIndexOverride: Int32? + /// AE#641: the source audio streams this session already found undecodable (their bridge decoded + /// nothing). When the pick lands on one again, the cascade goes straight to its video-only tail. + let undecodableAudioStreamIndices: Set + + /// Whether the cascade has to skip the stream it is about to build a pipeline for. + nonisolated static func isKnownUndecodable(_ indices: Set, sourceAudioStreamIndex: Int32) -> Bool { + sourceAudioStreamIndex >= 0 && indices.contains(sourceAudioStreamIndex) + } /// AE#443: whoever REPLACES one of these two mid-session owes the session the totals the outgoing /// instance held (`retireDemuxer` / `retireProducer` below). They carry the session's byte and /// restart counters, and a fresh instance starts them at zero. var demuxer: Demuxer? var cache: SegmentCache? // internal for the teardown-partial witness test - var producer: HLSSegmentProducer? + var producer: HLSSegmentProducer? { + didSet { + // Audit SEG-104: a gate open is checked against this under `anchorShiftLock`, atomically + // with what it records, so a producer that was replaced cannot record after. + anchorShiftLock.lock() + installedProducerEpoch = producer?.epoch + anchorShiftLock.unlock() + } + } private var server: HLSLocalServer? var provider: VideoSegmentProvider? @@ -183,6 +199,16 @@ public final class HLSVideoEngine: @unchecked Sendable { /// Set before `start()`. var enableNativeSubtitleTrackForSession: Bool = false + /// AE#682: the host asked for an I-frame rendition. Whether the session can serve one is decided + /// in `start()` and read back as `iFrameRenditionVerdict`. + var iFramePlaylistRequested = false + /// AE#682: an independent reader for a custom-IO source, which this session cannot reopen by URL. + /// Set before `start()`. The session owns it from then on and closes it: through the side reader + /// that took it, at once when the rendition stays absent, or in `stop()`. + var customIFrameReader: (reader: IOReader, formatHint: String?)? + private(set) var iFrameRenditionVerdict: IFrameRenditionEligibility.Verdict = .absent(.notRequested) + private var iFrameRendition: IFrameRendition? + /// Native subtitle rendition marked DEFAULT=YES in the master (Sodalite#32). Set before `start()`; the /// provider advertises this ordinal as the group default so a host-selected legible track renders. var nativeSubtitleDefaultOrdinal: Int = 0 @@ -282,14 +308,19 @@ public final class HLSVideoEngine: @unchecked Sendable { /// #112 rework: build an overlay decoder for any embedded subtitle stream (text or /// bitmap), seeded exactly like the tap routes. The drainer owns the returned decoder. func makeOverlayDecoder(streamIndex: Int32, preserveASSMarkup: Bool) -> EmbeddedSubtitleDecoder? { - guard let dem = demuxer, let stream = dem.stream(at: streamIndex) else { return nil } + guard let dem = demuxer else { return nil } let w = savedVideoConfig.map { Int32($0.codecpar.pointee.width) } ?? 1920 let h = savedVideoConfig.map { Int32($0.codecpar.pointee.height) } ?? 1080 - return EmbeddedSubtitleDecoder(stream: stream, - sourceVideoWidth: w > 0 ? w : 1920, - sourceVideoHeight: h > 0 ? h : 1080, - preserveASSMarkup: preserveASSMarkup, - teletextPage: teletextPageForSubtitleTap) + // Audit DMX-108: the decoder copies what it needs out of the stream inside the call, so a + // live reopen's close() cannot free the stream underneath it. + let made = dem.withStream(at: streamIndex) { stream in + EmbeddedSubtitleDecoder(stream: stream, + sourceVideoWidth: w > 0 ? w : 1920, + sourceVideoHeight: h > 0 ? h : 1080, + preserveASSMarkup: preserveASSMarkup, + teletextPage: teletextPageForSubtitleTap) + } + return made ?? nil } /// Sodalite#32 Phase 2: tap decoders honor the host's markup preference so the overlay can render @@ -315,6 +346,12 @@ public final class HLSVideoEngine: @unchecked Sendable { enableNativeSubtitleTrackForSession = true } + /// Ask for an I-frame rendition in the master (AE#682), the `LoadOptions.serveIFramePlaylist` + /// path a full session uses. Must precede `start()`. + public func requestIFramePlaylist() { + iFramePlaylistRequested = true + } + /// Attach `count` fresh cue stores (one per declared text track) to the current producer (#55). /// Call after `start()`. `languages` and `sourceStreamIndices` are ordinal-aligned, nil-padded. public func attachNativeSubtitleStores(count: Int, languages: [String?] = [], @@ -361,14 +398,16 @@ public final class HLSVideoEngine: @unchecked Sendable { let w = savedVideoConfig.map { Int32($0.codecpar.pointee.width) } ?? 1920 let h = savedVideoConfig.map { Int32($0.codecpar.pointee.height) } ?? 1080 for (ordinal, sidx) in nativeSubtitleSourceStreamIndicesForSession.enumerated() { - guard let sidx, ordinal < nativeSubtitleCueStoresForSession.count, - let stream = dem.stream(at: sidx), - let decoder = EmbeddedSubtitleDecoder(stream: stream, - sourceVideoWidth: w > 0 ? w : 1920, - sourceVideoHeight: h > 0 ? h : 1080, - preserveASSMarkup: preserveASSMarkupForSubtitleTap, - teletextPage: teletextPageForSubtitleTap) - else { continue } + guard let sidx, ordinal < nativeSubtitleCueStoresForSession.count else { continue } + // Audit DMX-108: see `makeOverlayDecoder`. + let made = dem.withStream(at: sidx) { stream in + EmbeddedSubtitleDecoder(stream: stream, + sourceVideoWidth: w > 0 ? w : 1920, + sourceVideoHeight: h > 0 ? h : 1080, + preserveASSMarkup: preserveASSMarkupForSubtitleTap, + teletextPage: teletextPageForSubtitleTap) + } + guard let decoder = made ?? nil else { continue } subtitleTapRoutes[sidx] = (decoder, nativeSubtitleCueStoresForSession[ordinal]) } if !subtitleTapRoutes.isEmpty { @@ -477,7 +516,7 @@ public final class HLSVideoEngine: @unchecked Sendable { /// AE#418 round 2 + PR #533: what each epoch left at the index it opened on, both what its first /// segment adds to the axis and what all of its bytes carry. See `EpochAxisTable`. private let anchorShiftLock = NSLock() - private var epochAxisByIndex = EpochAxisTable() + private(set) var epochAxisByIndex = EpochAxisTable() // internal read for the superseded-producer tests /// PR #533 round 2: how far AVPlayer's OWN timeline is displaced from the playlist, which is the /// quantity every AE#418 rule is about. /// @@ -529,7 +568,9 @@ public final class HLSVideoEngine: @unchecked Sendable { /// `play --picture-probe`: `axisErr` 0.000 at offsets of 1, 5 and 9 s), so its distance to its /// advertised start is not an axis offset and must not compose into one. Guarded by /// `anchorShiftLock`, alongside the table it keeps entries out of. - private var recutIndices: Set = [] + private var recutMark: RecutMark? + /// Epoch of `producer`, mirrored under `anchorShiftLock` (audit SEG-104). + private var installedProducerEpoch: UInt64? private let shiftLock = NSLock() private var _playlistShiftSeconds: Double = 0 @@ -622,11 +663,23 @@ public final class HLSVideoEngine: @unchecked Sendable { /// In-flight live reopen demuxer, registered before its blocking open so `stop()` can abort it /// (prevents orphan reconnect loops across channel zaps). var reopenDemuxer: Demuxer? + /// The #79 / #169 restart's replacement demuxer, from before its open until the restart installs + /// or drops it, so `stop()` can abort that open and the seek after it (audit SEG-3 follow-up). + private var restartReopenDemuxer: Demuxer? /// Fires on live program-boundary rebase: `(newShiftSeconds, seamOutputSeconds)`. AetherEngine /// defers applying the shift until playback crosses `seamOutputSeconds` so the clock doesn't jump. var onPlaylistShiftRebased: (@Sendable (Double, Double) -> Void)? /// Fires on `PumpExitReason.sourceReplay`; host must re-negotiate a fresh session. var onLiveSourceReset: (@Sendable () -> Void)? + /// AE#627: the first join read video for the whole keyframe wait and found no point the native + /// route can open a segment on. Reopening joins the same bitstream, so the engine decides between + /// the software path and telling the host (`escalateLiveReopenExhaustion`) instead. Unset, the + /// session falls back to the ordinary reopen budget. + var onLiveJoinWithoutEntryPoint: (@Sendable () -> Void)? + /// AE#641: the live audio bridge decoded not one frame of the selected stream (index, bridge + /// summary). Nothing inside the session can recover it: a producer rebuild hands the same decoder + /// the same bytes, and the served media keeps an audio track that will never be filled. + var onLiveAudioDecodesNothing: (@Sendable (Int32, String) -> Void)? /// #126: fires when a VOD pump dies on a read error having produced nothing (zero packets /// written, empty cache). The playlist exists but no segment will ever land, so AVPlayer /// would sit in waitingToPlay forever; the engine surfaces a fatal error instead. @@ -649,9 +702,20 @@ public final class HLSVideoEngine: @unchecked Sendable { /// Pairing the release with the surface makes that structural instead of a call site to remember. func surfaceVODSourceFailure(_ code: Int32, _ reason: String, kind: PlaybackErrorKind = .vodSourceFailed) { + // AE#641: a silent bridge is reported the moment it is structural and again by the muxer arms + // that follow it on the E-AC-3 route; the session learns it once. + if kind == .audioBridgeProducedNoOutput { + silentBridgeSurfaceLock.lock() + let first = !silentBridgeFailureSurfaced + silentBridgeFailureSurfaced = true + silentBridgeSurfaceLock.unlock() + guard first else { return } + } provider?.abortSequentialStartupWait() onVODSourceFailed?(code, reason, kind) } + private let silentBridgeSurfaceLock = NSLock() + private var silentBridgeFailureSurfaced = false /// Session-long FLAC bridge for codecs illegal in fMP4. Engine-owned (not producer-owned) so /// encoder state survives producer restarts; `startSegment()` rebases PTS on each restart. var audioBridge: (any AudioTranscodingBridge)? @@ -858,6 +922,7 @@ public final class HLSVideoEngine: @unchecked Sendable { matchContentEnabled: Bool = true, panelIsInHDRMode: Bool = false, audioSourceStreamIndexOverride: Int32? = nil, + undecodableAudioStreamIndices: Set = [], audioBridgeMode: AudioBridgeMode = .surroundCompat, objectAudioRendering: ObjectAudioRendering = .off, isLiveSession: Bool = false, @@ -868,6 +933,8 @@ public final class HLSVideoEngine: @unchecked Sendable { liveCadenceObservation: (@Sendable () -> Double?)? = nil, liveClosedCadenceObservation: (@Sendable () -> Double?)? = nil, liveUpstreamSegmentDurationObservation: (@Sendable () -> Double?)? = nil, + liveJoinBacklogObservation: (@Sendable () -> Double?)? = nil, + liveJoinSpentObservation: (@Sendable () -> Bool?)? = nil, upstreamSelfReportedTargetDuration: Double? = nil, preopenedDemuxer: Demuxer? = nil, sourceReopenableByURL: Bool = true, @@ -903,6 +970,7 @@ public final class HLSVideoEngine: @unchecked Sendable { self.matchContentEnabled = matchContentEnabled self.panelIsInHDRMode = panelIsInHDRMode self.audioSourceStreamIndexOverride = audioSourceStreamIndexOverride + self.undecodableAudioStreamIndices = undecodableAudioStreamIndices self.audioBridgeMode = audioBridgeMode self.objectAudioRendering = objectAudioRendering self.isLiveSession = isLiveSession @@ -925,7 +993,9 @@ public final class HLSVideoEngine: @unchecked Sendable { observeSealEvidence: { LiveCadenceEvidence( closedCadenceSeconds: liveClosedCadenceObservation?(), - servedSegmentDurationSeconds: liveUpstreamSegmentDurationObservation?() + servedSegmentDurationSeconds: liveUpstreamSegmentDurationObservation?(), + joinBacklogSeconds: liveJoinBacklogObservation?(), + joinIsSpent: liveJoinSpentObservation?() ) }, selfReportedTargetDurationSeconds: upstreamSelfReportedTargetDuration @@ -949,6 +1019,8 @@ public final class HLSVideoEngine: @unchecked Sendable { /// playlist advertises stays resident; the producer-side prefetch park it also feeds is /// VOD-only (`advanceMuxer`), so live cannot park on it. private var retentionBudgetBytes: Int = 0 + /// #687: this session's entry in the process-wide ledger, released on `stop()`. + private var retentionClaim: RetentionClaims.Claim? /// Clamp for `forwardWindowSegments`: below 4 the window would undercut AVPlayer's own ~5-7-segment /// prefetch and starve it (see `LiveWindowSizing.minSafeSegments`). The 2700 ceiling (~3 h at 4 s @@ -1408,9 +1480,12 @@ public final class HLSVideoEngine: @unchecked Sendable { } // 6. Reset demuxer cursor to 0 (cue prewarm moved it mid-file). Skipped for live - // (no prewarm, forward-only feed). + // (no prewarm, forward-only feed), and for a forward-only VOD source: the prewarm did not + // run there, so the cursor never left the head, and a seek on a pb that cannot rewind + // drops the probe's buffered packets and leaves matroskadec resyncing from wherever the + // stream has got to (a remote MKV lost its first second, on the software path 30 s). if !isLiveSession { - dem.seek(to: 0) + if dem.isSourceSeekable { dem.seek(to: 0) } dem.endIndexPass() // AE#585: the next read that lands outside a span is playback's } @@ -1426,20 +1501,27 @@ public final class HLSVideoEngine: @unchecked Sendable { .volumeAvailableCapacityForImportantUsage #endif let capRelaxed = Self.retentionCapRelaxed(forwardWindowSegments: forwardWindowSegments) - let retentionBudget = Self.sessionRetentionBudgetBytes(volumeAvailableBytes: availableBytes, - capRelaxed: capRelaxed) + // #687: sized from what the other running sessions leave, not from the raw free space. + let claim = RetentionClaims.shared.claim(volumeAvailableBytes: availableBytes) { + Self.sessionRetentionBudgetBytes(volumeAvailableBytes: $0, capRelaxed: capRelaxed) + } + let retentionBudget = claim.bytes self.retentionBudgetBytes = retentionBudget let segmentCache = SegmentCache( forwardWindow: forwardWindowSegments, retentionBudgetBytes: retentionBudget, onResidentSetChanged: { [weak self] in self?.noteResidentSetChanged() } ) + claim.track { [weak segmentCache] in segmentCache?.totalBytes ?? 0 } + self.retentionClaim = claim self.cache = segmentCache EngineLog.emit( "[HLSVideoEngine] segment retention budget: \(retentionBudget / (1 << 20)) MiB " + "(volumeAvailable=\(availableBytes.map { "\($0 / (1 << 20)) MiB" } ?? "unknown"), " + "forwardWindow=\(forwardWindowSegments) seg" - + (capRelaxed ? ", opt-in prefetch: default cap relaxed" : "") + ")", + + (capRelaxed ? ", opt-in prefetch: default cap relaxed" : "") + + (claim.heldBackBytes > 0 + ? ", \(claim.heldBackBytes / (1 << 20)) MiB held back for other sessions" : "") + ")", category: .session ) @@ -1504,6 +1586,10 @@ public final class HLSVideoEngine: @unchecked Sendable { category: .session ) } + // Audit BIT-104: one framing verdict for the session, seeded only by a measurement (#365). + // The extradata-derived framing is a claim, not a measurement. + var framingMeasuredLengthPrefixed = false + if case .lengthPrefixed? = measuredVideoNALFraming { framingMeasuredLengthPrefixed = true } let videoConfig = HLSSegmentProducer.StreamConfig( codecpar: UnsafePointer(ownedVideoParams.ptr), timeBase: videoTimeBase, @@ -1512,7 +1598,9 @@ public final class HLSVideoEngine: @unchecked Sendable { convertP7ToProfile81: convertP7ToProfile81, colorOverride: p5ColorOverride, extradataOverride: hevcExtradataOverride, - nalFramingOverride: measuredVideoNALFraming + nalFramingOverride: measuredVideoNALFraming, + convertsAnnexBSamples: framingNormalization.convertsAnnexBSamples, + nalFramingLatch: NALFramingLatch(confirmed: framingMeasuredLengthPrefixed) ) self.videoStreamIndex = videoIndex self.savedVideoConfig = videoConfig @@ -1836,6 +1924,7 @@ public final class HLSVideoEngine: @unchecked Sendable { streamCopyAudio: streamCopyAudio, sourceAudioStreamIndex: audioStreamIndex, sourceAudioStream: audioStreamIndex >= 0 ? audioDem.stream(at: audioStreamIndex) : nil, + sourceCarriesAudio: audioDem.firstAudioStreamIndexByType >= 0, audioHLSCodecs: &audioHLSCodecs, audioLanguage: audioLanguage ) @@ -1889,6 +1978,7 @@ public final class HLSVideoEngine: @unchecked Sendable { ), allowsBoundedDegradedStart: liveJoinProfile == .fastZap, boundedStartFloorsAtHoldback: LiveEdgePolicy.boundedStartFloorArmed, + firstServeLatchCoversEngineCut: LiveEdgePolicy.firstServeLatchAllArmed, blockingReloadOverride: blockingReloadOverride, liveCadencePolicy: liveCadencePolicy, restartHandler: isLiveSession ? nil : { [weak self] idx in @@ -1928,22 +2018,7 @@ public final class HLSVideoEngine: @unchecked Sendable { // AE#520 round 2: the close reads it on the playlist-build thread, so it is a closure over a // mirror rather than a read of the item. prov.consumerBufferedSecondsProvider = consumerBufferedSecondsProvider - if isLiveSession { - prod.onLiveSegmentFinalized = { [weak prov] index, durationSeconds, startPtsSeconds, discontinuous in - prov?.appendLiveSegment(index: index, - startSeconds: startPtsSeconds, - durationSeconds: durationSeconds, - discontinuous: discontinuous) - } - // AE#443: the runaway park has to sit above the window this session actually serves, or it - // bounds the window instead of backstopping it, and its enforcement (a sleeping read - // thread) stops the origin from being drained. - prod.liveResidentCapProvider = { [weak prov] in prov?.liveResidentParkCap() ?? 0 } - } else if sequentialOrigin { - prod.onSequentialSegmentFinalized = { [weak prov] index, durationSeconds in - prov?.appendSequentialSegmentDuration(index: index, durationSeconds: durationSeconds) - } - } + wireProviderCallbacks(prod, to: prov) EngineLog.emit( "[HLSVideoEngine] prepared: codec=\(manifestCodecs)" @@ -1996,6 +2071,18 @@ public final class HLSVideoEngine: @unchecked Sendable { // the standard sourceIsHDR && panelReadyForHDR check routes them correctly. // #15: a SUBTITLES rendition lives only in a master; the pure decision below forces the // master for routing-safe subtitled sources so PiP can show subtitles. + // AE#682: decided before the routing below, because an I-frame rendition is one of the + // reasons to serve a master at all. + let iFrameCandidate = IFrameRenditionEligibility.candidate(.init( + requested: iFramePlaylistRequested, + isLive: isLiveSession, + planBoundariesClaimRandomAccess: planBoundariesClaimRandomAccess, + sequentialOrigin: sequentialOrigin, + heldSourceConnection: openProfile.avioHeldConnection, + originIsSerial: sourceReopenableByURL + && OriginRequestBudget.shared.requiresSerialRequests(sourceURL), + isDiscSource: dem.isDiscSource, + secondReaderAvailable: sourceReopenableByURL || customIFrameReader != nil)) let hasNativeSubs = enableNativeSubtitleTrackForSession && !nativeSubtitleCueStoresForSession.isEmpty // AE#187: tvOS HW HEVC needs the codec advertised in a master's CODECS attribute; a bare media // playlist (H.264 is fine media-direct) fails the item with tracks count=0 / -12848. Scope to @@ -2013,7 +2100,8 @@ public final class HLSVideoEngine: @unchecked Sendable { builtInPanelEngagesOnDemand: Self.builtInPanelEngagesOnDemand, frameRateKnown: frameRate != nil, videoCodecNeedsMasterSignaling: videoCodecNeedsMasterSignaling, - hasAudioRendition: servedAudioLanguage != nil) + hasAudioRendition: servedAudioLanguage != nil, + hasIFrameRendition: iFrameCandidate == .served) let resolvedURL: URL? = useMasterPlaylist ? srv.playlistURL : srv.mediaPlaylistURL @@ -2022,6 +2110,21 @@ public final class HLSVideoEngine: @unchecked Sendable { throw HLSVideoEngineError.openFailed(reason: "server URL not ready") } self.servingMasterPlaylist = useMasterPlaylist + iFrameRenditionVerdict = IFrameRenditionEligibility.resolve( + candidate: iFrameCandidate, servingMaster: useMasterPlaylist) + if iFrameRenditionVerdict == .served, let rendition = makeIFrameRendition(plan: plan, cache: segmentCache) { + iFrameRendition = rendition + prov.setIFrameSource(rendition) + EngineLog.emit("[HLSVideoEngine] i-frame rendition: served segments=\(plan.count)", + category: .session) + } else if iFramePlaylistRequested { + closeUnusedCustomIFrameReader() + if iFrameRenditionVerdict == .served { iFrameRenditionVerdict = .absent(.noSecondReader) } + if case .absent(let reason) = iFrameRenditionVerdict { + EngineLog.emit("[HLSVideoEngine] i-frame rendition: absent reason=\(reason.rawValue)", + category: .session) + } + } self.servedSourceIsHDR = videoRange != .sdr self.servedDolbyVisionConversion = convertP7ToProfile81 ? .profile7ToProfile81 : nil EngineLog.emit("[HLSVideoEngine] serving on \(url.absoluteString) (dvModeAvailable=\(dvModeAvailable) effectiveDvMode=\(effectiveDvMode) panelIsHDR=\(panelIsInHDRMode) displaySupportsHDR=\(displaySupportsHDR) matchContent=\(matchContentEnabled) sourceIsHDR=\(videoRange != .sdr || effectiveDvMode) useMaster=\(useMasterPlaylist) videoRange=\(videoRange) dvVariant=\(dvVariant) audioLang=\(servedAudioLanguage ?? "none"))") @@ -2068,6 +2171,9 @@ public final class HLSVideoEngine: @unchecked Sendable { /// symptom (htrung14, tvOS transport bar, Apple TV 4K 3rd gen). Routing-safety still decides: an /// unready HDR panel is a -11848 rejection, and a language is not worth one. /// + /// AE#682: `hasIFrameRendition` is a third reason of the same shape. `EXT-X-I-FRAME-STREAM-INF` + /// lives only in a master, and it is likewise not worth a display rejection. + /// /// #130: `frameRateKnown` gates PQ/HLG masters. AVPlayer filters a VIDEO-RANGE=PQ/HLG /// EXT-X-STREAM-INF that has no FRAME-RATE attribute out of the master at parse time and fails /// the item with NSURLErrorDomain -1002 without ever fetching media.m3u8 (byte-exact local @@ -2083,7 +2189,8 @@ public final class HLSVideoEngine: @unchecked Sendable { builtInPanelEngagesOnDemand: Bool, frameRateKnown: Bool, videoCodecNeedsMasterSignaling: Bool = false, - hasAudioRendition: Bool = false + hasAudioRendition: Bool = false, + hasIFrameRendition: Bool = false ) -> Bool { let sourceIsHDR = videoRange != .sdr || effectiveDvMode let panelReadyForHDR = panelIsInHDRMode @@ -2101,7 +2208,7 @@ public final class HLSVideoEngine: @unchecked Sendable { // before any media fetch (macOS / the Simulator build the track from the init hvcC and never // reproduce it). Forcing the master where it is routing-safe (SDR on any panel, HDR on a ready // one) closes that gap; the caller scopes the flag to tvOS + HEVC. - if (hasNativeSubs || videoCodecNeedsMasterSignaling || hasAudioRendition) + if (hasNativeSubs || videoCodecNeedsMasterSignaling || hasAudioRendition || hasIFrameRendition) && routingSafeForMaster { return true } return sourceIsHDR && panelReadyForHDR } @@ -2142,7 +2249,11 @@ public final class HLSVideoEngine: @unchecked Sendable { public var hasServedMediaSegment: Bool { server?.hasServedMediaSegment ?? false } /// Flip the serving flag after the engine has reloaded the media playlist on a display rejection. - func markServingMediaAfterFallback() { servingMasterPlaylist = false } + func markServingMediaAfterFallback() { + servingMasterPlaylist = false + // AE#682: the rendition lives only in the master this session just stopped serving. + tearDownIFrameRendition() + } // MARK: - Diagnostics @@ -2245,6 +2356,10 @@ public final class HLSVideoEngine: @unchecked Sendable { return (producer, cache, server, demuxer, audioBridge) } + /// AE#514: bytes of the played streams by presentation time, fed by every producer of the session + /// (initial, seek restart, live reopen, revive), read by the telemetry sampler at the playhead. + let playedMediaLedger = PlayedMediaLedger() + /// Bytes this session pulled from the SOURCE, across every demuxer it has had (see /// `retiredDemuxerBytes`). Not the same link as `LiveTelemetry.networkTransferredBytes`, which on /// the native path counts what AVPlayer pulled from the loopback server. @@ -2399,15 +2514,29 @@ public final class HLSVideoEngine: @unchecked Sendable { return prov?.sealedLiveTargetDurationSeconds } - func scrubThumbnailSource(atSeconds seconds: Double) -> (data: Data, segmentIndex: Int)? { + /// A resident segment a scrub still can decode from. Resolving it reads nothing, so an extractor + /// LRU hit costs no I/O; `makeReader()` maps the segment file rather than reading it into the heap + /// (audit SEG-2). SegmentCache only ever replaces or unlinks a segment file, never rewrites it in + /// place, so a mapping stays valid for as long as the reader holds it. + struct ScrubThumbnailSource: Sendable { + let segmentIndex: Int + let initData: Data + let segmentURL: URL + + func makeReader() -> DataIOReader? { + guard let segment = try? Data(contentsOf: segmentURL, options: .alwaysMapped) else { return nil } + return DataIOReader(parts: [initData, segment]) + } + } + + func scrubThumbnailSource(atSeconds seconds: Double) -> ScrubThumbnailSource? { restartLock.lock() let prov = provider restartLock.unlock() guard let prov else { return nil } - guard let seg = prov.thumbnailSegment(atSeconds: seconds) else { return nil } - guard let initData = prov.peekInitSegment(), - let segData = try? Data(contentsOf: seg.fileURL) else { return nil } - return (initData + segData, seg.index) + guard let seg = prov.thumbnailSegment(atSeconds: seconds), + let initData = prov.peekInitSegment() else { return nil } + return ScrubThumbnailSource(segmentIndex: seg.index, initData: initData, segmentURL: seg.fileURL) } public func stop() { @@ -2422,6 +2551,8 @@ public final class HLSVideoEngine: @unchecked Sendable { // while the pump exits a parked HTTP byte-range read). restartLock.lock() sessionEpoch &+= 1 + retentionClaim?.release() + retentionClaim = nil let p = producer producer = nil let s = server @@ -2439,18 +2570,25 @@ public final class HLSVideoEngine: @unchecked Sendable { preopenedDemuxer = nil let prov = provider provider = nil + let iFrames = iFrameRendition + iFrameRendition = nil + closeUnusedCustomIFrameReader() savedVideoConfig = nil savedAudioConfig = nil let ownedParams = ownedCodecParams ownedCodecParams = [] let reopening = reopenDemuxer reopenDemuxer = nil + let restartReopening = restartReopenDemuxer + restartReopenDemuxer = nil // #199: factory-vended ingest reader feeding the current demuxer; session-owned, closed here. let reopenReader = reopenCustomReader reopenCustomReader = nil segmentPlan = [] restartLock.unlock() reopening?.markClosed() + // The restart that opened it closes it once it sees the epoch moved. + restartReopening?.markClosed() // Close before waitForFinish: cancels the reader's FIFO so a pump parked in a blocking // custom-IO read unblocks (mirrors markClosed for URL demuxers). reopenReader?.close() @@ -2459,6 +2597,7 @@ public final class HLSVideoEngine: @unchecked Sendable { // Wake LL-HLS blocking-reload waiters; without this they sleep out their full 18-30 s timeout. prov?.cancelWaiters() + prov?.setIFrameSource(nil) // markClosed unblocks a live pump parked in the AVIO reconnect loop (exits on closed flag, // not the producer cancel flag). Without this, waitForFinish blocks ~3 s while reconnects @@ -2470,6 +2609,7 @@ public final class HLSVideoEngine: @unchecked Sendable { // Detached cleanup: producer waitForFinish must precede demuxer/cache/server close // (pump accesses them during unwind). ownedParams released last (pump read them). Task.detached { + iFrames?.shutdown() _ = p?.waitForFinish(timeout: 3.0) s?.stop() c?.close() @@ -2485,6 +2625,92 @@ public final class HLSVideoEngine: @unchecked Sendable { stop() } + // MARK: - I-frame rendition (AE#682) + + private func makeIFrameRendition(plan: [Segment], cache segmentCache: SegmentCache) -> IFrameRendition? { + guard let cfg = savedVideoConfig, !plan.isEmpty else { return nil } + let directory = segmentCache.sessionDir.appendingPathComponent("iframes", isDirectory: true) + let staging = directory.appendingPathComponent("staging", isDirectory: true) + do { + try FileManager.default.createDirectory(at: staging, withIntermediateDirectories: true) + } catch { + EngineLog.emit("[HLSVideoEngine] i-frame rendition: staging dir failed: \(error)", + category: .session) + return nil + } + let builder = IFrameFragmentBuilder( + video: MP4SegmentMuxer.VideoConfig( + codecpar: cfg.codecpar, timeBase: cfg.timeBase, + codecTagOverride: cfg.codecTagOverride, doviConfig: cfg.doviConfig, + colorOverride: cfg.colorOverride, extradataOverride: cfg.extradataOverride, + convertsAnnexBSamples: cfg.convertsAnnexBSamples, + nalFramingLatch: cfg.nalFramingLatch), + stagingDir: staging) + let framing = cfg.nalFramingOverride ?? A53SEIParser.nalFraming( + codec: cfg.codecpar.pointee.codec_id == AV_CODEC_ID_HEVC ? .hevc : .h264, + extradata: cfg.codecpar.pointee.extradata.map { UnsafePointer($0) }, + size: Int(cfg.codecpar.pointee.extradata_size)) + let url = sourceURL, headers = sourceHTTPHeaders, custom = customIFrameReader + // The side reader owns the clone from here; `stop()` must not close it a second time. + customIFrameReader = nil + let reader = IFrameSideReader( + open: { dem in + if let custom { + try dem.open(reader: custom.reader, formatHint: custom.formatHint, + profile: .iFrameSideDemuxer) + } else { + try dem.open(url: url, extraHeaders: headers, profile: .iFrameSideDemuxer, isLive: false) + } + }, + cleanup: { custom?.reader.close() }, + convertP7ToProfile81: cfg.convertP7ToProfile81, + nalFraming: framing) + let entries = plan.map { + IFrameRendition.Entry(startPts: $0.startPts, startSeconds: $0.startSeconds, + durationSeconds: $0.durationSeconds) + } + return IFrameRendition( + entries: entries, + cache: IFramePayloadCache(directory: directory.appendingPathComponent("payloads", isDirectory: true)), + readPayload: { reader.payload(startPts: $0.startPts) }, + buildFragment: { payload, index, entry in + builder.build(payload: payload, index: index, + startSeconds: entry.startSeconds, durationSeconds: entry.durationSeconds) + }, + waitForLink: { [weak self] shouldStop in + // A seek or a producer restart owns the link for a moment; a thumbnail can wait that + // out, but not ordinary fetching, or it would stall whenever the pump is busy. + let deadline = Date().addingTimeInterval(2) + while Date() < deadline, !shouldStop(), let self, + self.restartInFlight || (self.sideReaderLinkGate?.state.seeking ?? false) { + Thread.sleep(forTimeInterval: 0.05) + } + }, + // The budget learns its limits mid-session (a 429 halves them and arms a pacer), so the + // check `start()` made is repeated before every read. A custom reader has no origin. + sourceReadsAllowed: { + custom != nil || !(OriginRequestBudget.shared.requiresSerialRequests(url) + || OriginRequestBudget.shared.isPaced(url)) + }, + interruptReads: { reader.interrupt() }, + closeReader: { reader.close() }) + } + + /// Stop answering I-frame requests. The shutdown waits for a read in flight, so it runs off the + /// caller's thread; the provider is cleared first, so no new request reaches the rendition. The + /// reference stays, because `stop()` has to drain the same shutdown before it frees the codec + /// parameters the builder reads. + private func tearDownIFrameRendition() { + provider?.setIFrameSource(nil) + guard let rendition = iFrameRendition else { return } + DispatchQueue.global(qos: .utility).async { rendition.shutdown() } + } + + private func closeUnusedCustomIFrameReader() { + customIFrameReader?.reader.close() + customIFrameReader = nil + } + // MARK: - Producer construction + restart /// Allocate a new `HLSSegmentProducer` at the given segment index (initial bring-up and scrub restarts). @@ -2543,6 +2769,7 @@ public final class HLSVideoEngine: @unchecked Sendable { segmentBoundaries.append(last.endPts) } + let producerEpoch = nextProducerEpoch() let prod = try HLSSegmentProducer( demuxer: dem, videoStreamIndex: videoStreamIndex, @@ -2582,22 +2809,24 @@ public final class HLSVideoEngine: @unchecked Sendable { // AE#464: read here rather than pushed, so every producer this session builds (seek // restart, live reopen, #99 revive) cuts with the offset currently in force. audioDelaySeconds: audioDelaySeconds, - epoch: nextProducerEpoch() + epoch: producerEpoch ) // #240: threaded onto every producer (initial + restart), like the wedge-detector providers // below. The side readers read one gate for the whole session, so a restart must not leave // a gap where nobody claims the link. prod.sideReaderLinkGate = sideReaderLinkGate + prod.playedMediaLedger = playedMediaLedger prod.onFirstHDR10PlusDetected = { [weak self] in self?.notifyHDR10PlusOnce() } prod.onVideoShiftKnown = { [weak self] shiftPts, firstItemTfdtPts, normalizationShiftPts in self?.handleVideoShiftKnown( shiftPts, firstItemTfdtPts: firstItemTfdtPts, - normalizationShiftPts: normalizationShiftPts) + normalizationShiftPts: normalizationShiftPts, producerEpoch: producerEpoch) } prod.onLiveTimelineRebase = { [weak self] shiftPts, seamOutputSeconds in - self?.handleLiveTimelineRebase(shiftPts, seamOutputSeconds: seamOutputSeconds) + self?.handleLiveTimelineRebase(shiftPts, seamOutputSeconds: seamOutputSeconds, + producerEpoch: producerEpoch) } prod.onPumpFinished = { [weak self, weak prod] reason in guard let self, let prod else { return } @@ -2635,9 +2864,38 @@ public final class HLSVideoEngine: @unchecked Sendable { rebuildSubtitleTapRoutes() } armSubtitleTap(on: prod) + // Audit HLS-101: the first producer is built before the provider exists and `start()` wires + // it; every later one (live reopen, in-place rebuild, AE#222 rebuild) is wired here. + if let prov = provider { + wireProviderCallbacks(prod, to: prov) + } return prod } + /// The producer-to-provider reports the playlist is built from. A live producer without them + /// cuts segments the playlist never lists (audit HLS-101). + func wireProviderCallbacks(_ prod: HLSSegmentProducer, to prov: VideoSegmentProvider) { + if isLiveSession { + prod.onLiveSegmentFinalized = { [weak prov] index, durationSeconds, startPtsSeconds, discontinuous in + prov?.appendLiveSegment(index: index, + startSeconds: startPtsSeconds, + durationSeconds: durationSeconds, + discontinuous: discontinuous) + } + prod.onLiveSegmentSound = { [weak prov] index, first, last in + prov?.noteLiveSegmentSound(index: index, firstSeconds: first, lastSeconds: last) + } + // AE#443: the runaway park has to sit above the window this session actually serves, or it + // bounds the window instead of backstopping it, and its enforcement (a sleeping read + // thread) stops the origin from being drained. + prod.liveResidentCapProvider = { [weak prov] in prov?.liveResidentParkCap() ?? 0 } + } else if sequentialOrigin { + prod.onSequentialSegmentFinalized = { [weak prov] index, durationSeconds in + prov?.appendSequentialSegmentDuration(index: index, durationSeconds: durationSeconds) + } + } + } + // MARK: - Live source-loss recovery /// Max reopen attempts per lost-source event (AVIO absorbs transient drops internally; @@ -2658,12 +2916,13 @@ public final class HLSVideoEngine: @unchecked Sendable { static let maxLiveMuxerRebuildCycles = 3 private func handleVideoShiftKnown(_ shiftPts: Int64, firstItemTfdtPts: Int64, - normalizationShiftPts: Int64) { + normalizationShiftPts: Int64, producerEpoch: UInt64) { let seconds = shiftPts == Int64.min ? 0 : Double(shiftPts) * sourceVideoTbSeconds let seamItemSeconds = Double(firstItemTfdtPts) * sourceVideoTbSeconds // Live rebases the whole timeline at a program boundary and nothing older comes back on // screen, so its axis is the epoch's own and it publishes here as it always has. guard !isLiveSession else { + guard isInstalledProducer(producerEpoch) else { return } // Live has no placement model: the epoch's shift IS the axis, so the displacement this // session tracks for VOD stays where it is (nothing below reads it on a live session). publishPlaylistShift(seconds, seamItemSeconds: seamItemSeconds) @@ -2674,10 +2933,21 @@ public final class HLSVideoEngine: @unchecked Sendable { // epoch AVPlayer never fetches from must not move the clock at all. let index = segmentIndexForPlaylistTime(seamItemSeconds) anchorShiftLock.lock() + // Audit SEG-104: a restart that was replaced while its first read was in flight can still + // open its gate. Its epoch describes bytes nobody will store, and recording it would drop + // the entries at and above its index. + guard installedProducerEpoch == producerEpoch else { + anchorShiftLock.unlock() + EngineLog.emit( + "[HLSVideoEngine] gate open at seg\(index) from a superseded producer, ignored", + category: .session) + return + } // AE#412: a re-cut opens below its boundary on purpose, and AVPlayer places what it produces // at its own tfdt, so the epoch is worth nothing to the axis. Recording zero still drops the // entries at and above it, which is what the rewrite calls for. - let isRecut = recutIndices.remove(index) != nil + let isRecut = recutMark?.isOpened(byProducerEpoch: producerEpoch, at: index) == true + if isRecut { recutMark = nil } // PR #533: the run keeps the source-to-item normalization its bytes were written with, even // where its opening segment has no placement offset left to carry. Those are the same number // only on a source whose timestamps start at zero. @@ -3450,6 +3720,7 @@ public final class HLSVideoEngine: @unchecked Sendable { /// already within reach below the target, or a re-cut whose gate did not open in time. func preparedSeekLanding(itemSeconds: Double) -> Double { guard !isLiveSession, let provider else { return itemSeconds } + dropUnstartedRecut() let index = segmentIndexForPlaylistTime(itemSeconds) guard let reach = provider.videoReach(at: index) else { return itemSeconds } guard let advertised = advertisedStartSeconds(index) else { return itemSeconds } @@ -3465,17 +3736,15 @@ public final class HLSVideoEngine: @unchecked Sendable { + "and no random-access point is within reach below it; re-cutting from the covering one", category: .session ) - gateOpenCondition.lock() - lastGateOpen = nil - gateOpenCondition.unlock() - anchorShiftLock.lock() - recutIndices.insert(index) - anchorShiftLock.unlock() - requestRestart(at: index, authoritative: true) + markRecut(at: index) + // Audit HLS-4: off this task, so the gate wait below bounds the whole re-cut. Inline, an idle + // coalescer ran the restart here (a 5 s stop wait, a #79 reopen, the demuxer seek) before the + // wait began, all outside the seek's 8 s landing bound. + DispatchQueue.global(qos: .userInitiated).async { [weak self] in + self?.requestRestart(at: index, authoritative: true) + } + // Audit HLS-105: a timeout leaves the mark with the restart, whose gate may open later. guard let opened = awaitGateOpen(forIndex: index, timeout: Self.recutGateWaitSeconds) else { - anchorShiftLock.lock() - recutIndices.remove(index) - anchorShiftLock.unlock() EngineLog.emit( "[HLSVideoEngine] #412 seg\(index) re-cut did not open a gate within " + "\(String(format: "%.1f", Self.recutGateWaitSeconds))s; seeking on the uncorrected position", @@ -3495,6 +3764,54 @@ public final class HLSVideoEngine: @unchecked Sendable { return itemSeconds } + /// Audit HLS-105: an AE#412 re-cut, owned by the restart that performs it rather than by the seek + /// waiting on it. The restart binds its producer's epoch at install, and only that producer's gate + /// open is the re-cut; the seek's wait timing out leaves it alone. + struct RecutMark: Equatable { + let index: Int + private(set) var producerEpoch: UInt64? + + init(index: Int) { self.index = index } + + /// The mark that survives a restart installing `producerEpoch` at `restartIndex`: bound when + /// it is the marked index, kept while the marked restart has yet to run, dropped once the + /// restart it was bound to has been replaced by one elsewhere. + func installing(producerEpoch epoch: UInt64, at restartIndex: Int) -> RecutMark? { + guard restartIndex == index else { return producerEpoch == nil ? self : nil } + var bound = self + bound.producerEpoch = epoch + return bound + } + + func isOpened(byProducerEpoch epoch: UInt64, at gateIndex: Int) -> Bool { + producerEpoch == epoch && gateIndex == index + } + } + + func markRecut(at index: Int) { + gateOpenCondition.lock() + lastGateOpen = nil + gateOpenCondition.unlock() + anchorShiftLock.lock() + recutMark = RecutMark(index: index) + anchorShiftLock.unlock() + } + + /// A newer seek supersedes a re-cut whose restart never installed a producer (dropped with the + /// coalescer's superseded slot, or failed), so a later unrelated restart at that index does not + /// inherit the mark. + private func dropUnstartedRecut() { + anchorShiftLock.lock() + if recutMark?.producerEpoch == nil { recutMark = nil } + anchorShiftLock.unlock() + } + + private func isInstalledProducer(_ epoch: UInt64) -> Bool { + anchorShiftLock.lock() + defer { anchorShiftLock.unlock() } + return installedProducerEpoch == epoch + } + /// AE#412 pure decision: whether the segment a cold seek lands in has to be re-cut before the /// seek goes out. /// @@ -3592,7 +3909,8 @@ public final class HLSVideoEngine: @unchecked Sendable { /// AVPlayer renders at ~buffer+holdback behind the producer edge, so the host must keep the OLD shift /// until playback crosses `seamOutputSeconds`. Internal `playlistShiftSeconds` tracks the edge immediately. /// #368: sequential chunk-seam rebases arrive here too, deliberately; same deferred-shift contract. - func handleLiveTimelineRebase(_ shiftPts: Int64, seamOutputSeconds: Double) { + func handleLiveTimelineRebase(_ shiftPts: Int64, seamOutputSeconds: Double, producerEpoch: UInt64) { + guard isInstalledProducer(producerEpoch) else { return } let seconds = shiftPts == Int64.min ? 0 : Double(shiftPts) * sourceVideoTbSeconds setPlaylistShiftSeconds(seconds) onPlaylistShiftRebased?(seconds, seamOutputSeconds) @@ -3692,6 +4010,12 @@ public final class HLSVideoEngine: @unchecked Sendable { /// AE#454: the placement is spent once the item that asked for it is running. func clearLiveRejoinStart() { provider?.clearLiveRejoinStart() } + /// See `VideoSegmentProvider.liveSegmentHeads(atOutputSeconds:)`. + func liveSegmentHeads(atOutputSeconds seconds: Double) + -> (index: Int, secondsIntoSegment: Double, pictureStart: Double, sound: (first: Double, last: Double)?)? { + provider?.liveSegmentHeads(atOutputSeconds: seconds) + } + /// See `VideoSegmentProvider.servedLiveRejoinPlacement`. var servedLiveRejoinPlacement: (timeOffset: Double, playlistStartOutputSeconds: Double)? { provider?.servedLiveRejoinPlacement @@ -3715,7 +4039,12 @@ public final class HLSVideoEngine: @unchecked Sendable { restartLock.unlock() } - func requestRestart(at idx: Int, authoritative: Bool = false) { + /// `reviving` names the dead producer and session a delayed revive was scheduled for (audit + /// HLS-104). They are checked again in the same `restartLock` hold as the admission: a seek + /// restart that began after the caller's own check would otherwise get the stale target queued + /// behind it. A restart already in flight is about to replace the dead producer, so it counts. + func requestRestart(at idx: Int, authoritative: Bool = false, + reviving revive: (deadProducer: HLSSegmentProducer?, sessionEpoch: UInt64)? = nil) { // A sequential origin has no restart. performRestart's demuxer seek has nowhere to land // on a non-seekable pb, and it ignores that failure: the new producer would keep reading // wherever the stream stands (or from byte 0 after a fresh reopen) and label those bytes @@ -3732,6 +4061,17 @@ public final class HLSVideoEngine: @unchecked Sendable { return } restartLock.lock() + if let revive, !Self.meteredReviveStillOwed( + sessionUnchanged: sessionEpoch == revive.sessionEpoch, + deadProducerInstalled: producer === revive.deadProducer && !restartCoalescer.isInFlight) { + restartLock.unlock() + EngineLog.emit( + "[HLSVideoEngine] revive at idx=\(idx) dropped at admission: a restart began or the " + + "dead producer was replaced after it was scheduled", + category: .session + ) + return + } let shouldRun = restartCoalescer.begin(idx, authoritative: authoritative) let seekTime = segmentStartSecondsLocked(idx) // under lock; segmentPlan guarded by restartLock (#38) restartLock.unlock() @@ -3807,6 +4147,23 @@ public final class HLSVideoEngine: @unchecked Sendable { return parts.joined(separator: " ") } + /// A new demuxer for the #79 / #169 reopen, registered for `stop()` before its blocking open, or + /// nil when a stop already superseded the restart at `epoch`. + func registerRestartReopenDemuxer(epoch: UInt64) -> Demuxer? { + restartLock.lock() + defer { restartLock.unlock() } + guard sessionEpoch == epoch else { return nil } + let fresh = Demuxer() + restartReopenDemuxer = fresh + return fresh + } + + private func unregisterRestartReopenDemuxer(_ dem: Demuxer) { + restartLock.lock() + if restartReopenDemuxer === dem { restartReopenDemuxer = nil } + restartLock.unlock() + } + // Driven exclusively through requestRestart(at:) so bursts coalesce (#35). private func performRestart(at idx: Int) { restartGate.lock() @@ -3865,9 +4222,8 @@ public final class HLSVideoEngine: @unchecked Sendable { // AE#169 round 2 takes the same path when the pump exited BECAUSE the demuxer's read // threw: the connection is known-bad, so the revive gets one fresh connection instead // of seeking the demuxer that just failed. - if !isLiveSession, sideAudioDemuxer == nil { + if !isLiveSession, sideAudioDemuxer == nil, let fresh = registerRestartReopenDemuxer(epoch: epoch) { let reopenStart = DispatchTime.now() - let fresh = Demuxer() do { // .restartReopen: bounded find_stream_info budget; the FULL playback budget was // the bulk of a 44 s wedge-reopen over WAN (#93 residual). The pass itself must @@ -3890,6 +4246,7 @@ public final class HLSVideoEngine: @unchecked Sendable { category: .session ) } catch { + unregisterRestartReopenDemuxer(fresh) fresh.close() EngineLog.emit( "[HLSVideoEngine] restart at idx=\(idx): " @@ -3925,6 +4282,7 @@ public final class HLSVideoEngine: @unchecked Sendable { // Re-validate: a stop() landing during waits bumped sessionEpoch; don't resurrect into a torn-down session. restartLock.lock() + if let freshDemuxer, restartReopenDemuxer === freshDemuxer { restartReopenDemuxer = nil } guard sessionEpoch == epoch else { restartLock.unlock() // #79: a reopened demuxer (replacing a wedged one) must not leak when stop() superseded us. @@ -3952,6 +4310,9 @@ public final class HLSVideoEngine: @unchecked Sendable { do { let newProd = try makeProducer(baseIndex: idx) producer = newProd + anchorShiftLock.lock() + recutMark = recutMark?.installing(producerEpoch: newProd.epoch, at: idx) + anchorShiftLock.unlock() restartLock.unlock() newProd.start() } catch { diff --git a/Sources/AetherEngine/Video/IFrameFragmentBuilder.swift b/Sources/AetherEngine/Video/IFrameFragmentBuilder.swift new file mode 100644 index 000000000..98d95312c --- /dev/null +++ b/Sources/AetherEngine/Video/IFrameFragmentBuilder.swift @@ -0,0 +1,60 @@ +// Sources/AetherEngine/Video/IFrameFragmentBuilder.swift +import Foundation +import AetherLibavcodec + +/// AE#682: wraps one keyframe into one fMP4 fragment on the main rendition's timeline. +/// +/// A fresh muxer per fragment, deliberately: the session muxer requires strictly increasing dts, +/// and AVKit asks for I-frames in any order (an overview across the whole title first, then +/// wherever the finger is). The init carries no timing, so every muxer writes the same one. +final class IFrameFragmentBuilder { + struct Output { + let initSegment: Data + let fragment: Data + } + + private let video: MP4SegmentMuxer.VideoConfig + private let stagingDir: URL + + init(video: MP4SegmentMuxer.VideoConfig, stagingDir: URL) { + self.video = video + self.stagingDir = stagingDir + } + + func build(payload: Data, index: Int, startSeconds: Double, durationSeconds: Double) -> Output? { + guard !payload.isEmpty, payload.count <= Int(Int32.max), + video.timeBase.num > 0, video.timeBase.den > 0 else { return nil } + var initBytes = Data() + guard let muxer = try? MP4SegmentMuxer( + initialSegmentIndex: index, sessionDir: stagingDir, video: video, audio: nil, + onInitCaptured: { initBytes = $0 }) else { return nil } + var completed: URL? + defer { + _ = muxer.finalize() + if let completed { try? FileManager.default.removeItem(at: completed) } + } + + var packet = av_packet_alloc() + defer { av_packet_free(&packet) } + guard let packet, av_new_packet(packet, Int32(payload.count)) == 0, + let dst = packet.pointee.data else { return nil } + payload.copyBytes(to: dst, count: payload.count) + + let ticksPerSecond = Double(video.timeBase.den) / Double(video.timeBase.num) + let start = Int64((startSeconds * ticksPerSecond).rounded()) + packet.pointee.pts = start + packet.pointee.dts = start + packet.pointee.duration = max(1, Int64((durationSeconds * ticksPerSecond).rounded())) + packet.pointee.flags = AV_PKT_FLAG_KEY + packet.pointee.stream_index = muxer.videoOutputStreamIndex + av_packet_rescale_ts(packet, video.timeBase, muxer.muxerVideoTimeBase) + + guard muxer.writePacket(packet).rc >= 0, + case .completed(let path, _) = muxer.cutFragmentForNextSegment(index + 1) else { return nil } + completed = path + guard let fragment = try? Data(contentsOf: path), !fragment.isEmpty, !initBytes.isEmpty else { + return nil + } + return Output(initSegment: initBytes, fragment: fragment) + } +} diff --git a/Sources/AetherEngine/Video/IFramePayloadCache.swift b/Sources/AetherEngine/Video/IFramePayloadCache.swift new file mode 100644 index 000000000..b5cdd02be --- /dev/null +++ b/Sources/AetherEngine/Video/IFramePayloadCache.swift @@ -0,0 +1,80 @@ +// Sources/AetherEngine/Video/IFramePayloadCache.swift +import Foundation + +/// AE#682: keyframe payloads read for the I-frame rendition, on disk with a byte cap. It holds the +/// raw keyframe rather than the built fragment so a neighbour can be re-stamped for an index whose +/// own read failed. +final class IFramePayloadCache: @unchecked Sendable { + private let directory: URL + private let byteLimit: Int + private let lock = NSLock() + private var sizes: [Int: Int] = [:] + private var recency: [Int] = [] + private var totalBytes = 0 + + init(directory: URL, byteLimit: Int = 64 * 1024 * 1024) { + self.directory = directory + self.byteLimit = byteLimit + } + + private func url(_ index: Int) -> URL { + directory.appendingPathComponent("payload-\(index).bin") + } + + private func forgetLocked(_ index: Int) { + guard let size = sizes.removeValue(forKey: index) else { return } + totalBytes -= size + recency.removeAll { $0 == index } + try? FileManager.default.removeItem(at: url(index)) + } + + func payload(for index: Int) -> Data? { + lock.lock(); defer { lock.unlock() } + guard sizes[index] != nil else { return nil } + guard let data = try? Data(contentsOf: url(index)), !data.isEmpty else { + forgetLocked(index) + return nil + } + recency.removeAll { $0 == index } + recency.append(index) + return data + } + + func store(_ data: Data, for index: Int) { + // Eviction never takes the entry it makes room for, so one larger than the cap would stay + // over it. The caller serves such a payload uncached. + guard !data.isEmpty, data.count <= byteLimit else { return } + lock.lock(); defer { lock.unlock() } + forgetLocked(index) + do { + try FileManager.default.createDirectory(at: directory, withIntermediateDirectories: true) + try data.write(to: url(index), options: .atomic) + } catch { + return + } + sizes[index] = data.count + totalBytes += data.count + recency.append(index) + while totalBytes > byteLimit, let oldest = recency.first, oldest != index { + forgetLocked(oldest) + } + } + + func nearestIndex(to index: Int) -> Int? { + lock.lock(); defer { lock.unlock() } + return sizes.keys + .filter { FileManager.default.fileExists(atPath: url($0).path) } + .min { a, b in + let da = abs(a - index), db = abs(b - index) + return da != db ? da < db : a < b + } + } + + func removeAll() { + lock.lock(); defer { lock.unlock() } + sizes.removeAll() + recency.removeAll() + totalBytes = 0 + try? FileManager.default.removeItem(at: directory) + } +} diff --git a/Sources/AetherEngine/Video/IFrameRendition.swift b/Sources/AetherEngine/Video/IFrameRendition.swift new file mode 100644 index 000000000..ab38e9819 --- /dev/null +++ b/Sources/AetherEngine/Video/IFrameRendition.swift @@ -0,0 +1,151 @@ +// Sources/AetherEngine/Video/IFrameRendition.swift +import Foundation + +/// What the segment provider needs from an I-frame rendition (AE#682). +protocol IFrameSegmentSource: AnyObject, Sendable { + func initSegment() -> Data? + func fragment(at index: Int) -> Data? +} + +/// AE#682: answers `iframe_init.mp4` and `iframe{N}.mp4`. One request at a time: there is one side +/// demuxer, and AVKit asks for up to five keyframes at once. +final class IFrameRendition: IFrameSegmentSource, @unchecked Sendable { + struct Entry { + let startPts: Int64 + let startSeconds: Double + let durationSeconds: Double + } + + private let entries: [Entry] + private let cache: IFramePayloadCache + private let readPayload: (Entry) -> Data? + private let buildFragment: (Data, Int, Entry) -> IFrameFragmentBuilder.Output? + private let waitForLink: (_ shouldStop: () -> Bool) -> Void + private let sourceReadsAllowed: () -> Bool + private let interruptReads: () -> Void + private let closeReader: () -> Void + + private let queue = DispatchQueue(label: "aether.iframe-rendition") + private let stateLock = NSLock() + private var isShutDown = false + private var capturedInit: Data? + private var failureLog = IFrameLogThrottle() + + init(entries: [Entry], + cache: IFramePayloadCache, + readPayload: @escaping (Entry) -> Data?, + buildFragment: @escaping (Data, Int, Entry) -> IFrameFragmentBuilder.Output?, + waitForLink: @escaping (_ shouldStop: () -> Bool) -> Void = { _ in }, + sourceReadsAllowed: @escaping () -> Bool = { true }, + interruptReads: @escaping () -> Void = {}, + closeReader: @escaping () -> Void = {}) { + self.entries = entries + self.cache = cache + self.readPayload = readPayload + self.buildFragment = buildFragment + self.waitForLink = waitForLink + self.sourceReadsAllowed = sourceReadsAllowed + self.interruptReads = interruptReads + self.closeReader = closeReader + } + + private var shutDown: Bool { + stateLock.lock(); defer { stateLock.unlock() } + return isShutDown + } + + func initSegment() -> Data? { + queue.sync { + guard !shutDown else { return nil } + if capturedInit == nil { _ = buildLocked(at: 0) } + return capturedInit + } + } + + func fragment(at index: Int) -> Data? { + queue.sync { + guard !shutDown else { return nil } + return buildLocked(at: index) + } + } + + private func buildLocked(at index: Int) -> Data? { + guard entries.indices.contains(index) else { return nil } + let entry = entries[index] + var payload = cache.payload(for: index) + if payload == nil { + // An origin that went serial or started metering mid-session has one slot and a few + // tokens, and the pump needs them; AVKit's overview burst would take them all. + var read: Data? + if sourceReadsAllowed() { + waitForLink { self.shutDown } + guard !shutDown else { return nil } + let started = DispatchTime.now() + read = readPayload(entry) + // A read that teardown aborted is not a miss to paper over. + guard !shutDown else { return nil } + if let read { + let ms = Double(DispatchTime.now().uptimeNanoseconds - started.uptimeNanoseconds) / 1_000_000 + EngineLog.emit("[IFrameRendition] read idx=\(index) bytes=\(read.count) " + + "ms=\(String(format: "%.0f", ms))", category: .session, level: .verbose) + } + } + if let read { + cache.store(read, for: index) + payload = read + } else if let neighbour = cache.nearestIndex(to: index), + let substitute = cache.payload(for: neighbour) { + // AVKit keeps the last good picture for a keyframe it cannot fetch, and rate trick + // play wedges on a 404, so a near keyframe at the right time beats no answer. + payload = substitute + if let suppressed = failureLog.shouldEmit(now: Date()) { + EngineLog.emit("[IFrameRendition] no keyframe for idx=\(index), serving idx=\(neighbour) " + + "in its place (\(suppressed) more since the last line)", category: .session) + } + } else { + if let suppressed = failureLog.shouldEmit(now: Date()) { + EngineLog.emit("[IFrameRendition] no keyframe for idx=\(index) and nothing cached to stand in " + + "(\(suppressed) more since the last line)", category: .session) + } + return nil + } + } + guard let payload, let output = buildFragment(payload, index, entry) else { return nil } + if capturedInit == nil { capturedInit = output.initSegment } + return output.fragment + } + + /// Idempotent, and every call drains: the session frees the codec parameters the builder reads + /// as soon as its own call returns, so a second caller must not come back while the first is + /// still waiting for a request to finish. + func shutdown() { + stateLock.lock() + let first = !isShutDown + isShutDown = true + stateLock.unlock() + if first { interruptReads() } + queue.sync { + guard first else { return } + closeReader() + cache.removeAll() + } + } +} + +/// One line per window for a failure that repeats on every request (AVKit retries a missing +/// keyframe several times a second), with the count of the ones it swallowed. +struct IFrameLogThrottle { + var windowSeconds: TimeInterval = 5 + private var lastEmit = Date.distantPast + private var suppressed = 0 + + /// The number of lines swallowed since the last one, when this one should be written. + mutating func shouldEmit(now: Date) -> Int? { + guard now.timeIntervalSince(lastEmit) >= windowSeconds else { + suppressed += 1 + return nil + } + defer { suppressed = 0; lastEmit = now } + return suppressed + } +} diff --git a/Sources/AetherEngine/Video/IFrameRenditionEligibility.swift b/Sources/AetherEngine/Video/IFrameRenditionEligibility.swift new file mode 100644 index 000000000..58b2feec4 --- /dev/null +++ b/Sources/AetherEngine/Video/IFrameRenditionEligibility.swift @@ -0,0 +1,46 @@ +// Sources/AetherEngine/Video/IFrameRenditionEligibility.swift +import Foundation + +/// AE#682: whether a session serves the I-frame rendition. Pure, so the rule is testable without a +/// session. The rendition is all or nothing: AVKit shows a stale picture for a keyframe it cannot +/// fetch, so a session that cannot answer every listed entry lists none. +enum IFrameRenditionEligibility { + enum AbsentReason: String, Equatable { + case notRequested, live, planNotKeyframeAligned, sequentialOrigin, heldSourceConnection + case serialOrigin, discSource, noSecondReader, mediaPlaylistRouting + } + + enum Verdict: Equatable { + case served + case absent(AbsentReason) + } + + struct Inputs { + var requested: Bool + var isLive: Bool + var planBoundariesClaimRandomAccess: Bool + var sequentialOrigin: Bool + var heldSourceConnection: Bool + var originIsSerial: Bool + var isDiscSource: Bool + var secondReaderAvailable: Bool + } + + /// Everything except the master decision, which needs this verdict as one of its inputs. + static func candidate(_ i: Inputs) -> Verdict { + guard i.requested else { return .absent(.notRequested) } + if i.isLive { return .absent(.live) } + guard i.planBoundariesClaimRandomAccess else { return .absent(.planNotKeyframeAligned) } + if i.sequentialOrigin { return .absent(.sequentialOrigin) } + if i.heldSourceConnection { return .absent(.heldSourceConnection) } + if i.originIsSerial { return .absent(.serialOrigin) } + if i.isDiscSource { return .absent(.discSource) } + guard i.secondReaderAvailable else { return .absent(.noSecondReader) } + return .served + } + + static func resolve(candidate: Verdict, servingMaster: Bool) -> Verdict { + guard candidate == .served else { return candidate } + return servingMaster ? .served : .absent(.mediaPlaylistRouting) + } +} diff --git a/Sources/AetherEngine/Video/IFrameSideReader.swift b/Sources/AetherEngine/Video/IFrameSideReader.swift new file mode 100644 index 000000000..8e1bf2838 --- /dev/null +++ b/Sources/AetherEngine/Video/IFrameSideReader.swift @@ -0,0 +1,181 @@ +// Sources/AetherEngine/Video/IFrameSideReader.swift +import Foundation +import AetherLibavcodec + +/// AE#682: reads the keyframe that opens a plan segment, on a demuxer of its own. Only the bytes +/// are used; the fragment is stamped from the plan, so this demuxer's timestamp ladder (DTS on mp4, +/// PTS on Matroska, the #409 shift) never has to agree with the session's. +final class IFrameSideReader: @unchecked Sendable { + private let open: (Demuxer) throws -> Void + private let cleanup: () -> Void + private let convertP7ToProfile81: Bool + private let nalFraming: VideoNALFraming + private let readDeadlineSeconds: TimeInterval + private let now: () -> Date + + private let lock = NSLock() + private var demuxer: Demuxer? + private var isOpen = false + private var videoIndex: Int32 = -1 + private var interrupted = false + private var cleanedUp = false + private var consecutiveFailures = 0 + private var blockedUntil = Date.distantPast + + /// A keyframe sits at the seek target or right behind it; this bounds a source that claims an + /// index entry no keyframe backs. + private static let maxPacketsPerRead = 256 + + /// How long the reader refuses to touch the source after a failed open or read. Requests are + /// served one at a time and AVKit sends five at once, so without this a stalled connection + /// answers them a full read deadline apart; with it they fall through to a cached neighbour. + static func coolDownSeconds(afterConsecutiveFailures failures: Int) -> TimeInterval { + failures <= 1 ? 5 : 30 + } + + /// `open` receives a demuxer that `interrupt()` can already reach, so a slow open is abortable. + /// `cleanup` runs once, from `close()`, whether or not the demuxer was ever opened. + init(open: @escaping (Demuxer) throws -> Void, + cleanup: @escaping () -> Void = {}, + convertP7ToProfile81: Bool = false, + nalFraming: VideoNALFraming = .lengthPrefixed(size: 4), + readDeadlineSeconds: TimeInterval = 8, + now: @escaping () -> Date = Date.init) { + self.open = open + self.cleanup = cleanup + self.convertP7ToProfile81 = convertP7ToProfile81 + self.nalFraming = nalFraming + self.readDeadlineSeconds = readDeadlineSeconds + self.now = now + } + + private var isInterrupted: Bool { + lock.lock(); defer { lock.unlock() } + return interrupted + } + + private func drop(_ dem: Demuxer) { + lock.lock() + if demuxer === dem { + demuxer = nil + isOpen = false + } + lock.unlock() + dem.markClosed() + dem.close() + } + + private func noteFailure(_ what: String) { + lock.lock() + guard !interrupted else { lock.unlock(); return } + consecutiveFailures += 1 + let coolDown = Self.coolDownSeconds(afterConsecutiveFailures: consecutiveFailures) + blockedUntil = now().addingTimeInterval(coolDown) + let failures = consecutiveFailures + lock.unlock() + EngineLog.emit("[IFrameSideReader] \(what); leaving the source alone for " + + "\(Int(coolDown))s (consecutive failures: \(failures))", category: .session) + } + + private func readyDemuxer() -> Demuxer? { + lock.lock() + if interrupted { lock.unlock(); return nil } + if let demuxer, isOpen { lock.unlock(); return demuxer } + let dem = Demuxer() + demuxer = dem + lock.unlock() + + let started = DispatchTime.now() + do { + try open(dem) + } catch { + drop(dem) + noteFailure("open failed: \(error)") + return nil + } + let index = dem.videoStreamIndex + guard index >= 0 else { + drop(dem) + noteFailure("open failed: no video stream") + return nil + } + dem.discardAllStreamsExcept([index]) + lock.lock() + if interrupted { + lock.unlock() + drop(dem) + return nil + } + isOpen = true + videoIndex = index + lock.unlock() + let ms = Double(DispatchTime.now().uptimeNanoseconds - started.uptimeNanoseconds) / 1_000_000 + EngineLog.emit("[IFrameSideReader] opened in \(String(format: "%.0f", ms))ms", category: .session) + return dem + } + + func payload(startPts: Int64) -> Data? { + lock.lock() + let held = interrupted || now() < blockedUntil + lock.unlock() + if held { return nil } + guard let dem = readyDemuxer() else { return nil } + if let data = read(dem, startPts: startPts) { + lock.lock(); consecutiveFailures = 0; lock.unlock() + return data + } + guard !isInterrupted else { return nil } + // The session treats a demuxer whose read failed as suspect-dead and reopens; so does this. + drop(dem) + noteFailure("read failed at pts=\(startPts)") + return nil + } + + private func read(_ dem: Demuxer, startPts: Int64) -> Data? { + dem.beginReadDeadline(secondsFromNow: readDeadlineSeconds) + defer { dem.endReadDeadline() } + if dem.isDiscSource || !dem.seek(to: startPts, streamIndex: videoIndex) { + guard let stream = dem.stream(at: videoIndex) else { return nil } + let tb = stream.pointee.time_base + guard tb.den > 0, dem.seek(to: Double(startPts) * Double(tb.num) / Double(tb.den)) else { + return nil + } + } + for _ in 0..? = packet + defer { av_packet_free(&owned) } + guard packet.pointee.stream_index == videoIndex, + (packet.pointee.flags & AV_PKT_FLAG_KEY) != 0 else { continue } + if convertP7ToProfile81 { + _ = DoviRpuConverter.convertPacketToProfile81(packet, framing: nalFraming) + } + guard let bytes = packet.pointee.data, packet.pointee.size > 0 else { return nil } + return Data(bytes: bytes, count: Int(packet.pointee.size)) + } + return nil + } + + /// Any thread. Aborts an open or a read in flight and makes every later call answer nil. + func interrupt() { + lock.lock() + interrupted = true + let dem = demuxer + lock.unlock() + dem?.markClosed() + } + + /// The reading thread, after `interrupt()`. + func close() { + lock.lock() + let dem = demuxer + demuxer = nil + isOpen = false + let runCleanup = !cleanedUp + cleanedUp = true + lock.unlock() + dem?.close() + if runCleanup { cleanup() } + } +} diff --git a/Sources/AetherEngine/Video/Issue65LivelockBreakers.swift b/Sources/AetherEngine/Video/Issue65LivelockBreakers.swift index f1c52e811..8a8151941 100644 --- a/Sources/AetherEngine/Video/Issue65LivelockBreakers.swift +++ b/Sources/AetherEngine/Video/Issue65LivelockBreakers.swift @@ -10,9 +10,10 @@ import Foundation /// (`SegmentCache.targetIndex`) reaches a release target. A genuine wedge is the consumer target /// frozen for `breakThresholdSeconds` while AVPlayer is stuck and issuing no forward segment request; /// a slow-but-advancing consumer (cold cache, throttled CDN) keeps nudging the target up and must -/// NEVER trip the breaker. Feed `observe(currentTarget:)` once per ~1 s poll (`ParkClock` makes the -/// caller's wakeups into seconds): it resets the stuck timer whenever the target MOVES, so only a -/// target that is frozen for the whole window trips. +/// NEVER trip the breaker, and neither must a quiet consumer that is still rendering (AE#649). +/// Feed `observe(currentTarget:)` once per ~1 s poll (`ParkClock` makes the caller's wakeups into +/// seconds): it resets the stuck timer whenever the target MOVES, so only a target that is frozen for +/// the whole window trips. struct BackpressureWedgeDetector { let breakThresholdSeconds: Int /// #93 retest fast path: trip after this many consecutive polls where the fetch target AND the @@ -27,10 +28,17 @@ struct BackpressureWedgeDetector { /// at all. `maxTargetSeen` cannot: it only ever climbs. private var lastTarget: Int private var stuckSeconds: Int = 0 + /// AE#649: what the slow path trips on. Polls since the target last moved in which the rendered + /// clock did not advance either; a poll whose clock advanced is held, not counted. Without a + /// rendered position every poll counts, so it equals `stuckSeconds` exactly as before. + private var idleSeconds: Int = 0 private var lastRenderedPosition: Double? private var flatSeconds: Int = 0 /// Diagnostic: whether the last `true` from `observe` came from the fast path. private(set) var lastTripFast = false + /// AE#649 diagnostic: whether the last poll saw the rendered clock move. The PARK line names a + /// quiet consumer that is playing out what it already holds, so it cannot be read as a stuck one. + private(set) var lastPollRendered = false /// AE#528 diagnostic: polls since the consumer's fetch target last moved. The PARK line carries /// it, so a park that is plain backpressure behind a consumer that keeps fetching (stuck=0s, a @@ -38,6 +46,11 @@ struct BackpressureWedgeDetector { /// instead of being inferred from `cacheTarget` deltas across log lines after the fact. var secondsSinceTargetMoved: Int { stuckSeconds } + /// AE#649 diagnostic: the count the slow path trips on (see `idleSeconds`). The PARK line carries + /// it beside `stuck=`, so a park behind a consumer that is quiet but playing (stuck climbing, idle + /// at 0) reads differently from one behind a consumer that stopped both fetching and rendering. + var secondsWithoutProgress: Int { idleSeconds } + /// Rendered-clock deltas below this are aliasing/representation jitter, not playback progress /// (one poll second of real playback advances the clock by ~1 s). static let renderedClockFlatEpsilon: Double = 0.1 @@ -54,6 +67,17 @@ struct BackpressureWedgeDetector { /// Returns `true` once the consumer fetch target has been frozen for `breakThresholdSeconds`, or /// (fast path) once target and rendered clock have both been frozen for `fastBreakThresholdSeconds`. /// + /// AE#649: with a rendered position wired, frozen for the slow path means frozen with NO + /// playback progress. A consumer can go quiet for longer than the threshold while playing + /// perfectly: AVPlayer on a cellular iPhone fetches in bursts, several segments at once and + /// then nothing for 24 to 58 s while it plays out a forward buffer of up to 109 s (the same + /// title on Wi-Fi fetched one segment every 3 to 7 s). Counting that silence tripped the + /// breaker every few minutes, and the #421 nudge seek it led to flushed the buffer on screen: a + /// half-second freeze of picture and sound, each time. Seconds in which the rendered clock + /// advanced are therefore held rather than counted. A consumer that really stopped fetching + /// still plays its buffer dry and then renders nothing, and that is the fast path's shape, so + /// it breaks within `fastBreakThresholdSeconds` of the clock going flat. + /// /// AE#528: frozen means UNCHANGED, not "no higher than before". A viewer scrubbing backwards /// declares a LOWER fetch target on every GET, and against a monotone high-water that was /// indistinguishable from a consumer that had stopped asking for anything: the busiest consumer @@ -70,8 +94,9 @@ struct BackpressureWedgeDetector { /// (AVPlayer wants to play but is starved, `timeControlStatus == .waitingToPlay`) keeps `wantsToPlay` /// true and still trips. Defaults to true so existing callers and live keep their prior behaviour. /// - /// `renderedPosition` feeds the fast path; nil (not wired: tests, live) keeps it inert. Any move - /// beyond the flat epsilon, forward or backward (a new seek landing), restarts the flat window. + /// `renderedPosition` feeds the fast path and holds the slow count while the clock moves; nil (not + /// wired: tests, live) keeps both inert. Any move beyond the flat epsilon, forward or backward (a new + /// seek landing), restarts the flat window. /// /// `hasStartedRendering` is the cold-startup guard: before AVPlayer has ever presented a frame /// (`timeControlStatus` never reached `.playing`), a flat rendered clock is normal pre-roll, NOT a @@ -87,7 +112,9 @@ struct BackpressureWedgeDetector { if currentTarget > maxTargetSeen { maxTargetSeen = currentTarget } lastTarget = currentTarget stuckSeconds = 0 + idleSeconds = 0 flatSeconds = 0 + lastPollRendered = false if let rendered = renderedPosition { lastRenderedPosition = rendered } return false } @@ -97,12 +124,20 @@ struct BackpressureWedgeDetector { if targetAdvanced { maxTargetSeen = currentTarget } stuckSeconds = targetMoved ? 0 : stuckSeconds + 1 var clockFlat = false + var clockMoved = false if let rendered = renderedPosition { if let last = lastRenderedPosition { clockFlat = abs(rendered - last) < Self.renderedClockFlatEpsilon + clockMoved = !clockFlat } lastRenderedPosition = rendered } + lastPollRendered = clockMoved + if targetMoved { + idleSeconds = 0 + } else if !clockMoved { + idleSeconds += 1 + } if targetAdvanced || !clockFlat { flatSeconds = 0 } else { @@ -112,7 +147,7 @@ struct BackpressureWedgeDetector { lastTripFast = true return true } - if stuckSeconds >= breakThresholdSeconds { + if idleSeconds >= breakThresholdSeconds { lastTripFast = false return true } diff --git a/Sources/AetherEngine/Video/LiveCadencePolicy.swift b/Sources/AetherEngine/Video/LiveCadencePolicy.swift index affdba151..8b97338ab 100644 --- a/Sources/AetherEngine/Video/LiveCadencePolicy.swift +++ b/Sources/AetherEngine/Video/LiveCadencePolicy.swift @@ -7,6 +7,11 @@ struct LiveCadenceEvidence: Sendable, Equatable { var closedCadenceSeconds: Double? /// Longest segment duration (EXTINF) the upstream has actually served, nil until the first arrival. var servedSegmentDurationSeconds: Double? + /// AE#684: the media the join took, summed EXTINF, nil until it has joined. + var joinBacklogSeconds: Double? = nil + /// AE#684: the reader has handed over the whole join and its consumer is waiting on it for more. + /// nil where nothing can say. + var joinIsSpent: Bool? = nil } /// Turns the OBSERVED arrival cadence of a live ingest source (`LiveArrivalCadenceMeter`, surfaced by the @@ -53,6 +58,12 @@ final class LiveCadencePolicy: @unchecked Sendable { /// AE#447: the running floor, the monotonic max of the CLOSED evidence. Not "the observed cadence": /// the open gap the gate holds open is deliberately not in here. private var measuredFloorSeconds: Double + /// AE#684: the served-segment term of that floor, kept apart. Monotonic like the floor. + private var longestUpstreamSegmentSeconds: Double = 0 + /// AE#684: what the join took. First value wins: a reopen joins again, and its seal is long taken. + private var joinedBacklogSeconds: Double? + /// AE#684: latched once true, so a later delivery that fills the reader again cannot unspend it. + private var joinSpent: Bool? /// - Parameters: /// - observe: reader's current `observedLiveCadenceSeconds`; nil until the first upstream arrival. @@ -96,6 +107,13 @@ final class LiveCadencePolicy: @unchecked Sendable { if let evidence = observeSealEvidence?() { if let served = evidence.servedSegmentDurationSeconds, served > 0 { measuredFloorSeconds = max(measuredFloorSeconds, served) + longestUpstreamSegmentSeconds = max(longestUpstreamSegmentSeconds, served) + } + if joinedBacklogSeconds == nil, let joined = evidence.joinBacklogSeconds, joined > 0 { + joinedBacklogSeconds = joined + } + if let spent = evidence.joinIsSpent, joinSpent != true { + joinSpent = spent } if let closed = evidence.closedCadenceSeconds, closed > 0 { measuredFloorSeconds = max(measuredFloorSeconds, closed) @@ -119,6 +137,31 @@ final class LiveCadencePolicy: @unchecked Sendable { } } + /// AE#684: the longest segment the upstream has served, on its own. Inside the floor above it is + /// one more term of a max that is divided by the client's patience; a segment duration is the + /// period the upstream delivers at, not a worst case, so the seal takes it whole + /// (`LiveEdgePolicy.targetDurationForUpstreamSegment`). nil until the first arrival. + var upstreamSegmentDurationSeconds: Double? { + lock.lock(); defer { lock.unlock() } + advanceLocked() + return longestUpstreamSegmentSeconds > 0 ? longestUpstreamSegmentSeconds : nil + } + + /// AE#684: whether the join has been handed over and consumed, bar the GOP the next upstream + /// delivery will close. A fact from the reader, nil where there is none to ask. + var joinIsSpent: Bool? { + lock.lock(); defer { lock.unlock() } + advanceLocked() + return joinSpent + } + + /// AE#684: the media the join listed (summed EXTINF), for the seal line. nil until it has joined. + var joinBacklogSeconds: Double? { + lock.lock(); defer { lock.unlock() } + advanceLocked() + return joinedBacklogSeconds + } + var blockingReloadEnabled: Bool { lock.lock(); defer { lock.unlock() } advanceLocked() diff --git a/Sources/AetherEngine/Video/MP4SegmentMuxer.swift b/Sources/AetherEngine/Video/MP4SegmentMuxer.swift index 68818467a..45fd56f63 100644 --- a/Sources/AetherEngine/Video/MP4SegmentMuxer.swift +++ b/Sources/AetherEngine/Video/MP4SegmentMuxer.swift @@ -72,6 +72,12 @@ final class MP4SegmentMuxer { /// has numOfArrays=0 (in-band parameter sets) and the engine rebuilt a proper hvcC with /// VPS/SPS/PPS arrays; the mp4 muxer writes extradata directly into the hvcC/avcC box. let extradataOverride: [UInt8]? + /// The video samples are Annex B while `extradataOverride` is a length-prefixed record: the + /// muxer converts each sample itself and keeps its in-band parameter sets, which movenc's own + /// `hvc1` conversion would drop. See `AnnexBSampleConverter`. + let convertsAnnexBSamples: Bool + /// The session's framing verdict for this track (audit BIT-104); nil gives the muxer its own. + let nalFramingLatch: NALFramingLatch? init( codecpar: UnsafePointer, @@ -79,7 +85,9 @@ final class MP4SegmentMuxer { codecTagOverride: String?, doviConfig: DoviConfigPolicy = .keep, colorOverride: ColorOverride? = nil, - extradataOverride: [UInt8]? = nil + extradataOverride: [UInt8]? = nil, + convertsAnnexBSamples: Bool = false, + nalFramingLatch: NALFramingLatch? = nil ) { self.codecpar = codecpar self.timeBase = timeBase @@ -87,6 +95,8 @@ final class MP4SegmentMuxer { self.doviConfig = doviConfig self.colorOverride = colorOverride self.extradataOverride = extradataOverride + self.convertsAnnexBSamples = convertsAnnexBSamples + self.nalFramingLatch = nalFramingLatch } } @@ -173,13 +183,30 @@ final class MP4SegmentMuxer { /// length-prefixed at all. Latched at init: it is a property of the configuration record that /// lands in the sample entry, and the AE#561 sanitizer walks every video sample with it. private let videoNALLengthPrefixSize: Int? + /// Latched from `VideoConfig.convertsAnnexBSamples`. + private let convertsAnnexBVideoSamples: Bool /// AE#561 harness switch: the sanitizer removes the only shape that reproduces a segment Apple's /// parser refuses, so the rung underneath it (the software-path escalation) would have nothing to /// be measured against. Read once from the environment, never set in a shipped configuration. static let nalChainSanitizerDisabled = ProcessInfo.processInfo.environment["AETHER_DISABLE_NAL_SANITIZER"] != nil /// How many video samples the AE#561 sanitizer has had to cut, over this muxer's life. - private var truncatedVideoSamples: Int = 0 + private(set) var truncatedVideoSamples: Int = 0 + /// Audit BIT-1: a video sample of this track walked exactly as a length-prefixed chain, so a + /// `00 00 01` head is a 256-511 byte length from here on, not an Annex B start code. The + /// session's latch when it passed one (audit BIT-104), so a rebuilt muxer keeps the verdict. + private let videoNALFraming: NALFramingLatch + + private func sanitizerCut(_ bytes: UnsafeRawBufferPointer, lengthPrefixSize: Int) -> Int? { + let confirmed = videoNALFraming.isConfirmed + let cut = NALUnitChain.completeRunLength( + bytes, lengthPrefixSize: lengthPrefixSize, framingConfirmed: confirmed) + if cut == nil, !confirmed, + NALUnitChain.walksExactly(bytes, lengthPrefixSize: lengthPrefixSize) { + videoNALFraming.confirm() + } + return cut + } /// Only AC-3 / E-AC-3 / TrueHD build their mp4 sample entry from a parsed packet (dac3/dec3/dmlp), /// so only they can hit the "moov before audio parsed" wedge and need the #64-flush guard. Shared with @@ -230,6 +257,20 @@ final class MP4SegmentMuxer { /// Output-TB DTS of the first video packet since the last flush; Int64.min = no window open yet. private var fragmentWindowFirstVideoDts: Int64 = Int64.min + /// AE#684: the sound handed to the segment being cut, first and last packet, in + /// `muxerAudioTimeBase`. A segment opens on a video keyframe and carries whatever audio the + /// source interleaved up to there, so where its sound begins against its picture is a property + /// of the source's mux, and it is what an item placed in that segment starts its audio from. + private var segmentSoundFirstPts: Int64 = Int64.min + private var segmentSoundLastPts: Int64 = Int64.min + + /// The span since the last call, nil when the segment carried no audio packet. Consumes it. + func takeSegmentSoundSpan() -> (first: Int64, last: Int64)? { + defer { segmentSoundFirstPts = Int64.min; segmentSoundLastPts = Int64.min } + guard segmentSoundFirstPts != Int64.min else { return nil } + return (segmentSoundFirstPts, segmentSoundLastPts) + } + let videoOutputStreamIndex: Int32 = 0 let audioOutputStreamIndex: Int32 = 1 @@ -256,6 +297,8 @@ final class MP4SegmentMuxer { self.audioDelaySeconds = audioDelaySeconds self.audioNeedsParsedPacketForMoov = audio.map { Self.audioNeedsParsedPacketForMoov($0.codecpar.pointee.codec_id) } ?? false + self.videoNALFraming = video.nalFramingLatch ?? NALFramingLatch() + self.convertsAnnexBVideoSamples = video.convertsAnnexBSamples // AE#561: the override, when there is one, is the record that reaches the sample entry. Both // carry the same width (the #19 rebuild keeps the source header's first 22 bytes), so this // only matters for a source whose own extradata is missing or Annex B. @@ -427,7 +470,8 @@ final class MP4SegmentMuxer { /// (a DTS reset) never triggers; boundTicks <= 0 disables the bound. static func bufferedTicksExceedsBound(firstDts: Int64, currentDts: Int64, boundTicks: Int64) -> Bool { guard boundTicks > 0, firstDts != Int64.min, currentDts >= firstDts else { return false } - return (currentDts - firstDts) >= boundTicks + let (span, overflow) = currentDts.subtractingReportingOverflow(firstDts) + return overflow || span >= boundTicks } // MARK: - Diagnostic probes @@ -604,16 +648,25 @@ final class MP4SegmentMuxer { if packet.pointee.dts != Int64.min { packet.pointee.dts &+= audioDelayTicks } } - let clean = timestampSanitizer.sanitize( + guard let clean = timestampSanitizer.sanitize( streamIndex: packet.pointee.stream_index, pts: packet.pointee.pts, dts: packet.pointee.dts - ) + ) else { + av_packet_unref(packet) + return (0, .none) + } packet.pointee.pts = clean.pts packet.pointee.dts = clean.dts let streamIndex = packet.pointee.stream_index + // Before the AE#561 sanitizer, which walks the sample as the length-prefixed chain the + // record declares. A packet with no start code is written as it came. + if convertsAnnexBVideoSamples, streamIndex == videoOutputStreamIndex { + _ = AnnexBSampleConverter.convertToLengthPrefixed(packet) + } + // #64 mid-segment flush bound: cap libavformat's interleaver RAM on a very long segment // (degenerate sparse-keyframe plan, or an audio stream that decodes to nothing) by emitting a // moof+mdat into the current staging file before the buffered span grows without bound. Tracked @@ -644,7 +697,7 @@ final class MP4SegmentMuxer { let lengthPrefixSize = videoNALLengthPrefixSize, let data = packet.pointee.data, packet.pointee.size > 0, - let complete = NALUnitChain.completeRunLength( + let complete = sanitizerCut( UnsafeRawBufferPointer(start: data, count: Int(packet.pointee.size)), lengthPrefixSize: lengthPrefixSize ) { @@ -689,6 +742,10 @@ final class MP4SegmentMuxer { // must keep the exact stock code path, no extra early fragment flush, so nothing perturbs its audio. if streamIndex == audioOutputStreamIndex { audioPacketWritten = true + if rc >= 0, clean.pts != Int64.min { + if segmentSoundFirstPts == Int64.min { segmentSoundFirstPts = clean.pts } + segmentSoundLastPts = clean.pts + } if audioNeedsParsedPacketForMoov, !moovFlushed, fragmentWindowFirstVideoDts != Int64.min { flushPendingFragment() } diff --git a/Sources/AetherEngine/Video/NALUnitChain.swift b/Sources/AetherEngine/Video/NALUnitChain.swift index 859d48579..24042142a 100644 --- a/Sources/AetherEngine/Video/NALUnitChain.swift +++ b/Sources/AetherEngine/Video/NALUnitChain.swift @@ -2,6 +2,21 @@ import Foundation import AetherLibavcodec import AetherLibavutil +/// Audit BIT-104: whether a video sample of the session's track has walked exactly as a +/// length-prefixed chain (BIT-1). Held by the session rather than by one muxer, because every seek, +/// restart and reload builds a fresh muxer, and a muxer's first sample is exactly the one a restart +/// lands on. +final class NALFramingLatch: @unchecked Sendable { + private let lock = NSLock() + private var confirmed: Bool + + init(confirmed: Bool = false) { self.confirmed = confirmed } + + var isConfirmed: Bool { lock.withLock { confirmed } } + + func confirm() { lock.withLock { confirmed = true } } +} + /// The length-prefixed NAL chain (avcC / hvcC framing) as Apple's fMP4 parser walks it. /// /// A sample in an mp4 video track is a run of NAL units, each introduced by a big-endian length of @@ -43,12 +58,20 @@ enum NALUnitChain { /// /// Zero means not one NAL unit in the payload is complete, which leaves the caller nothing to /// write. - static func completeRunLength(_ bytes: UnsafeRawBufferPointer, lengthPrefixSize: Int) -> Int? { + /// + /// `framingConfirmed` says an earlier sample of this track already walked exactly as a + /// length-prefixed chain. Audit BIT-1: with a 4-byte prefix, every first NAL of 256 to 511 bytes + /// starts `00 00 01 xx`, so the head test alone let exactly those damaged samples through. Once + /// the track is known to be length-prefixed that head is a length, not a start code. + static func completeRunLength( + _ bytes: UnsafeRawBufferPointer, lengthPrefixSize: Int, framingConfirmed: Bool = false + ) -> Int? { let count = bytes.count guard (1...4).contains(lengthPrefixSize), count >= lengthPrefixSize else { return nil } // An Annex B payload is not this framing at all, and reading its start code as a length would // cut every frame of a healthy stream down to nothing. - if count >= 3, bytes[0] == 0, bytes[1] == 0, bytes[2] == 1 { return nil } + let threeByteHeadIsLength = framingConfirmed && lengthPrefixSize == 4 + if !threeByteHeadIsLength, count >= 3, bytes[0] == 0, bytes[1] == 0, bytes[2] == 1 { return nil } if count >= 4, bytes[0] == 0, bytes[1] == 0, bytes[2] == 0, bytes[3] == 1 { return nil } var offset = 0 @@ -62,4 +85,11 @@ enum NALUnitChain { } return offset == count ? nil : offset } + + /// Whether the payload walks as a length-prefixed chain that ends exactly on its last byte. Annex B + /// data cannot do that by accident, since it would have to encode its own byte offsets. + static func walksExactly(_ bytes: UnsafeRawBufferPointer, lengthPrefixSize: Int) -> Bool { + guard let base = bytes.baseAddress?.assumingMemoryBound(to: UInt8.self) else { return false } + return VideoConfigRecord.walksAsLengthPrefixed(base, size: bytes.count, lengthSize: lengthPrefixSize) + } } diff --git a/Sources/AetherEngine/Video/NoCutStallWatchdog.swift b/Sources/AetherEngine/Video/NoCutStallWatchdog.swift index ce749db3a..32ca5cbfb 100644 --- a/Sources/AetherEngine/Video/NoCutStallWatchdog.swift +++ b/Sources/AetherEngine/Video/NoCutStallWatchdog.swift @@ -192,7 +192,7 @@ final class NoCutStallWatchdog: @unchecked Sendable { let readRate = stalledFor > 0 ? Double(progress) / stalledFor : 0 let ptsAdvance = (firstVideoPts != Int64.min && lastVideoPts != Int64.min && videoTimeBaseSeconds > 0) - ? Double(lastVideoPts - firstVideoPts) * videoTimeBaseSeconds + ? Double(SourceTimestampBounds.difference(lastVideoPts, firstVideoPts)) * videoTimeBaseSeconds : -1 switch HLSSegmentProducer.noCutStallAction( stalledFor: stalledFor, diff --git a/Sources/AetherEngine/Video/OutputTimestampSanitizer.swift b/Sources/AetherEngine/Video/OutputTimestampSanitizer.swift index fd3ebf18d..56db506fd 100644 --- a/Sources/AetherEngine/Video/OutputTimestampSanitizer.swift +++ b/Sources/AetherEngine/Video/OutputTimestampSanitizer.swift @@ -4,12 +4,16 @@ import Foundation struct OutputTimestampSanitizer { private var lastDtsByStream: [Int32: Int64] = [:] - mutating func sanitize(streamIndex: Int32, pts: Int64, dts: Int64) -> (pts: Int64, dts: Int64) { + /// nil when no dts after the stream's last one exists (it already sits at Int64.max, audit + /// SEG-101); the caller drops the packet. + mutating func sanitize(streamIndex: Int32, pts: Int64, dts: Int64) -> (pts: Int64, dts: Int64)? { guard dts != Int64.min else { return (pts, dts) } // NOPTS: nothing to enforce var outDts = dts if let last = lastDtsByStream[streamIndex], outDts <= last { - outDts = last + 1 + let (bumped, overflow) = last.addingReportingOverflow(1) + guard !overflow else { return nil } + outDts = bumped } let outPts = pts == Int64.min ? outDts : max(pts, outDts) // NOPTS pts collapses to dts (safe floor for audio and video) diff --git a/Sources/AetherEngine/Video/PacketRingBuffer.swift b/Sources/AetherEngine/Video/PacketRingBuffer.swift index ea7f4872b..89946d7bd 100644 --- a/Sources/AetherEngine/Video/PacketRingBuffer.swift +++ b/Sources/AetherEngine/Video/PacketRingBuffer.swift @@ -1,13 +1,21 @@ +import Darwin import Foundation -/// Keyframe-indexed disk-spooled DVR ring buffer. Eviction is keyframe-aligned (retained span always starts at a decodable keyframe). Bytes stored as flat files under a scratch dir; in-RAM index holds only metadata; `Data(contentsOf:,.alwaysMapped)` keeps RSS flat. NSLock guards the index (demux-thread appends, seek-thread reads). -// Thread-safe: all mutable state is guarded by `lock` (NSLock), so it is safe to share across the -// demux/seek/feeder threads and capture in @Sendable closures. +/// Keyframe-indexed disk-spooled DVR ring buffer. Eviction is keyframe-aligned (once the ring has +/// evicted, the retained span starts at a decodable keyframe). Packet bytes are appended into a few +/// large chunk files under a scratch dir; the in-RAM index holds one 24-byte record per packet. +/// +/// Audit PERF-101 / SEG-106: this used to write one atomically renamed file per packet and keep a +/// heap `URL` per index entry (554-586 B per entry measured, ~230 MB of index at a 90 min window, and +/// ~305 us of file work per packet). Audit VPERF-101: eviction was by time only and ran only after a +/// successful write, so a full disk stopped the ring, and with it the software live feeder, for good. +// Thread-safe: `lock` guards the index, the chunk table and the read-handle cache; `writeLock` +// serializes appends (tail chunk, tail offset). Lock order: writeLock, then lock. final class PacketRingBuffer: @unchecked Sendable { // MARK: - Public types - /// A single packet as returned by `packets(fromPts:)`. + /// A single packet as returned by `packet(atSeq:)` and `packets(fromPts:)`. struct Packet { let pts: Double let isKeyframe: Bool @@ -15,34 +23,114 @@ final class PacketRingBuffer: @unchecked Sendable { let bytes: Data } + /// Writes all of `bytes` at `offset` or throws. Injectable so tests can fill the disk. + typealias WriteAll = @Sendable (_ fd: Int32, _ bytes: UnsafeRawBufferPointer, _ offset: off_t) throws -> Void + + enum Failure: Error, Equatable { + case closed + case packetTooLarge + case capacityExceeded + } + + /// The software live route plays FROM this ring, so it is a playback buffer before it is seek + /// history: the quarter-of-free-space retention rule may read 0 on a nearly full volume, and a + /// ring that small evicts packets before the feeder reaches them. A disk that is really full is + /// handled by the evict-and-retry in `append`. + static let minimumLiveByteBudget = 64 << 20 + + /// The native path's retention funnel (min(2 GiB, a quarter of free space)) with the live floor. + static func liveByteBudget(volumeAvailableBytes: Int64?, capRelaxed: Bool = false) -> Int { + max(minimumLiveByteBudget, + HLSVideoEngine.sessionRetentionBudgetBytes(volumeAvailableBytes: volumeAvailableBytes, + capRelaxed: capRelaxed)) + } + // MARK: - Private types - private struct Entry { + struct Entry { let pts: Double - let isKeyframe: Bool - let isVideo: Bool - let fileURL: URL - let byteCount: Int + let offset: UInt32 + let length: UInt32 + let chunk: UInt32 + let flags: UInt8 + + static let keyframeFlag: UInt8 = 1 + static let videoFlag: UInt8 = 2 + var isKeyframe: Bool { flags & Self.keyframeFlag != 0 } + var isVideo: Bool { flags & Self.videoFlag != 0 } + } + + /// Closed when the last holder lets go, so a reader that picked a handle up under the lock can + /// finish its `pread` on a chunk that eviction unlinked meanwhile. + private final class ChunkHandle: @unchecked Sendable { + let id: UInt32 + let fd: Int32 + init(id: UInt32, fd: Int32) { + self.id = id + self.fd = fd + } + deinit { Darwin.close(fd) } + } + + private struct ChunkRecord { + let id: UInt32 + var bytes: Int } // MARK: - State private let lock = NSLock() + private let writeLock = NSLock() private let windowSeconds: Double + private let byteBudget: Int + private let chunkTargetBytes: Int private let scratch: URL + private let writeAll: WriteAll + /// The retained span is `entries[head...]`; the dead prefix is compacted away in batches. private var entries: [Entry] = [] - /// Sequence number of `entries[0]`; eviction advances this instead of renumbering. Feeder cursor below `firstSeq` = fell out of window. + private var head = 0 + /// Sequence number of `entries[head]`; eviction advances this instead of renumbering. Feeder cursor below `firstSeq` = fell out of window. private var firstSeq: Int = 0 - private var counter: Int = 0 + /// Absolute sequence numbers of retained keyframes, `keyframeSeqs[keyframeHead...]`. One per GOP, + /// so eviction and keyframe lookups walk GOPs instead of packets. + private var keyframeSeqs: [Int] = [] + private var keyframeHead = 0 private var edge: Double = -.infinity private var closed: Bool = false + /// Oldest first; the last record is the chunk the writer appends to. + private var chunks: [ChunkRecord] = [] + private var retainedDiskBytes = 0 + /// Most recently used first, at most `readHandleCap`. Opening a descriptor per retained chunk + /// would hold hundreds at a 2 GiB budget. + private var readHandles: [ChunkHandle] = [] + private static let readHandleCap = 4 + /// Audit SEG-105: the ring's directory sits beside the segment cache's under `aether-segments/` + /// and must read as live to its stale sweep. + private var markerFD: Int32 = -1 + + // Writer state, guarded by `writeLock`. + private var tail: ChunkHandle? + private var tailOffset = 0 + private var nextChunkID: UInt32 = 0 + private var loggedOtherWriteFailure = false // MARK: - Init / close - init(windowSeconds: Double, scratch: URL) throws { + init(windowSeconds: Double, scratch: URL, byteBudget: Int = .max, + chunkTargetBytes: Int = 4 << 20, writeAll: WriteAll? = nil) throws { self.windowSeconds = windowSeconds self.scratch = scratch + self.byteBudget = max(0, byteBudget) + // Several chunks per budget, so evicting the oldest one frees a fraction of it, not all. + let target = min(chunkTargetBytes, max(64 << 10, self.byteBudget / 16)) + self.chunkTargetBytes = max(1, min(target, Int(UInt32.max / 2))) + self.writeAll = writeAll ?? Self.pwriteAll + self.markerFD = SessionDirectoryLiveness.acquire(sessionDir: scratch, logPrefix: "[PacketRingBuffer]") + } + + deinit { + if markerFD >= 0 { Darwin.close(markerFD) } } /// Idempotent teardown. State is cleared synchronously so the ring is immediately @@ -52,33 +140,93 @@ final class PacketRingBuffer: @unchecked Sendable { lock.lock() guard !closed else { lock.unlock(); return } closed = true - entries.removeAll(keepingCapacity: false) - edge = -.infinity - counter = 0 + entries = [] + head = 0 firstSeq = 0 + keyframeSeqs = [] + keyframeHead = 0 + edge = -.infinity + chunks = [] + retainedDiskBytes = 0 + readHandles = [] + let marker = markerFD + markerFD = -1 lock.unlock() + // The writer's chunk would otherwise stay open, blocks and all, until the next append. An + // append in flight is waited out here, after the index is cleared, so readers never wait on it. + writeLock.lock() + tail = nil + tailOffset = 0 + writeLock.unlock() + DispatchQueue.global(qos: .userInitiated).async { [scratch] in try? FileManager.default.removeItem(at: scratch) + // Released after the removal, so the directory reads live until it is gone. + if marker >= 0 { Darwin.close(marker) } } } // MARK: - Writer func append(pts: Double, isKeyframe: Bool, isVideo: Bool, bytes: Data) throws { - let fileURL = scratch.appendingPathComponent("pkt-\(nextCounter()).bin") - try bytes.write(to: fileURL, options: [.atomic]) - let entry = Entry(pts: pts, isKeyframe: isKeyframe, isVideo: isVideo, fileURL: fileURL, byteCount: bytes.count) + try bytes.withUnsafeBytes { raw in + try append(pts: pts, isKeyframe: isKeyframe, isVideo: isVideo, bytes: raw) + } + } + + /// Takes the packet's memory as is, so the host appends straight from the `AVPacket` without an + /// intermediate copy. + func append(pts: Double, isKeyframe: Bool, isVideo: Bool, bytes: UnsafeRawBufferPointer) throws { + guard bytes.count <= Int(UInt32.max / 2) else { throw Failure.packetTooLarge } + writeLock.lock() + defer { writeLock.unlock() } + guard !isClosed else { + tail = nil + throw Failure.closed + } + + let placement: (chunk: UInt32, offset: Int) + do { + placement = try writeLocked(bytes) + } catch { + // Audit VPERF-101: eviction used to run only after a successful write, so once the volume + // filled, every append failed, nothing was ever freed, and the feeder waited at the live + // edge forever. Shrink the window and try once more before dropping the packet. Only for a + // full volume or quota: any other failure (no descriptors left, a purged tmp directory) + // would evict real history on every packet and still fail. + guard Self.isOutOfSpace(error) else { + noteWriteFailureOnce(error) + throw error + } + guard makeRoomAfterFailedWrite(incomingKeyframe: isVideo && isKeyframe) else { throw error } + placement = try writeLocked(bytes) + } + tailOffset = placement.offset + bytes.count + + var flags: UInt8 = 0 + if isKeyframe { flags |= Entry.keyframeFlag } + if isVideo { flags |= Entry.videoFlag } + let entry = Entry(pts: pts, offset: UInt32(placement.offset), length: UInt32(bytes.count), + chunk: placement.chunk, flags: flags) lock.lock() + guard !closed else { + lock.unlock() + tail = nil + throw Failure.closed + } + if isKeyframe { keyframeSeqs.append(firstSeq + (entries.count - head)) } entries.append(entry) if pts > edge { edge = pts } - let evictedURLs = evictLocked() + if let last = chunks.indices.last, chunks[last].id == placement.chunk { + retainedDiskBytes += tailOffset - chunks[last].bytes + chunks[last].bytes = tailOffset + } + let doomed = evictLocked() lock.unlock() - for url in evictedURLs { // delete outside the lock; removeItem is filesystem I/O - try? FileManager.default.removeItem(at: url) - } + Self.unlinkAll(doomed) } // MARK: - Reader @@ -86,25 +234,18 @@ final class PacketRingBuffer: @unchecked Sendable { func keyframePts(atOrBefore target: Double) throws -> Double? { lock.lock() defer { lock.unlock() } - return entries - .filter { $0.isKeyframe && $0.pts <= target } - .last - .map(\.pts) + return lastKeyframeIndexLocked(atOrBefore: target).map { entries[$0].pts } } /// Returns packets with `pts >= startPts`. Entries evicted between index snapshot and off-lock disk read are skipped (eviction is front-only, so skipping preserves keyframe alignment). func packets(fromPts startPts: Double) throws -> [Packet] { lock.lock() - let slice = entries.filter { $0.pts >= startPts } + let start = firstSeq + let seqs = entries[head...].indices.filter { entries[$0].pts >= startPts }.map { start + ($0 - head) } lock.unlock() - var packets = slice.compactMap { entry -> Packet? in - guard let data = try? Data(contentsOf: entry.fileURL, options: [.alwaysMapped, .uncached]) else { - return nil - } - return Packet(pts: entry.pts, isKeyframe: entry.isKeyframe, isVideo: entry.isVideo, bytes: data) - } - // Off-lock reads can race deferred eviction deletions: trim to the first video keyframe to guarantee a clean decode start. + var packets = seqs.compactMap { packet(atSeq: $0) } + // Off-lock reads can race eviction: trim to the first video keyframe to guarantee a clean decode start. if packets.contains(where: { $0.isVideo }), let kf = packets.firstIndex(where: { $0.isVideo && $0.isKeyframe }) { if kf > 0 { packets.removeFirst(kf) } @@ -119,31 +260,37 @@ final class PacketRingBuffer: @unchecked Sendable { var seqBounds: (first: Int, end: Int) { lock.lock() defer { lock.unlock() } - return (firstSeq, firstSeq + entries.count) + return (firstSeq, firstSeq + (entries.count - head)) + } + + /// Whether `seq` is a video packet, from the index alone, or nil if evicted or not yet appended. + /// Audit PERF-101: the audio pump and the feeder skip the other stream's packets through this, + /// where they used to map every packet file only to read the flag. + func isVideo(atSeq seq: Int) -> Bool? { + lock.lock() + defer { lock.unlock() } + return indexLocked(seq).map { entries[$0].isVideo } } - /// Returns packet for `seq`, or nil if evicted or not yet appended. Index lock NOT held across the disk read. + /// Returns packet for `seq`, or nil if evicted, not yet appended, or unreadable. Index lock NOT held across the disk read. func packet(atSeq seq: Int) -> Packet? { lock.lock() - let idx = seq - firstSeq - guard idx >= 0, idx < entries.count else { + guard let idx = indexLocked(seq) else { lock.unlock() return nil } let entry = entries[idx] + let cached = cachedReadHandleLocked(entry.chunk) lock.unlock() - guard let data = try? Data(contentsOf: entry.fileURL, - options: [.alwaysMapped, .uncached]) else { return nil } - return Packet(pts: entry.pts, isKeyframe: entry.isKeyframe, - isVideo: entry.isVideo, bytes: data) + guard let handle = cached ?? openReadHandle(entry.chunk), + let bytes = Self.read(handle.fd, offset: entry.offset, length: entry.length) else { return nil } + return Packet(pts: entry.pts, isKeyframe: entry.isKeyframe, isVideo: entry.isVideo, bytes: bytes) } func seq(forKeyframeAtOrBefore target: Double) -> Int? { lock.lock() defer { lock.unlock() } - return entries.indices - .last(where: { entries[$0].isKeyframe && entries[$0].pts <= target }) - .map { firstSeq + $0 } + return lastKeyframeIndexLocked(atOrBefore: target).map { firstSeq + ($0 - head) } } /// Sequence of the EARLIEST retained keyframe, or nil if the ring holds no keyframe yet. DVR reseed @@ -152,9 +299,7 @@ final class PacketRingBuffer: @unchecked Sendable { func firstKeyframeSeq() -> Int? { lock.lock() defer { lock.unlock() } - return entries.indices - .first(where: { entries[$0].isKeyframe }) - .map { firstSeq + $0 } + return keyframeHead < keyframeSeqs.count ? keyframeSeqs[keyframeHead] : nil } // MARK: - Still runs (#544) @@ -222,8 +367,7 @@ final class PacketRingBuffer: @unchecked Sendable { maxSpanSeconds: Double, reorderTail: Int) -> [Packet]? { lock.lock() - guard let startIdx = entries.indices - .last(where: { entries[$0].isKeyframe && entries[$0].pts <= target }) else { + guard let startIdx = lastKeyframeIndexLocked(atOrBefore: target) else { lock.unlock() return nil } @@ -232,7 +376,7 @@ final class PacketRingBuffer: @unchecked Sendable { let window = entries[startIdx.. Packet? in + window[seq - base].isVideo ? packet(atSeq: seq) : nil + } // Eviction between the snapshot and the off-lock reads would cost the run its keyframe, and // a run that does not open on one decodes as garbage. guard let first = run.first, first.isKeyframe else { return nil } @@ -252,32 +398,315 @@ final class PacketRingBuffer: @unchecked Sendable { var oldestPts: Double? { lock.lock() defer { lock.unlock() } - return entries.first.map(\.pts) + return head < entries.count ? entries[head].pts : nil + } + + /// PTS of the earliest retained keyframe: the oldest position a DVR rewind can land on. + var oldestKeyframePts: Double? { + lock.lock() + defer { lock.unlock() } + guard keyframeHead < keyframeSeqs.count, let idx = indexLocked(keyframeSeqs[keyframeHead]) else { + return nil + } + return entries[idx].pts + } + + /// The oldest position a rewind can land on, on the session axis, once eviction has taken history + /// the window alone would have kept. nil until then, so a young session keeps its window arithmetic. + /// Audit VPERF-101: the byte budget can shrink the ring below `windowSeconds`, and the transport rail + /// must not advertise history that was evicted for space. + func residentFloorSessionSeconds(sessionStartPts: Double) -> Double? { + lock.lock() + defer { lock.unlock() } + guard firstSeq > 0, sessionStartPts.isFinite, + keyframeHead < keyframeSeqs.count, let idx = indexLocked(keyframeSeqs[keyframeHead]) else { + return nil + } + return max(0, entries[idx].pts - sessionStartPts) + } + + /// Bytes the retained chunk files occupy, including the evicted prefix of the oldest chunk. + var diskBytes: Int { + lock.lock() + defer { lock.unlock() } + return retainedDiskBytes + } + + /// Heap the index occupies, dead prefix and spare capacity included. + var indexFootprintBytes: Int { + lock.lock() + defer { lock.unlock() } + return entries.capacity * MemoryLayout.stride + keyframeSeqs.capacity * MemoryLayout.stride } // MARK: - Internal - private func nextCounter() -> Int { + private var isClosed: Bool { lock.lock() defer { lock.unlock() } - let c = counter - counter += 1 - return c + return closed + } + + private func chunkURL(_ id: UInt32) -> URL { + scratch.appendingPathComponent("chunk-\(id).bin", isDirectory: false) } - /// Drop leading entries outside `edge - windowSeconds`, keyframe-aligned. Forward scan: backward scan walked ~150k entries per append at 80 pkt/s; forward scan touches a few hundred at most. Caller deletes returned URLs after releasing the lock. + private func indexLocked(_ seq: Int) -> Int? { + let offset = seq - firstSeq + guard !closed, offset >= 0, offset < entries.count - head else { return nil } + return head + offset + } + + private func lastKeyframeIndexLocked(atOrBefore target: Double) -> Int? { + var k = keyframeSeqs.count + while k > keyframeHead { + k -= 1 + if let idx = indexLocked(keyframeSeqs[k]), entries[idx].pts <= target { return idx } + } + return nil + } + + /// Writer side, `writeLock` held. Rolls to a fresh chunk when the packet would overflow a + /// non-empty one, then writes it at the tail offset. Nothing moves on failure: a partial write + /// is overwritten by the next attempt at the same offset. + private func writeLocked(_ bytes: UnsafeRawBufferPointer) throws -> (chunk: UInt32, offset: Int) { + if tail == nil || (tailOffset > 0 && tailOffset + bytes.count > chunkTargetBytes) { + try rollChunkLocked() + } + guard let tail else { throw Failure.closed } + try writeAll(tail.fd, bytes, off_t(tailOffset)) + return (tail.id, tailOffset) + } + + private func rollChunkLocked() throws { + guard nextChunkID < UInt32.max else { throw Failure.capacityExceeded } + let id = nextChunkID + let fd = open(chunkURL(id).path, O_CREAT | O_EXCL | O_RDWR | O_CLOEXEC, 0o600) + guard fd >= 0 else { throw NSError(domain: NSPOSIXErrorDomain, code: Int(errno)) } + nextChunkID += 1 + let handle = ChunkHandle(id: id, fd: fd) + tail = handle + tailOffset = 0 + + lock.lock() + guard !closed else { + lock.unlock() + throw Failure.closed + } + chunks.append(ChunkRecord(id: id, bytes: 0)) + // Readers of the newest packets share the writer's descriptor. + insertReadHandleLocked(handle) + let doomed = releaseUnreferencedChunksLocked(includingTail: false) + lock.unlock() + Self.unlinkAll(doomed) + } + + /// A write that failed because the volume or the quota has no room, which freeing a chunk can cure. + static func isOutOfSpace(_ error: Error) -> Bool { + let nsError = error as NSError + return nsError.domain == NSPOSIXErrorDomain + && (nsError.code == Int(ENOSPC) || nsError.code == Int(EDQUOT)) + } + + /// `writeLock` held. A failure that eviction cannot cure drops only its packet, so a steady one + /// must not write a line per packet. + private func noteWriteFailureOnce(_ error: Error) { + guard !loggedOtherWriteFailure, error as? Failure != .closed else { return } + loggedOtherWriteFailure = true + EngineLog.emit("[PacketRingBuffer] write failed (\(error)); packet dropped, history kept " + + "(logged once)", category: .swPlayback) + } + + /// `writeLock` held. Frees at least one chunk file while keeping the retained span on a keyframe: + /// up to the first keyframe in a later chunk, or, when the incoming packet is itself a video + /// keyframe that can open a fresh span, everything including the tail chunk. False when nothing + /// could be freed, and the packet is dropped as before. + private func makeRoomAfterFailedWrite(incomingKeyframe: Bool) -> Bool { + lock.lock() + var doomed: [URL] = [] + var releasedTail = false + if let pivot = keyframeInLaterChunkLocked() { + dropEntriesLocked(upTo: pivot) + doomed = releaseUnreferencedChunksLocked(includingTail: false) + } else if incomingKeyframe { + dropEntriesLocked(upTo: entries.count) + doomed = releaseUnreferencedChunksLocked(includingTail: true) + releasedTail = chunks.isEmpty + } + compactIfDueLocked() + lock.unlock() + if releasedTail { + tail = nil + tailOffset = 0 + } + Self.unlinkAll(doomed) + if !doomed.isEmpty { + EngineLog.emit("[PacketRingBuffer] write failed; freed \(doomed.count) chunk(s) and retrying " + + "(the rewind window shrinks instead of the live feed stopping)", + category: .swPlayback) + } + return !doomed.isEmpty + } + + /// Drop leading entries outside `edge - windowSeconds` or past the byte budget, keyframe-aligned, + /// and return the chunk files no retained entry references any more (the caller unlinks them + /// after releasing the lock). Walks the keyframe list, one step per GOP. private func evictLocked() -> [URL] { let cutoff = edge - windowSeconds var pivot: Int? = nil - var i = 0 - while i < entries.count, entries[i].pts <= cutoff { - if entries[i].isKeyframe { pivot = i } + var k = keyframeHead + while k < keyframeSeqs.count, let idx = indexLocked(keyframeSeqs[k]), entries[idx].pts <= cutoff { + pivot = idx + k += 1 + } + if let p = pivot, p > head { dropEntriesLocked(upTo: p) } + var doomed = releaseUnreferencedChunksLocked(includingTail: false) + + // Audit VPERF-101: time alone put 5.4 GB on disk at 8 Mbit/s over the 90 min default, and an + // infinite window never evicted at all. + while retainedDiskBytes > byteBudget, chunks.count > 1 { + if let p = keyframeInLaterChunkLocked() { + dropEntriesLocked(upTo: p) + } else if retainedDiskBytes - byteBudget > byteBudget, let p = firstIndexInLaterChunkLocked() { + // A GOP longer than the whole budget (or a stream that never flags a keyframe) must + // not grow the directory without bound; the rewind floor then decodes from the next + // keyframe, which `firstKeyframeSeq` points at. + dropEntriesLocked(upTo: p) + } else { + break + } + let freed = releaseUnreferencedChunksLocked(includingTail: false) + if freed.isEmpty { break } + doomed += freed + } + compactIfDueLocked() + return doomed + } + + private func dropEntriesLocked(upTo idx: Int) { + guard idx > head else { return } + firstSeq += idx - head + head = idx + while keyframeHead < keyframeSeqs.count, keyframeSeqs[keyframeHead] < firstSeq { keyframeHead += 1 } + } + + /// The first retained keyframe stored in a later chunk than the oldest retained entry. + private func keyframeInLaterChunkLocked() -> Int? { + guard head < entries.count else { return nil } + let headChunk = entries[head].chunk + var k = keyframeHead + while k < keyframeSeqs.count { + if let idx = indexLocked(keyframeSeqs[k]), entries[idx].chunk > headChunk { return idx } + k += 1 + } + return nil + } + + private func firstIndexInLaterChunkLocked() -> Int? { + guard head < entries.count else { return nil } + let headChunk = entries[head].chunk + var i = head + 1 + while i < entries.count { + if entries[i].chunk > headChunk { return i } i += 1 } - guard let p = pivot, p > 0 else { return [] } - let urls = entries[.. [URL] { + let floorChunk: UInt32? = head < entries.count ? entries[head].chunk : nil + var doomed: [URL] = [] + while let first = chunks.first, + chunks.count > 1 || includingTail, + floorChunk.map({ first.id < $0 }) ?? true { + chunks.removeFirst() + retainedDiskBytes -= first.bytes + readHandles.removeAll { $0.id == first.id } + doomed.append(chunkURL(first.id)) + } + return doomed + } + + /// Batched, so the dead prefix costs one memmove per few minutes of packets rather than one per GOP. + private func compactIfDueLocked() { + if head >= 1024, head * 8 >= entries.count - head { + entries.removeSubrange(0..= 256, keyframeHead * 8 >= keyframeSeqs.count - keyframeHead { + keyframeSeqs.removeSubrange(0.. ChunkHandle? { + guard let i = readHandles.firstIndex(where: { $0.id == chunk }) else { return nil } + let handle = readHandles[i] + if i > 0 { + readHandles.remove(at: i) + readHandles.insert(handle, at: 0) + } + return handle + } + + private func insertReadHandleLocked(_ handle: ChunkHandle) { + readHandles.removeAll { $0.id == handle.id } + readHandles.insert(handle, at: 0) + if readHandles.count > Self.readHandleCap { readHandles.removeLast() } + } + + private func openReadHandle(_ chunk: UInt32) -> ChunkHandle? { + let fd = open(chunkURL(chunk).path, O_RDONLY | O_CLOEXEC) + guard fd >= 0 else { return nil } + let handle = ChunkHandle(id: chunk, fd: fd) + lock.lock() + if !closed, let first = chunks.first, let last = chunks.last, first.id <= chunk, chunk <= last.id { + insertReadHandleLocked(handle) + } + lock.unlock() + return handle + } + + private static func read(_ fd: Int32, offset: UInt32, length: UInt32) -> Data? { + let count = Int(length) + if count == 0 { return Data() } + guard let buffer = malloc(count) else { return nil } + var done = 0 + while done < count { + let n = pread(fd, buffer + done, count - done, off_t(offset) + off_t(done)) + if n > 0 { + done += n + } else if n < 0, errno == EINTR { + continue + } else { + free(buffer) + return nil + } + } + return Data(bytesNoCopy: buffer, count: count, deallocator: .free) + } + + static let pwriteAll: WriteAll = { fd, bytes, offset in + guard var base = bytes.baseAddress else { return } + var remaining = bytes.count + var at = offset + while remaining > 0 { + let n = pwrite(fd, base, remaining, at) + if n > 0 { + base += n + remaining -= n + at += off_t(n) + } else if n < 0, errno == EINTR { + continue + } else { + throw NSError(domain: NSPOSIXErrorDomain, code: Int(n < 0 ? errno : EIO)) + } + } + } + + private static func unlinkAll(_ urls: [URL]) { + for url in urls { unlink(url.path) } } } diff --git a/Sources/AetherEngine/Video/RetentionClaims.swift b/Sources/AetherEngine/Video/RetentionClaims.swift new file mode 100644 index 000000000..d42a7b3f6 --- /dev/null +++ b/Sources/AetherEngine/Video/RetentionClaims.swift @@ -0,0 +1,68 @@ +import Foundation + +/// What the running sessions may still write to the tmp volume (#687). +/// +/// A session sizes its retention from the volume's free space, measured once at its start. A claim +/// another session has made but not yet written is still free at that moment, so several sessions +/// started together each took a quarter of the same space: four engines (a multiview host) claimed +/// four quarters, i.e. the volume the clamp exists to protect. +/// +/// So every session claims here, and a new one is sized from what is really left: the free space +/// minus what the others have claimed and not yet written. A session alone gets what it always got; +/// n sessions together leave at least `(3/4)^n` of the volume. +final class RetentionClaims: @unchecked Sendable { + + static let shared = RetentionClaims() + + /// One session's allowance. Released by `release()` or when dropped. + final class Claim: @unchecked Sendable { + let bytes: Int + /// What the other sessions could still write when this claim was sized; for the log. + let heldBackBytes: Int + private let id = UUID() + private let ledger: RetentionClaims + + fileprivate init(bytes: Int, heldBackBytes: Int, ledger: RetentionClaims) { + self.bytes = bytes + self.heldBackBytes = heldBackBytes + self.ledger = ledger + ledger.entries[id] = Entry(bytes: bytes, used: nil) + } + + /// Reports what the session has written so far. Until it is set the whole claim counts as + /// unwritten, which only ever sizes a later session smaller. + func track(_ used: @escaping @Sendable () -> Int) { + ledger.lock.lock() + defer { ledger.lock.unlock() } + ledger.entries[id]?.used = used + } + + func release() { + ledger.lock.lock() + defer { ledger.lock.unlock() } + ledger.entries.removeValue(forKey: id) + } + + deinit { release() } + } + + private struct Entry { + let bytes: Int + var used: (@Sendable () -> Int)? + } + + private let lock = NSLock() + private var entries: [UUID: Entry] = [:] + + /// Sizes a session with `sizing` over the free space the other claims leave, and records the + /// result. Unknown capacity is passed through as unknown. + /// + /// The `used` closures of the other claims run under this ledger's lock; they must not claim. + func claim(volumeAvailableBytes: Int64?, sizing: (Int64?) -> Int) -> Claim { + lock.lock() + defer { lock.unlock() } + let heldBack = entries.values.reduce(0) { $0 + max(0, $1.bytes - ($1.used?() ?? 0)) } + let left = volumeAvailableBytes.map { max(0, $0 - Int64(heldBack)) } + return Claim(bytes: max(0, sizing(left)), heldBackBytes: heldBack, ledger: self) + } +} diff --git a/Sources/AetherEngine/Video/SegmentCache.swift b/Sources/AetherEngine/Video/SegmentCache.swift index 95c202d1c..4fcc1388c 100644 --- a/Sources/AetherEngine/Video/SegmentCache.swift +++ b/Sources/AetherEngine/Video/SegmentCache.swift @@ -2,7 +2,7 @@ import Darwin import Foundation /// Sliding-window disk-backed cache for HLS-fMP4 segments. Bytes go to -/// /aether-segments//seg-N.m4s; only URLs stay in RAM. +/// /aether-segments//seg-N-G.m4s; only URLs stay in RAM. /// Reads use .alwaysMapped (kernel pages in/out under memory pressure). Window: /// [currentTargetIndex - backwardWindow, currentTargetIndex + forwardWindow]. /// The producer pauses via awaitFetchHighWater once forwardWindow ahead of target. @@ -68,26 +68,28 @@ final class SegmentCache: @unchecked Sendable { let sessionDir: URL - /// AE#451: an flock(2) held on `sessionDir/session.lock` for as long as this cache lives. - /// This is the liveness half of the stale-session sweep: the age check alone cannot tell a - /// session an hour into a film from one that crashed an hour ago, because the directory's - /// creation date IS the session's start time. - /// - /// flock and not fcntl: flock locks belong to the open file description, so a second cache in - /// the SAME process fails to take it too, which is the case the report is about. fcntl locks - /// are per-process and a process never blocks itself. - /// - /// The kernel drops it when the process dies, so a crashed session leaves an unheld marker and - /// sweeps exactly as before. -1 means unheld (open or flock failed); such a session is swept - /// like today's, which is the pre-AE#451 behaviour rather than a new failure. + /// AE#451: the `SessionDirectoryLiveness` marker held for as long as this cache lives. -1 means + /// unheld (open or flock failed); such a session is swept by age like before AE#451. private var lockFD: Int32 = -1 - private static let liveMarkerName = "session.lock" private var _totalBytes: Int = 0 /// Monotonic across prunes; NOT decremented by pruneOutsideWindow. Lets VideoSegmentProvider /// detect gaps below the producer's write head after eviction erases them from indexRange(). private var _highestStoredIndex: Int = -1 + + /// Audit SEG-4: every stored generation of an index gets its own file name, so a URL names exactly + /// one set of bytes. A reader dropping a vanished entry, or a prune deleting a doomed one after + /// unlocking, can then never hit a newer adoption of the same index. + private var fileGeneration: UInt64 = 0 + + private func nextSegmentFileURL(index: Int) -> URL { + condition.lock() + fileGeneration += 1 + let generation = fileGeneration + condition.unlock() + return sessionDir.appendingPathComponent("seg-\(index)-\(generation).m4s") + } /// Plan index -> how many pumps passed it without opening a segment (#358). Survives producer /// restarts on purpose: the repeat across a restart is the signal. private var foldCounts: [Int: Int] = [:] @@ -136,20 +138,7 @@ final class SegmentCache: @unchecked Sendable { } private static func acquireLiveMarker(sessionDir: URL) -> Int32 { - let path = sessionDir.appendingPathComponent(liveMarkerName).path - let fd = open(path, O_CREAT | O_RDWR, 0o600) - guard fd >= 0 else { - EngineLog.emit("[SegmentCache] live marker open failed at \(path): errno=\(errno)", - category: .session) - return -1 - } - guard flock(fd, LOCK_EX | LOCK_NB) == 0 else { - EngineLog.emit("[SegmentCache] live marker lock failed at \(path): errno=\(errno)", - category: .session) - Darwin.close(fd) - return -1 - } - return fd + SessionDirectoryLiveness.acquire(sessionDir: sessionDir, logPrefix: "[SegmentCache]") } private func releaseLiveMarker() { @@ -160,20 +149,6 @@ final class SegmentCache: @unchecked Sendable { if fd >= 0 { Darwin.close(fd) } } - /// Whether some open file description still holds `entry`'s marker. A missing marker answers - /// false: it is a directory from a build without one, and the age check decides it as before. - private static func isSessionDirLive(_ entry: URL) -> Bool { - let path = entry.appendingPathComponent(liveMarkerName).path - let fd = open(path, O_RDONLY) - guard fd >= 0 else { return false } - defer { Darwin.close(fd) } - if flock(fd, LOCK_EX | LOCK_NB) == 0 { - flock(fd, LOCK_UN) - return false - } - return true - } - private static func sweepStaleSessionDirs(baseDir: URL, currentSession: String) { let fm = FileManager.default guard let entries = try? fm.contentsOfDirectory(at: baseDir, @@ -186,7 +161,7 @@ final class SegmentCache: @unchecked Sendable { let created = (try? entry.resourceValues(forKeys: [.creationDateKey]))?.creationDate guard created == nil || created! < cutoff else { continue } // AE#451: age says how long it has been there, not whether anyone is still using it. - if isSessionDirLive(entry) { + if SessionDirectoryLiveness.isLive(entry) { EngineLog.emit("[SegmentCache] sweep spared live session dir \(entry.lastPathComponent)", category: .session) continue @@ -264,7 +239,7 @@ final class SegmentCache: @unchecked Sendable { } func store(index: Int, data: Data) { - let fileURL = sessionDir.appendingPathComponent("seg-\(index).m4s") + let fileURL = nextSegmentFileURL(index: index) var writeOK: Bool do { try data.write(to: fileURL, options: [.atomic]) @@ -289,11 +264,14 @@ final class SegmentCache: @unchecked Sendable { // A re-store of a resident index changes bytes, not residency; only an insertion or an // eviction moves the set, and both are already known here without walking it. var residentSetChanged = false + var supersededFile: URL? if writeOK { if let oldBytes = entryBytes[index] { _totalBytes -= oldBytes } - residentSetChanged = entries.updateValue(fileURL, forKey: index) == nil + let superseded = entries.updateValue(fileURL, forKey: index) + residentSetChanged = superseded == nil + if let superseded { supersededFile = superseded } entryBytes[index] = data.count _totalBytes += data.count if index > _highestStoredIndex { _highestStoredIndex = index } @@ -302,6 +280,7 @@ final class SegmentCache: @unchecked Sendable { if !doomed.isEmpty { residentSetChanged = true } condition.broadcast() condition.unlock() + if let supersededFile { try? FileManager.default.removeItem(at: supersededFile) } for url in doomed { try? FileManager.default.removeItem(at: url) } if residentSetChanged { onResidentSetChanged?() } } @@ -311,12 +290,9 @@ final class SegmentCache: @unchecked Sendable { /// `videoReach` (AE#412) is what this segment's video offers a cold arrival; nil leaves the /// previous claim in place only if the index is re-adopted without one, which no caller does. func adopt(index: Int, stagingPath: URL, byteCount: Int, videoReach: VideoReach? = nil) { - let fileURL = sessionDir.appendingPathComponent("seg-\(index).m4s") + let fileURL = nextSegmentFileURL(index: index) var renameOK: Bool do { - if FileManager.default.fileExists(atPath: fileURL.path) { - try FileManager.default.removeItem(at: fileURL) - } try FileManager.default.moveItem(at: stagingPath, to: fileURL) renameOK = true } catch { @@ -338,11 +314,14 @@ final class SegmentCache: @unchecked Sendable { return } var residentSetChanged = false + var supersededFile: URL? if renameOK { if let oldBytes = entryBytes[index] { _totalBytes -= oldBytes } - residentSetChanged = entries.updateValue(fileURL, forKey: index) == nil + let superseded = entries.updateValue(fileURL, forKey: index) + residentSetChanged = superseded == nil + if let superseded { supersededFile = superseded } entryBytes[index] = byteCount _totalBytes += byteCount if index > _highestStoredIndex { _highestStoredIndex = index } @@ -357,6 +336,7 @@ final class SegmentCache: @unchecked Sendable { if !doomed.isEmpty { residentSetChanged = true } condition.broadcast() condition.unlock() + if let supersededFile { try? FileManager.default.removeItem(at: supersededFile) } for url in doomed { try? FileManager.default.removeItem(at: url) } if residentSetChanged { onResidentSetChanged?() } } @@ -457,6 +437,12 @@ final class SegmentCache: @unchecked Sendable { category: .session) } + var isClosed: Bool { + condition.lock() + defer { condition.unlock() } + return closed + } + func fetch(index: Int, timeout: TimeInterval = 15.0) -> Data? { condition.lock() if let url = entries[index] { diff --git a/Sources/AetherEngine/Video/SessionDirectoryLiveness.swift b/Sources/AetherEngine/Video/SessionDirectoryLiveness.swift new file mode 100644 index 000000000..3cf9e7e96 --- /dev/null +++ b/Sources/AetherEngine/Video/SessionDirectoryLiveness.swift @@ -0,0 +1,50 @@ +import Darwin +import Foundation + +/// AE#451: the liveness half of the `aether-segments/` stale-session sweep. A session directory's +/// creation date is its session's START time, so age alone cannot tell a session an hour in from one +/// that crashed an hour ago; an flock(2) held on `/session.lock` for the owner's lifetime can. +/// +/// flock and not fcntl: flock locks belong to the open file description, so a second owner in the +/// SAME process fails to take it too. fcntl locks are per-process and a process never blocks itself. +/// The kernel drops the lock when the process dies, so a crashed session leaves an unheld marker and +/// sweeps as before. +/// +/// Audit SEG-105: shared by `SegmentCache` and the software DVR ring, which lives under the same base +/// directory and was swept an hour into a DVR session because its directory carried no marker. +enum SessionDirectoryLiveness { + static let markerName = "session.lock" + + /// The held marker's descriptor, or -1 when open or flock failed (such a directory is swept by + /// age, the pre-AE#451 behaviour rather than a new failure). + static func acquire(sessionDir: URL, logPrefix: String) -> Int32 { + let path = sessionDir.appendingPathComponent(markerName).path + let fd = open(path, O_CREAT | O_RDWR | O_CLOEXEC, 0o600) + guard fd >= 0 else { + EngineLog.emit("\(logPrefix) live marker open failed at \(path): errno=\(errno)", + category: .session) + return -1 + } + guard flock(fd, LOCK_EX | LOCK_NB) == 0 else { + EngineLog.emit("\(logPrefix) live marker lock failed at \(path): errno=\(errno)", + category: .session) + Darwin.close(fd) + return -1 + } + return fd + } + + /// Whether some open file description still holds `entry`'s marker. A missing marker answers + /// false: it is a directory from a build without one, and the age check decides it as before. + static func isLive(_ entry: URL) -> Bool { + let path = entry.appendingPathComponent(markerName).path + let fd = open(path, O_RDONLY | O_CLOEXEC) + guard fd >= 0 else { return false } + defer { Darwin.close(fd) } + if flock(fd, LOCK_EX | LOCK_NB) == 0 { + flock(fd, LOCK_UN) + return false + } + return true + } +} diff --git a/Sources/AetherEngine/Video/SourceTimestampBounds.swift b/Sources/AetherEngine/Video/SourceTimestampBounds.swift new file mode 100644 index 000000000..90e2c1a29 --- /dev/null +++ b/Sources/AetherEngine/Video/SourceTimestampBounds.swift @@ -0,0 +1,115 @@ +import Foundation +import CoreMedia +import AetherLibavcodec + +/// Audit SEG-1: demuxed timestamps reach the pump unchecked (matroskadec stores a uint64 cluster +/// time into an int64 pts, a live fMP4 tfdt is whatever the origin wrote), and Swift traps on +/// Int64 overflow. Values outside `plausibleMagnitude` become AV_NOPTS_VALUE at the source funnel, +/// which the pump's NOPTS repair already handles, and the tick arithmetic downstream saturates +/// instead of trapping. +enum SourceTimestampBounds { + + /// 2^60 still holds an epoch-anchored tfdt at a 10 MHz timescale by three orders of magnitude, + /// and leaves room for the pump's three-term shift chains below 2^63. + static let plausibleMagnitude: Int64 = 1 << 60 + + static func plausible(_ ticks: Int64) -> Int64 { + guard ticks != Int64.min else { return ticks } + return ticks > -plausibleMagnitude && ticks < plausibleMagnitude ? ticks : Int64.min + } + + /// Audit NAT-101 / DEC-101 / FEA-101 / SUB-105: the demuxer-wide bound, applied to every packet + /// the engine reads. 2^62 keeps any difference of two bounded values inside Int64 and holds an + /// epoch-anchored nanosecond timestamp (1.79e18 in 2026) until about 2116. + static let demuxedMagnitude: Int64 = 1 << 62 + + /// Epoch seconds pass until about 2096. `ticks * num == seconds * den` and + /// `4e9 * (2^31 - 1) < 2^63`, so a tick inside this bound times any AVRational numerator stays + /// inside Int64, and a bounded time scaled to milliseconds or centiseconds stays inside Int. + static let maxPlausibleSeconds: Double = 4e9 + + /// AV_NOPTS_VALUE for a tick at or past `demuxedMagnitude`, or one whose time on `timeBase` is at + /// or past `maxPlausibleSeconds`. A degenerate time base leaves only the tick rule. + static func plausible(_ ticks: Int64, timeBase: AVRational) -> Int64 { + guard ticks != Int64.min else { return ticks } + guard ticks > -demuxedMagnitude, ticks < demuxedMagnitude else { return Int64.min } + guard timeBase.num > 0, timeBase.den > 0 else { return ticks } + let seconds = Double(ticks) * Double(timeBase.num) / Double(timeBase.den) + return abs(seconds) < maxPlausibleSeconds ? ticks : Int64.min + } + + /// The demuxer's funnel: implausible pts / dts become AV_NOPTS_VALUE, an implausible or negative + /// duration becomes 0. Idempotent, so a second pass over repaired output is free. Returns whether + /// any field was replaced. + @discardableResult + static func sanitize(_ packet: UnsafeMutablePointer, timeBase: AVRational) -> Bool { + let pts = plausible(packet.pointee.pts, timeBase: timeBase) + let dts = plausible(packet.pointee.dts, timeBase: timeBase) + let duration = packet.pointee.duration + let durationOK = duration >= 0 && plausible(duration, timeBase: timeBase) == duration + guard pts != packet.pointee.pts || dts != packet.pointee.dts || !durationOK else { return false } + packet.pointee.pts = pts + packet.pointee.dts = dts + if !durationOK { packet.pointee.duration = 0 } + return true + } + + /// Returns whether any field was out of range and got replaced. + @discardableResult + static func sanitize(_ packet: UnsafeMutablePointer) -> Bool { + let pts = plausible(packet.pointee.pts) + let dts = plausible(packet.pointee.dts) + let duration = packet.pointee.duration + let durationOK = duration >= 0 && duration < plausibleMagnitude + guard pts != packet.pointee.pts || dts != packet.pointee.dts || !durationOK else { return false } + packet.pointee.pts = pts + packet.pointee.dts = dts + if !durationOK { packet.pointee.duration = 0 } + return true + } + + /// Saturating tick arithmetic. The floor is `Int64.min + 1` so a result never reads as AV_NOPTS_VALUE. + static func difference(_ lhs: Int64, _ rhs: Int64) -> Int64 { + let (value, overflow) = lhs.subtractingReportingOverflow(rhs) + guard overflow else { return max(value, Int64.min + 1) } + return lhs < rhs ? Int64.min + 1 : Int64.max + } + + static func sum(_ lhs: Int64, _ rhs: Int64) -> Int64 { + let (value, overflow) = lhs.addingReportingOverflow(rhs) + guard overflow else { return max(value, Int64.min + 1) } + return lhs < 0 ? Int64.min + 1 : Int64.max + } + + /// Audit DEC-101: `ticks * num` as a CMTime, `.invalid` for AV_NOPTS_VALUE, a degenerate time + /// base or a product outside Int64. Decoder-computed frame timestamps and the software host's + /// folded ones never pass the demuxer's bound, so the conversion itself has to be total. + static func cmTime(ticks: Int64, timeBase: AVRational) -> CMTime { + guard ticks != Int64.min, timeBase.den > 0 else { return .invalid } + let (value, overflow) = ticks.multipliedReportingOverflow(by: Int64(timeBase.num)) + guard !overflow else { return .invalid } + return CMTimeMake(value: value, timescale: timeBase.den) + } + + /// A tick count computed in Double, rounded to the nearest tick. nil when it is not finite or + /// falls outside Int64, Int64.min included (it reads as AV_NOPTS_VALUE). + static func roundedTicks(_ value: Double) -> Int64? { + guard let ticks = Int64(exactly: value.rounded()), ticks != Int64.min else { return nil } + return ticks + } + + /// Audit NAT-101: moves a timestamp back by `offset` ticks. AV_NOPTS_VALUE when the timestamp or + /// the offset is unset, or when the result leaves Int64. + static func shifted(_ ticks: Int64, back offset: Int64?) -> Int64 { + guard ticks != Int64.min, let offset else { return Int64.min } + let (value, overflow) = ticks.subtractingReportingOverflow(offset) + return overflow ? Int64.min : value + } + + /// Audit SUB-105: a cue or display time about to be scaled into an integer. NaN reads as 0 and + /// anything past `maxPlausibleSeconds` pins to it. + static func clampedSeconds(_ seconds: Double) -> Double { + guard !seconds.isNaN else { return 0 } + return min(max(seconds, -maxPlausibleSeconds), maxPlausibleSeconds) + } +} diff --git a/Sources/AetherEngine/Video/VTCapabilityProbe.swift b/Sources/AetherEngine/Video/VTCapabilityProbe.swift index c2bf17ce5..5187c9c1f 100644 --- a/Sources/AetherEngine/Video/VTCapabilityProbe.swift +++ b/Sources/AetherEngine/Video/VTCapabilityProbe.swift @@ -53,7 +53,19 @@ enum VTCapabilityProbe { /// wrongly forces the software path. Not the question `SoftwarePlaybackHost` asks, see /// `HardwareDecodeVerdict`. static func canHardwareDecode(codecpar: UnsafePointer) -> Bool { - hardwareDecodeVerdict(codecpar: codecpar).keepsNativeRoute + if codecpar.pointee.codec_id == AV_CODEC_ID_AV1 { + let av1C = codecpar.pointee.extradata.map { + Array(UnsafeBufferPointer(start: $0, count: Int(max(0, codecpar.pointee.extradata_size)))) + } + let fits = VideoRoutingPolicy.av1FitsHardwareDecoder( + av1C: av1C, codecparProfile: codecpar.pointee.profile) + EngineLog.emit( + "[VTProbe] canHardwareDecode codec=av01 profile=\(codecpar.pointee.profile) -> \(fits)", + category: .engine + ) + return fits + } + return hardwareDecodeVerdict(codecpar: codecpar).keepsNativeRoute } /// The throwaway session is invalidated immediately; the whole probe costs well under a millisecond and diff --git a/Sources/AetherEngine/Video/VideoConfigRecord.swift b/Sources/AetherEngine/Video/VideoConfigRecord.swift index 4e4d415ce..6df1b4ff5 100644 --- a/Sources/AetherEngine/Video/VideoConfigRecord.swift +++ b/Sources/AetherEngine/Video/VideoConfigRecord.swift @@ -192,7 +192,7 @@ enum VideoConfigRecord { /// Zero bytes immediately before a start code belong to the start code (a 4-byte code is a 3-byte /// one with a leading `trailing_zero_8bits`), so they are trimmed off the preceding NAL. A /// parameter set never ends in `0x00`: its last byte carries the rbsp_stop_one_bit. - private static func splitAnnexBNALs(_ bytes: [UInt8]) -> [[UInt8]] { + static func splitAnnexBNALs(_ bytes: [UInt8]) -> [[UInt8]] { var starts: [Int] = [] var i = 0 while i + 3 <= bytes.count { diff --git a/Sources/AetherEngine/Video/VideoSegmentProvider.swift b/Sources/AetherEngine/Video/VideoSegmentProvider.swift index 134d63dad..886b7a8b0 100644 --- a/Sources/AetherEngine/Video/VideoSegmentProvider.swift +++ b/Sources/AetherEngine/Video/VideoSegmentProvider.swift @@ -16,7 +16,7 @@ struct NativeSubtitleRenditionInfo: Sendable, Equatable { /// Single source of truth for sliding live window size. Playlist firstVisible and cache evictBelow /// both read this so they can never drift (drift = playlist lists a segment the cache deleted, or vice versa). -/// effectiveWindowSeconds = dvrWindowSeconds ?? liveOnlyFloorSeconds; +/// effectiveWindowSeconds = dvrWindowSeconds clamped to [0, 1 day] ?? liveOnlyFloorSeconds; /// windowSegmentCount = max(minSafeSegments, ceil(effective / targetSegmentDurationSeconds)). struct LiveWindowSizing { /// Live-only floor: 60 s so disk and playlist stay finite even without DVR seek. @@ -58,11 +58,19 @@ struct LiveWindowSizing { /// has to: comparing the served count against a value that already carries the clamps would report /// every clamped window as unclamped. func requestedSegmentCount(observedSegmentDurationSeconds: Double?) -> Int { - let effective = dvrWindowSeconds ?? Self.liveOnlyFloorSeconds + let effective = Self.effectiveWindowSeconds(dvrWindowSeconds) let divisor = max(max(0.5, targetSegmentDurationSeconds), observedSegmentDurationSeconds ?? 0) return Int(ceil(effective / divisor)) } + /// Audit SEG-107: the window is a host value, and `.infinity` ("keep everything"), NaN or anything + /// past `Int` trapped the conversion above. A day is far deeper than the playlist ceiling ever + /// serves, and keeps the log's `requested x cadence` product in range. + static func effectiveWindowSeconds(_ dvrWindowSeconds: Double?) -> Double { + guard let window = dvrWindowSeconds, !window.isNaN else { return liveOnlyFloorSeconds } + return min(max(window, 0), Double(LiveEdgePolicy.maxCoveredWholeSeconds)) + } + /// AE#443: how many segments of the observed size the session's disk allowance holds. /// /// The window a host asks for is a promise in SECONDS and the disk is a fact in BYTES, and until @@ -104,6 +112,13 @@ enum LiveEdgePolicy { static let boundedStartFloorArmed = ProcessInfo.processInfo.environment["AETHER_BOUNDED_START_FLOOR"] == "1" + /// AE#686: #684's first-serve latch covers sources the engine cuts itself too, so AVPlayer's second + /// plain manifest request no longer waits out a second grace. On by default since the Apple TV A/B + /// (first picture 1.003 to 1.039 s sooner on every bounded start, no stall in 18 launches); + /// `AETHER_FIRST_SERVE_LATCH_ALL=0` restores the old gate for a comparison run. Read once. + static let firstServeLatchAllArmed = + ProcessInfo.processInfo.environment["AETHER_FIRST_SERVE_LATCH_ALL"] != "0" + /// AVPlayer's unchanged-playlist patience: it tolerates a playlist that has not changed for this /// multiple of the served TARGETDURATION before drawing `-12888`. The one number the cadence floor /// is answerable to. @@ -267,6 +282,66 @@ enum LiveEdgePolicy { wholeSecondsCovering(cadenceSeconds / unchangedPlaylistPatienceMultiplier) } + /// AE#684: the TARGETDURATION an ingested upstream's segment duration requires, which is the whole + /// of it and not `targetDurationForCadence`'s two thirds. + /// + /// The division by 1.5 is right for a measured gap because the meter hands over a robust maximum. + /// A segment duration is the other end of the distribution: the period the upstream delivers at + /// when nothing at all is late, since it cannot publish a segment before it has finished one. Every + /// real delivery is that period plus the upstream's jitter plus this reader's own reload interval + /// (half a segment, the RFC's), so a patience of exactly one period is under all of them. It only + /// looked covered while the served segments were the upstream's own, where `ceil(max EXTINF)` puts + /// the whole segment into the value anyway. `.fastZap` re-cuts each upstream segment at its GOPs, + /// that term then reads the GOP, and a 6 s upstream sealed at `ceil(6 / 1.5)` = 4: a 6.0 s patience + /// for deliveries measured in the field at 6.17 to 6.83 s, -12888 on the first one that also missed + /// a poll, a skipped reload after it and a stall with the content already listed. + /// + /// Re-cutting changes how finely the window is listed, not how often it changes, so the playlist + /// promises what a playlist of the upstream's segments would: at least their longest duration. + /// + /// Bounded by `upstreamSegmentSealCeilingSeconds`: above it the term stops growing and + /// `targetDurationForCadence` (two thirds of the segment) is the larger one again. + static func targetDurationForUpstreamSegment(_ segmentSeconds: Double) -> Int { + guard !segmentSeconds.isNaN else { return 0 } + return wholeSecondsCovering(Swift.min(segmentSeconds, upstreamSegmentSealCeilingSeconds)) + } + + /// AE#684: the longest upstream segment the seal takes whole. + /// + /// A holdback is `3 x` the value, and the live-only window is 60 s + /// (`LiveWindowSizing.liveOnlyFloorSeconds`): a 20 s segment taken whole would ask the client to + /// sit a whole window behind an edge that window can barely list. At 10, segments up to 10 s are + /// asked for whole and a 20 s one seals 14, as it did before the term existed. What is asked for + /// is then paid only as far as the join reaches (`targetDurationTheJoinCanPay`), and the join is + /// not deepened for it, so in practice a 10 s provider lands at 9. (The join's one addition, + /// `HLSPlaylistTracker.phaseEqualisedJoinDepth`, only lifts a tune to the depth another phase + /// of the same listing already takes.) + static let upstreamSegmentSealCeilingSeconds: Double = 10 + + /// AE#684: the upstream-segment seal, as far as the join can pay for it. + /// + /// A seal is a holdback of `3 x` itself, and a holdback the first window cannot hold is not + /// protection, it is a start the client spends waiting. The join is three upstream segments on + /// anything longer than about 5 s and stays that deep (one more would be one more download on + /// every zap, AE#678; the exception is an upstream whose segment lengths alternate, where a tune + /// is lifted to the depth its other phase already takes, 6 s / 4 s to four), and its last GOP + /// stays open until the next upstream delivery: three 6 s + /// segments are 18 s joined and 16 s cut, against the 18 s a seal of 6 asks for. So once the + /// reader has handed the whole join over and the cutter has nothing left to read, the seal is the + /// largest value that cut content covers, never under what it was without the upstream term and + /// never over the full one: 16 s pays for 5 (patience 7.5 s where the old rule gave 6.0 s), 28 s + /// of a 10 s provider for 9. + /// + /// Whether the join is spent is a FACT the reader states (`joinIsSpent`), not arithmetic on + /// EXTINF: a playlist that rounds its durations up (6.3 for 6 s of media) never adds up to what + /// was cut, and a tolerance on the sum decided every such start wrongly. + static func targetDurationTheJoinCanPay(full: Int, withoutUpstreamSegment base: Int, + finalizedSeconds: Double) -> Int { + guard finalizedSeconds.isFinite, finalizedSeconds > 0 else { return Swift.max(base, 0) } + let affordable = Int((servedSeconds(finalizedSeconds) / 3).rounded(.down)) + return Swift.max(base, Swift.min(full, affordable)) + } + /// A duration as the playlist actually serves it: `#EXTINF` is written with `%.3f`, so a millisecond /// is the finest distinction any client can ever read, and nothing below it may decide anything. /// @@ -286,10 +361,20 @@ enum LiveEdgePolicy { } /// Whole seconds of promise covering a measured duration, taken at the served resolution. + /// + /// Total: a duration from a hostile or broken source (infinite, NaN, beyond `Int`) must not trap the + /// playlist writer. An unbounded one saturates at `maxCoveredWholeSeconds`, which keeps `3 x TD` + /// arithmetic far from overflow; NaN and anything at or below zero cover nothing. static func wholeSecondsCovering(_ seconds: Double) -> Int { - Int(servedSeconds(seconds).rounded(.up)) + guard !seconds.isNaN else { return 0 } + let covered = servedSeconds(seconds).rounded(.up) + guard covered < Double(maxCoveredWholeSeconds) else { return maxCoveredWholeSeconds } + return covered > 0 ? Int(covered) : 0 } + /// One day. Far above any real segment or cadence, far below where `Int` arithmetic on it overflows. + static let maxCoveredWholeSeconds = 86_400 + /// AE#454: the served `EXT-X-START:TIME-OFFSET` for a rejoin, or nil when this playlist cannot /// carry the placement. /// @@ -320,20 +405,75 @@ enum LiveEdgePolicy { } /// Served `#EXT-X-TARGETDURATION`, in whole seconds: `>= ceil(max EXTINF)` (HLS requirement), floored - /// by `ceil(1.5 x cut target)` (widens AVPlayer's unchanged-playlist patience, anti -12888) and by - /// what the observed cadence needs to stay inside that patience. `cutTargetSeconds` / - /// `cadenceFloorSeconds` are nil for VOD/EVENT. Every term is taken at the resolution the playlist - /// serves (`servedSeconds`), so the value covers the EXTINFs the client is actually handed and no - /// sub-millisecond noise can buy a whole second of holdback. + /// by `ceil(1.5 x cut target)` (widens AVPlayer's unchanged-playlist patience, anti -12888), by + /// what the observed cadence needs to stay inside that patience, and by the longest segment an + /// ingested upstream delivers in (`targetDurationForUpstreamSegment`, AE#684). `cutTargetSeconds` / + /// `cadenceFloorSeconds` are nil for VOD/EVENT, `upstreamSegmentSeconds` for everything but an + /// ingest. Every term is taken at the resolution the playlist serves (`servedSeconds`), so the + /// value covers the EXTINFs the client is actually handed and no sub-millisecond noise can buy a + /// whole second of holdback. + /// + /// AE#670: `segmentsAreCutHere` adds `ceil(1.5 x max EXTINF)` where that headroom has no other source + /// (`gopHeadroomApplies`). static func targetDurationSeconds(maxSegmentDuration: Double, cutTargetSeconds: Double?, - cadenceFloorSeconds: Double?) -> Int { + cadenceFloorSeconds: Double?, + segmentsAreCutHere: Bool = false, + upstreamSegmentSeconds: Double? = nil) -> Int { var td = wholeSecondsCovering(max(1.0, maxSegmentDuration)) if let cut = cutTargetSeconds { td = max(td, wholeSecondsCovering(cut * 1.5)) } + if gopHeadroomApplies(cutTargetSeconds: cutTargetSeconds, segmentsAreCutHere: segmentsAreCutHere) { + td = max(td, targetDurationForGOPHeadroom(maxSegmentDuration)) + } if let floor = cadenceFloorSeconds { td = max(td, targetDurationForCadence(floor)) } + if let upstream = upstreamSegmentSeconds { td = max(td, targetDurationForUpstreamSegment(upstream)) } return td } + /// AE#670: does the sealed value need headroom over the GOPs seen so far? + /// + /// A cut target under a second never bounds a segment: the cutter waits for the next keyframe, so + /// every segment it cuts is one whole source GOP, and the value is sealed from the first few of them. + /// A broadcast's GOPs are not regular (reported: 1.0 to 2.4 s, sealed on three 1.000 s ones), and a + /// later longer one then breaks `EXTINF <= TD` and holds the playlist unchanged past AVPlayer's + /// patience. `.standard` has that headroom already in its `1.5 x cut target` floor, and an ingest + /// is bounded by the upstream segment its GOPs were cut from (`targetDurationForUpstreamSegment`; + /// AE#447 keeps TD 2 on 2.000 s ones), so only the engine's own sub-second cut needs it from the + /// segments. + static func gopHeadroomApplies(cutTargetSeconds: Double?, segmentsAreCutHere: Bool) -> Bool { + guard segmentsAreCutHere, let cut = cutTargetSeconds else { return false } + return cut < 1.0 + } + + /// AE#670: the smallest TARGETDURATION under which a GOP 1.5 x the longest seen would still be + /// served without fault: listed legally (RFC 8216 4.3.3.1 compares the EXTINF ROUNDED TO THE NEAREST + /// integer with TD) and finalized inside AVPlayer's `1.5 x TD` patience with + /// `gopHeadroomDeliveryMarginSeconds` to spare. + /// + /// Round 2: the first version took `ceil(1.5 x max EXTINF)`, which asks the virtual GOP for more than + /// any real segment is asked for and puts a cliff a millisecond wide into the seal. Reported on a + /// 59.94 fps source with 1.001 s GOPs: a 79-frame segment (1.318 s) sealed 2, an 80-frame one + /// (1.335 s, 2.002 x 1.5) sealed 3, a 9 s holdback a 6.6 s rebuild backlog could not hold, and the + /// first frame came 2.5 s later. Under TD 2 a GOP of 2.499 s still lists and leaves 0.5 s of patience, + /// so segments up to 1.666 s now seal 2, and the reporter's TD 2 sessions that later met 1.485 s ones + /// ran without a stall or -12888. Never above the first version: `round <= ceil`, and the patience + /// term only binds where the GOP is too short for the rounding to (TD 1 leaves a listed 1.5 x GOP no + /// patience at all, so 1.0 s GOPs keep their TD 2). + static func targetDurationForGOPHeadroom(_ maxSegmentSeconds: Double) -> Int { + guard maxSegmentSeconds.isFinite else { return wholeSecondsCovering(maxSegmentSeconds) } + let longerGOP = servedSeconds(maxSegmentSeconds * 1.5) + guard longerGOP > 0 else { return 0 } + let listed = wholeSecondsCovering(longerGOP.rounded(.toNearestOrAwayFromZero)) + let patient = wholeSecondsCovering((longerGOP + gopHeadroomDeliveryMarginSeconds) + / unchangedPlaylistPatienceMultiplier) + return max(listed, patient) + } + + /// AE#670 round 2: how long before AVPlayer's patience runs out a GOP 1.5 x the longest seen must + /// have been finalized. Half a second is what the first version already accepted for the 2.4 s GOP + /// it was reported for (TD 2, patience 3.0 s), so the relaxation gives up nothing that one kept. + static let gopHeadroomDeliveryMarginSeconds: Double = 0.5 + /// AVPlayer's default (and our explicitly advertised) live-edge holdback: `3 x TARGETDURATION`, the /// RFC 8216bis floor for `EXT-X-SERVER-CONTROL:HOLD-BACK`. static func holdBackSeconds(targetDuration: Int) -> Double { Double(3 * targetDuration) } @@ -359,10 +499,12 @@ enum LiveEdgePolicy { maxSegmentDuration: Double, cutTargetSeconds: Double?, cadenceFloorSeconds: Double?, + segmentsAreCutHere: Bool = false, windowSegmentCount: Int) -> Bool { let td = targetDurationSeconds(maxSegmentDuration: maxSegmentDuration, cutTargetSeconds: cutTargetSeconds, - cadenceFloorSeconds: cadenceFloorSeconds) + cadenceFloorSeconds: cadenceFloorSeconds, + segmentsAreCutHere: segmentsAreCutHere) return startupCushionSatisfied( segmentCount: segmentCount, summedDurationSeconds: summedDurationSeconds, @@ -500,19 +642,48 @@ enum CadenceFloorTerm { /// line why the session will hold this value for its whole life. Every term but `selfReported` feeds /// the max; `selfReported` is what the upstream CLAIMED, printed beside what was measured. struct LiveTargetDurationDerivation { - let value: Int + var value: Int let maxSegmentDuration: Double let cutTargetFloor: Double? + var gopHeadroomApplies = false let cadenceFloor: CadenceFloorTerm + /// AE#684: the longest segment an ingested upstream has served, nil off the ingest path. + var upstreamSegment: Double? = nil + /// AE#684 review: set when the join could not pay for the full upstream-segment seal: what it + /// held, what of that is cut, and the value the terms alone would have sealed. + var joinBound: (backlogSeconds: Double, finalizedSeconds: Double, full: Int)? = nil let selfReported: Double? + /// The upstream-segment term as both seal lines state it, nil off the ingest path. + var upstreamSegmentAccount: String? { + guard let upstreamSegment else { return nil } + var text = "upstream segment \(LiveEdgePolicy.seconds(upstreamSegment))s " + + "(one delivery, however finely it is cut here" + if upstreamSegment > LiveEdgePolicy.upstreamSegmentSealCeilingSeconds { + text += ", taken up to \(LiveEdgePolicy.seconds(LiveEdgePolicy.upstreamSegmentSealCeilingSeconds))s" + } + text += ")" + if let joinBound { + text += ", of which the join pays \(value)s of \(joinBound.full)s " + + "(\(LiveEdgePolicy.seconds(joinBound.finalizedSeconds))s cut of the " + + "\(LiveEdgePolicy.seconds(joinBound.backlogSeconds))s it listed)" + } + return text + } + /// One line, in the order the terms are maxed. Reads as an argument for the number it reports. var account: String { var terms = ["max EXTINF \(LiveEdgePolicy.seconds(maxSegmentDuration))s"] if let cutTargetFloor { terms.append("1.5 x cut target \(LiveEdgePolicy.seconds(cutTargetFloor * 1.5))s") } + if gopHeadroomApplies { + terms.append("1.5 x max EXTINF \(LiveEdgePolicy.seconds(maxSegmentDuration * 1.5))s " + + "needs \(LiveEdgePolicy.targetDurationForGOPHeadroom(maxSegmentDuration))s " + + "(each segment is one whole GOP)") + } terms.append(cadenceFloor.account) + if let upstreamSegmentAccount { terms.append(upstreamSegmentAccount) } let claim = selfReported.map { "; upstream advertises \(LiveEdgePolicy.seconds($0))s (reported, not used)" } ?? "" @@ -549,6 +720,8 @@ final class VideoSegmentProvider: HLSSegmentProvider, @unchecked Sendable { private let cache: SegmentCache /// Immutable for VOD; grows under stateLock for live (producer appends via appendLiveSegment). private var segments: [HLSVideoEngine.Segment] + private let iFrameSourceLock = NSLock() + private var _iFrameSource: IFrameSegmentSource? private let isLive: Bool /// Sequential-origin session: playlist grows with finalized real durations (see _seqDurations). private let sequentialAppendPlaylist: Bool @@ -559,9 +732,20 @@ final class VideoSegmentProvider: HLSSegmentProvider, @unchecked Sendable { /// AE#594 arm B: skip the bounded branch, so the wait ends at the full holdback cushion or at the /// outer wall-clock deadline. Measurement arm, off unless the environment asks for it. private let boundedStartFloorsAtHoldback: Bool + /// AE#686: the first-serve latch also covers a source the engine cuts itself. The engine passes + /// `LiveEdgePolicy.firstServeLatchAllArmed`, on unless the environment opts out. + private let firstServeLatchCoversEngineCut: Bool /// AE#374: whether the first-serve gate has already reported the interval it held. Read and written /// only under `firstSegmentCondition`, inside `waitForFirstLiveSegment` and its two account helpers. private var didAccountForFirstServe = false + /// AE#686: whether the first repeat pass through the gate has been reported. Same lock. + private var didAccountForRepeatServe = false + /// AE#684: on an ingest the gate is a FIRST-serve gate. Every `/media.m3u8` request without an + /// `_HLS_msn` re-enters it, and AVPlayer opens a session with two of them back to back, so a + /// bounded start (served under the holdback, after its grace) held the second request for a + /// whole second grace: measured 2.012 s to the first manifest and 2.02 s more before `init.mp4` + /// was asked for. Read for ingest sessions only (see the gate); same lock as the flag above. + private var firstManifestServed = false /// Host override for blocking-reload (`LoadOptions.liveBlockingReload`): nil = auto (observed policy for /// ingest, on by default for signal-less live), true/false = force. Wins over the policy (#167). private let blockingReloadOverride: Bool? @@ -781,6 +965,7 @@ final class VideoSegmentProvider: HLSSegmentProvider, @unchecked Sendable { liveWindowSizing: LiveWindowSizing = LiveWindowSizing(targetSegmentDurationSeconds: 4.0, dvrWindowSeconds: nil), allowsBoundedDegradedStart: Bool = false, boundedStartFloorsAtHoldback: Bool = false, + firstServeLatchCoversEngineCut: Bool = false, blockingReloadOverride: Bool? = nil, liveCadencePolicy: LiveCadencePolicy? = nil, restartHandler: ((Int) -> Void)? = nil, @@ -811,6 +996,7 @@ final class VideoSegmentProvider: HLSSegmentProvider, @unchecked Sendable { self.liveWindowSizing = liveWindowSizing self.allowsBoundedDegradedStart = allowsBoundedDegradedStart self.boundedStartFloorsAtHoldback = boundedStartFloorsAtHoldback + self.firstServeLatchCoversEngineCut = firstServeLatchCoversEngineCut self.blockingReloadOverride = blockingReloadOverride self.liveCadencePolicy = liveCadencePolicy self.codecsString = codecsString @@ -843,6 +1029,29 @@ final class VideoSegmentProvider: HLSSegmentProvider, @unchecked Sendable { self.segmentServedHandler = segmentServedHandler } + /// AE#684: where a live segment's sound begins and ends on the output axis, by segment index. + /// Diagnostic only: it lets an item start say what it started its audio from. Bounded, oldest out. + private var _liveSegmentSound: [Int: (first: Double, last: Double)] = [:] + static let liveSegmentSoundRetained = 1024 + + func noteLiveSegmentSound(index: Int, firstSeconds: Double, lastSeconds: Double) { + stateLock.lock() + _liveSegmentSound[index] = (firstSeconds, lastSeconds) + _liveSegmentSound.removeValue(forKey: index - Self.liveSegmentSoundRetained) + stateLock.unlock() + } + + /// AE#684: the segment an output-axis position falls in, with its picture's start and its + /// sound's span. nil for a position no listed segment covers. + func liveSegmentHeads(atOutputSeconds seconds: Double) + -> (index: Int, secondsIntoSegment: Double, pictureStart: Double, sound: (first: Double, last: Double)?)? { + stateLock.lock() + defer { stateLock.unlock() } + guard let idx = Self.thumbnailSegmentIndex(atSeconds: seconds, segments: segments) else { return nil } + return (idx, Swift.max(0, seconds - segments[idx].startSeconds), segments[idx].startSeconds, + _liveSegmentSound[idx]) + } + /// Append a finalized live segment. Index must equal segments.count; out-of-order ignored. func appendLiveSegment(index: Int, startSeconds: Double, durationSeconds: Double, discontinuous: Bool = false) { @@ -1349,6 +1558,19 @@ final class VideoSegmentProvider: HLSSegmentProvider, @unchecked Sendable { // Cache gate: backwardWindow=20 covers Continuous-Audio handover refetches (~7-10 segments // backward); unconditional proactive restart re-armed the FLAC bridge and caused audible glitches. if cache.peekURL(index: index) != nil { + // A sequential origin has no restart to offer (its pump can only read from byte 0) + // and keeps every segment it cut, so a resident target is served from the cache. + // The contiguity scan below would read the plan indices its cutter left as holes + // (no URI, never listed) as a gap and hand the refusal a source failure to publish, + // which is what an AirPlay hop back to the loopback item did at 22 s in. + if sequentialAppendPlaylist { + EngineLog.emit( + "[HLSVideoEngine] declareTarget backward jump \(previousTarget) -> \(index): " + + "resident on a sequential origin, served from the cache", + category: .session + ) + return + } let frontier = cache.contiguousForwardFrontier(from: index) let front = activeMarchFront if Self.residentBackwardTargetKeepsProducer( @@ -1570,6 +1792,9 @@ final class VideoSegmentProvider: HLSSegmentProvider, @unchecked Sendable { if let bytes = cache.fetch(index: index, timeout: repositionWaitSlice) { return logServed(index: index, bytes: bytes, totalStart: totalStart, restarted: true) } + // Audit SEG-3: a closed cache answers fetch at once, so riding a restart that + // outlives stop() would spin this thread until the ride cap. + if cache.isClosed { break } } return logServed(index: index, bytes: nil, totalStart: totalStart, restarted: true) } @@ -2248,19 +2473,59 @@ final class VideoSegmentProvider: HLSSegmentProvider, @unchecked Sendable { let floor: CadenceFloorTerm = liveCadencePolicy.map { policy in policy.targetDurationFloorSeconds.map { CadenceFloorTerm.measured($0) } ?? .pending } ?? .unmeasurable + let cutHere = liveCadencePolicy == nil + let upstreamSegment = liveCadencePolicy?.upstreamSegmentDurationSeconds return LiveTargetDurationDerivation( value: LiveEdgePolicy.targetDurationSeconds( maxSegmentDuration: maxSegmentDuration, cutTargetSeconds: cutTarget, - cadenceFloorSeconds: floor.seconds + cadenceFloorSeconds: floor.seconds, + segmentsAreCutHere: cutHere, + upstreamSegmentSeconds: upstreamSegment ), maxSegmentDuration: maxSegmentDuration, cutTargetFloor: cutTarget, + gopHeadroomApplies: LiveEdgePolicy.gopHeadroomApplies(cutTargetSeconds: cutTarget, + segmentsAreCutHere: cutHere), cadenceFloor: floor, + upstreamSegment: upstreamSegment, selfReported: liveCadencePolicy?.selfReportedTargetDurationSeconds ) } + /// AE#684 review: the derivation the first-serve gate judges its cushion against, which is the + /// full one unless an ingest's join is exhausted short of it + /// (`LiveEdgePolicy.targetDurationTheJoinCanPay`). Only the gate asks this: once it has sealed, + /// the value is frozen, and a later build's fuller candidate is the drift line's to report. + /// + /// `joinIsSpent` has to have been read BEFORE `snap` was taken. The fact is monotone, so a + /// snapshot taken after it was true holds everything the join will ever cut; asked the other way + /// round, the cutter can append its last segment and park between the two reads, and the seal is + /// paid from the sum without it (4 where the window holds 5). + func firstServeTargetDuration( + _ snap: (count: Int, summed: Double, maxDuration: Double), + joinIsSpent: Bool? + ) -> LiveTargetDurationDerivation { + var derivation = currentLiveTargetDuration(maxSegmentDuration: snap.maxDuration) + guard derivation.upstreamSegment != nil, + snap.count >= LiveEdgePolicy.minStartupSegments, + joinIsSpent == true + else { return derivation } + let backlog = liveCadencePolicy?.joinBacklogSeconds ?? 0 + let base = LiveEdgePolicy.targetDurationSeconds( + maxSegmentDuration: snap.maxDuration, + cutTargetSeconds: derivation.cutTargetFloor, + cadenceFloorSeconds: derivation.cadenceFloor.seconds, + segmentsAreCutHere: false, + upstreamSegmentSeconds: nil) + let paid = LiveEdgePolicy.targetDurationTheJoinCanPay( + full: derivation.value, withoutUpstreamSegment: base, finalizedSeconds: snap.summed) + guard paid < derivation.value else { return derivation } + derivation.joinBound = (backlog, snap.summed, derivation.value) + derivation.value = paid + return derivation + } + /// Takes the seal and, on the call that actually takes it, publishes the derivation. The value is /// frozen for the session (RFC 8216 forbids a changing TARGETDURATION, AE#209), so the one line that /// explains it has to be emitted here or nowhere: a host reading a 9 s holdback later has no way to @@ -2290,6 +2555,7 @@ final class VideoSegmentProvider: HLSSegmentProvider, @unchecked Sendable { + "\(resolved.value)s, later candidate \(candidate.value)s " + "(max segment \(String(format: "%.3f", maxSegmentDuration))s, " + candidate.cadenceFloor.account + + (candidate.upstreamSegmentAccount.map { ", " + $0 } ?? "") + ")", category: .session ) @@ -2346,8 +2612,16 @@ final class VideoSegmentProvider: HLSSegmentProvider, @unchecked Sendable { defer { parkedWaiters -= 1 } while true { if waitersCancelled { return false } + // AE#684 latched ingest sessions; AE#686 extends it to a source the engine cuts itself, + // where the second grace bought nothing on device but a session 1 s further from the edge. + if firstManifestServed, liveCadencePolicy != nil || firstServeLatchCoversEngineCut { + accountForRepeatServe(since: enteredAt, note: "first-serve latch") + return true + } + // Spent first, snapshot second (see `firstServeTargetDuration`). + let spent = liveCadencePolicy?.joinIsSpent let snap = liveCushionSnapshot() - let target = currentLiveTargetDuration(maxSegmentDuration: snap.maxDuration) + let target = firstServeTargetDuration(snap, joinIsSpent: spent) if LiveEdgePolicy.startupCushionSatisfied(segmentCount: snap.count, summedDurationSeconds: snap.summed, targetDuration: target.value, @@ -2366,12 +2640,18 @@ final class VideoSegmentProvider: HLSSegmentProvider, @unchecked Sendable { degradedGrace = grace degradedDeadline = Date().addingTimeInterval(grace) } - let effectiveDeadline = degradedDeadline.map { min(deadline, $0) } ?? deadline + var effectiveDeadline = degradedDeadline.map { min(deadline, $0) } ?? deadline + // AE#684: a join becomes spent when the cutter parks on an empty reader, which is no + // event this condition hears. While that is still to come, look again shortly. + if spent == false { + effectiveDeadline = min(effectiveDeadline, Date().addingTimeInterval(Self.joinSpentPollSeconds)) + } if !firstSegmentCondition.wait(until: effectiveDeadline) { // Re-read after the timed-out wait: an append racing the deadline would otherwise be judged // on the stale snapshot (waitForLiveSegment below already does this). + let afterSpent = liveCadencePolicy?.joinIsSpent let after = liveCushionSnapshot() - let afterTarget = currentLiveTargetDuration(maxSegmentDuration: after.maxDuration) + let afterTarget = firstServeTargetDuration(after, joinIsSpent: afterSpent) if LiveEdgePolicy.startupCushionSatisfied(segmentCount: after.count, summedDurationSeconds: after.summed, targetDuration: afterTarget.value, @@ -2416,6 +2696,9 @@ final class VideoSegmentProvider: HLSSegmentProvider, @unchecked Sendable { } } + /// How often the gate asks an ingest whether its join is spent. Well under a frame of start latency. + static let joinSpentPollSeconds: TimeInterval = 0.025 + /// AE#374: one account per live session, emitted at whichever exit ends the first-serve gate. /// /// Every `/media.m3u8` request without an `_HLS_msn` re-enters the gate, so without the latch a @@ -2428,7 +2711,11 @@ final class VideoSegmentProvider: HLSSegmentProvider, @unchecked Sendable { warning: Bool = false, note: String? = nil ) { - guard !didAccountForFirstServe else { return } + firstManifestServed = true + guard !didAccountForFirstServe else { + accountForRepeatServe(since: entered, note: note) + return + } didAccountForFirstServe = true let account = LiveEdgePolicy.firstServeAccount( waitedSeconds: Self.secondsSince(entered), @@ -2443,6 +2730,19 @@ final class VideoSegmentProvider: HLSSegmentProvider, @unchecked Sendable { ) } + /// AE#686: AVPlayer opens with two plain manifest requests, and how long the second one waited is + /// the measurement the latch arm is about. Reported once, from whichever exit let it through. + private func accountForRepeatServe(since entered: DispatchTime, note: String?) { + guard !didAccountForRepeatServe else { return } + didAccountForRepeatServe = true + EngineLog.emit( + "[HLSVideoEngine] repeat live manifest request held " + + "\(LiveEdgePolicy.seconds(Self.secondsSince(entered)))s" + + (note.map { ", \($0)" } ?? ""), + category: .session + ) + } + /// The gate's other ending: the deadline passed and not one segment was ever cut, so there is no /// window to describe and the manifest cannot be served at all. private func accountForUnservedFirstManifest(since entered: DispatchTime) { @@ -2507,6 +2807,23 @@ final class VideoSegmentProvider: HLSSegmentProvider, @unchecked Sendable { /// AE#446: how often a blocking-reload hold re-asks whether the source is still alive. static let liveHoldRecheckSeconds: TimeInterval = 1.0 + /// AE#682. Set once by the session after the routing decision, before the playback URL leaves + /// `start()`; cleared when the session falls back to the media playlist or stops. + func setIFrameSource(_ source: IFrameSegmentSource?) { + iFrameSourceLock.lock() + _iFrameSource = source + iFrameSourceLock.unlock() + } + + private var iFrameSource: IFrameSegmentSource? { + iFrameSourceLock.lock(); defer { iFrameSourceLock.unlock() } + return _iFrameSource + } + + var iFrameRenditionServed: Bool { iFrameSource != nil } + func iFrameInitSegment() -> Data? { iFrameSource?.initSegment() } + func iFrameSegment(at index: Int) -> Data? { iFrameSource?.fragment(at: index) } + var masterCodecs: String? { codecsString } var masterSupplementalCodecs: String? { supplementalCodecsString } var masterResolution: (width: Int, height: Int)? { diff --git a/Sources/AetherEngine/VideoStreamFormat.swift b/Sources/AetherEngine/VideoStreamFormat.swift new file mode 100644 index 000000000..870ba61bc --- /dev/null +++ b/Sources/AetherEngine/VideoStreamFormat.swift @@ -0,0 +1,173 @@ +import Foundation +import CoreVideo +import AetherLibavutil +import AetherLibavcodec + +/// AE#658: a video picture's format in FFmpeg's vocabulary, for a host's stats panel. +/// +/// The raw fields are libav names (`yuv420p10le`, `bt2020`, `smpte2084`, `bt2020nc`, `tv`), nil where +/// the stream leaves a value unspecified. A gap stays a gap: an untagged source is not reported as +/// BT.709 here, even though that is what the display layer is handed for it (AE#654). The `…Label` +/// properties turn them into the names a viewer reads. +public struct VideoStreamFormat: Sendable, Equatable { + public let pixelFormat: String? + /// Bits per luma sample, from the pixel format's descriptor (or the stream's declared sample depth + /// when the pixel format is unknown). + public let bitDepth: Int? + public let colorPrimaries: String? + public let transfer: String? + public let matrix: String? + public let range: String? + /// Codec profile as libavcodec names it ("Main 10", "High", "Main"), nil when undeclared. + public let profile: String? + + public init(pixelFormat: String?, bitDepth: Int?, colorPrimaries: String?, transfer: String?, + matrix: String?, range: String?, profile: String?) { + self.pixelFormat = pixelFormat + self.bitDepth = bitDepth + self.colorPrimaries = colorPrimaries + self.transfer = transfer + self.matrix = matrix + self.range = range + self.profile = profile + } + + init(pixelFormat: AVPixelFormat, + declaredBitDepth: Int32, + color: ColorDescription, + codecID: AVCodecID, + profile: Int32) { + let pixName = pixelFormat == AV_PIX_FMT_NONE ? nil : av_get_pix_fmt_name(pixelFormat).map { String(cString: $0) } + var depth: Int? = nil + if pixelFormat != AV_PIX_FMT_NONE, let desc = av_pix_fmt_desc_get(pixelFormat), desc.pointee.nb_components > 0 { + depth = Int(desc.pointee.comp.0.depth) + } else if declaredBitDepth > 0 { + depth = Int(declaredBitDepth) + } + self.init( + pixelFormat: pixName, + bitDepth: depth, + colorPrimaries: color.primaries == AVCOL_PRI_UNSPECIFIED + ? nil : av_color_primaries_name(color.primaries).map { String(cString: $0) }, + transfer: color.transfer == AVCOL_TRC_UNSPECIFIED + ? nil : av_color_transfer_name(color.transfer).map { String(cString: $0) }, + matrix: color.matrix == AVCOL_SPC_UNSPECIFIED + ? nil : av_color_space_name(color.matrix).map { String(cString: $0) }, + range: color.range == AVCOL_RANGE_UNSPECIFIED + ? nil : av_color_range_name(color.range).map { String(cString: $0) }, + profile: Self.profileName(codecID: codecID, profile: profile)) + } + + /// What the container and the probe's decoder declared for the stream. + init(codecpar: UnsafePointer) { + self.init( + pixelFormat: AVPixelFormat(rawValue: codecpar.pointee.format), + declaredBitDepth: codecpar.pointee.bits_per_raw_sample, + color: ColorDescription(codecpar: codecpar), + codecID: codecpar.pointee.codec_id, + profile: codecpar.pointee.profile) + } + + static func profileName(codecID: AVCodecID, profile: Int32) -> String? { + guard profile != AV_PROFILE_UNKNOWN else { return nil } + return avcodec_profile_name(codecID, profile).map { String(cString: $0) } + } + + public var colorPrimariesLabel: String? { colorPrimaries.map(Self.primariesLabel) } + public var transferLabel: String? { transfer.map(Self.transferLabel) } + public var matrixLabel: String? { matrix.map(Self.matrixLabel) } + public var rangeLabel: String? { range.map(Self.rangeLabel) } + + static func primariesLabel(_ name: String) -> String { + switch name { + case "bt709": return "BT.709" + case "bt2020": return "BT.2020" + case "smpte432": return "Display P3" + case "smpte431": return "DCI-P3" + case "bt470bg": return "BT.601 PAL" + case "smpte170m": return "BT.601 NTSC" + case "bt470m": return "BT.470 M" + case "smpte240m": return "SMPTE 240M" + default: return name + } + } + + static func transferLabel(_ name: String) -> String { + switch name { + case "bt709": return "BT.709 (SDR)" + case "smpte2084": return "PQ (SMPTE ST 2084)" + case "arib-std-b67": return "HLG" + case "smpte170m", "bt601": return "BT.601 (SDR)" + case "bt2020-10", "bt2020-12": return "BT.2020 (SDR)" + case "iec61966-2-1": return "sRGB" + case "linear": return "Linear" + default: return name + } + } + + static func matrixLabel(_ name: String) -> String { + switch name { + case "bt709": return "BT.709" + case "bt2020nc": return "BT.2020 NCL" + case "bt2020c": return "BT.2020 CL" + case "bt470bg", "smpte170m": return "BT.601" + case "ictcp": return "ICtCp" + case "gbr": return "RGB" + default: return name + } + } + + static func rangeLabel(_ name: String) -> String { + switch name { + case "tv": return "Limited" + case "pc": return "Full" + default: return name + } + } +} + +/// AE#658: what the engine's own software decoder produced, as it produced it. Published only on the +/// software path, because on the native path AVPlayer decodes and the frames never pass through the engine. +public struct DecodedVideoFormat: Sendable, Equatable { + /// The decoded frame: libavcodec's output pixel format and the colour description after the + /// container filled the fields the bitstream left open. + public let frame: VideoStreamFormat + /// The CoreVideo buffer the frame was converted into for display, as its four-character code + /// (`420v` = NV12, `x420` = P010). + public let pixelBufferFormat: String + + public init(frame: VideoStreamFormat, pixelBufferFormat: String) { + self.frame = frame + self.pixelBufferFormat = pixelBufferFormat + } + + /// The display buffer in the names a viewer reads, "P010 (x420)". + public var pixelBufferLabel: String { + switch pixelBufferFormat { + case "420v": return "NV12 (420v)" + case "420f": return "NV12 full range (420f)" + case "x420": return "P010 (x420)" + case "xf20": return "P010 full range (xf20)" + default: return pixelBufferFormat + } + } + + static func libavPixelFormat(forPixelBufferType type: OSType) -> AVPixelFormat { + switch type { + case kCVPixelFormatType_420YpCbCr8BiPlanarVideoRange, + kCVPixelFormatType_420YpCbCr8BiPlanarFullRange: return AV_PIX_FMT_NV12 + case kCVPixelFormatType_420YpCbCr10BiPlanarVideoRange, + kCVPixelFormatType_420YpCbCr10BiPlanarFullRange: return AV_PIX_FMT_P010LE + default: return AV_PIX_FMT_NONE + } + } + + static func fourCC(_ type: OSType) -> String { + let bytes = [UInt8((type >> 24) & 0xFF), UInt8((type >> 16) & 0xFF), + UInt8((type >> 8) & 0xFF), UInt8(type & 0xFF)] + if bytes.allSatisfy({ $0 >= 0x20 && $0 < 0x7F }) { + return String(decoding: bytes, as: UTF8.self) + } + return "0x" + String(type, radix: 16) + } +} diff --git a/Sources/aetherctl/ArgParsing.swift b/Sources/aetherctl/ArgParsing.swift index f4e2818aa..e65aec6c6 100644 --- a/Sources/aetherctl/ArgParsing.swift +++ b/Sources/aetherctl/ArgParsing.swift @@ -43,6 +43,38 @@ func takeDoubleFlag(_ name: String, from rest: inout [String]) -> Double? { return value } +/// Longest run a duration flag accepts: a week. `isFinite` alone let `--seconds 1e30` through to the +/// run loop's `Int(_:)`, which trapped after the session was up (audit OPS-110). +let maxRunSeconds: Double = 7 * 24 * 3600 + +/// `takeDoubleFlag`, and the value must lie in `range`. Exits 64 otherwise. +func takeDoubleFlag(_ name: String, in range: ClosedRange, from rest: inout [String]) -> Double? { + guard let value = takeDoubleFlag(name, from: &rest) else { return nil } + guard range.contains(value) else { + print("ERROR: \(name) expects a value in \(range.lowerBound)...\(range.upperBound), got \(value)") + exit(64) + } + return value +} + +/// `takeIntFlag`, and the value must lie in `range`. Exits 64 otherwise: a negative count reached +/// `0.., from rest: inout [String]) -> Int? { + guard let value = takeIntFlag(name, from: &rest) else { return nil } + guard range.contains(value) else { + print("ERROR: \(name) expects an integer in \(range.lowerBound)...\(range.upperBound), got \(value)") + exit(64) + } + return value +} + +/// A host-call delay in milliseconds as a sleep, saturating: `UInt64(ms) * 1_000_000` overflowed past +/// about 1.8e13 ms (audit OPS-110). +func sleepNanoseconds(milliseconds: Int) -> UInt64 { + let (nanoseconds, overflow) = UInt64(max(0, milliseconds)).multipliedReportingOverflow(by: 1_000_000) + return overflow ? .max : nanoseconds +} + /// Exit 64 with an error message if any `--flag` remains in `rest` after plucking known flags (typos would silently become the URL positional otherwise). func rejectStrayFlags(_ rest: [String], subcommand: String) { if let stray = rest.first(where: { $0.hasPrefix("--") }) { diff --git a/Sources/aetherctl/AudioCmd.swift b/Sources/aetherctl/AudioCmd.swift index 1deefb842..5d3752aef 100644 --- a/Sources/aetherctl/AudioCmd.swift +++ b/Sources/aetherctl/AudioCmd.swift @@ -5,7 +5,7 @@ import AetherEngine /// Load a source through the audio-only path and play it, printing the synchronizer clock once a second. Smoke-tests FFmpeg decode -> AVSampleBufferAudioRenderer on macOS. func runAudio(url: URL, seconds playSeconds: Double) -> Int32 { - print("aetherctl audio: \(url.absoluteString) (play \(playSeconds)s)") + print(EngineLog.redacted("aetherctl audio: \(url.absoluteString) (play \(playSeconds)s)")) // Must use CFRunLoopRun, not a blocking semaphore: AetherEngine is @MainActor, so parking the main thread would deadlock the executor. let box = UncheckedBox(nil) Task { @MainActor in diff --git a/Sources/aetherctl/AudioTapCmd.swift b/Sources/aetherctl/AudioTapCmd.swift index d1c50aad1..12c62f19a 100644 --- a/Sources/aetherctl/AudioTapCmd.swift +++ b/Sources/aetherctl/AudioTapCmd.swift @@ -8,7 +8,7 @@ func runAudioTap(url: URL, duration: Double, outPath: String, if software { return runSoftwareAudioTap(url: url, duration: duration, outPath: outPath) } EngineLog.handler = { print($0) } let mode = remote ? "remote-HLS" : "loopback" - print("aetherctl audiotap (\(mode)): \(url.absoluteString) duration=\(duration)s out=\(outPath)") + print(EngineLog.redacted("aetherctl audiotap (\(mode)): \(url.absoluteString) duration=\(duration)s out=\(outPath)")) do { let report = remote ? try AudioTapProbe.runRemote(url: url, durationSeconds: duration, outPath: outPath) @@ -26,7 +26,7 @@ func runAudioTap(url: URL, duration: Double, outPath: String, /// main thread would deadlock its executor (same shape as `bgaudio`). private func runSoftwareAudioTap(url: URL, duration: Double, outPath: String) -> Int32 { EngineLog.handler = { print($0) } - print("aetherctl audiotap (software): \(url.absoluteString) duration=\(duration)s out=\(outPath)") + print(EngineLog.redacted("aetherctl audiotap (software): \(url.absoluteString) duration=\(duration)s out=\(outPath)")) let box = UncheckedBox(nil) Task { @MainActor in do { diff --git a/Sources/aetherctl/BackgroundAudioCmd.swift b/Sources/aetherctl/BackgroundAudioCmd.swift index b0d37ee80..3504611a7 100644 --- a/Sources/aetherctl/BackgroundAudioCmd.swift +++ b/Sources/aetherctl/BackgroundAudioCmd.swift @@ -9,7 +9,7 @@ import AetherEngine /// (the macOS stand-in for the iOS background lifecycle) and verify audio keeps advancing while video is /// dropped, then resumes on foreground return. Exercises the real runDemuxLoop background branch headless. func runBackgroundAudio(url: URL, fgSeconds: Double, bgSeconds: Double) -> Int32 { - print("aetherctl bgaudio: \(url.absoluteString) (fg=\(fgSeconds)s bg=\(bgSeconds)s)") + print(EngineLog.redacted("aetherctl bgaudio: \(url.absoluteString) (fg=\(fgSeconds)s bg=\(bgSeconds)s)")) // Must use CFRunLoopRun, not a blocking semaphore: AetherEngine is @MainActor, so parking the main thread // would deadlock the executor. The run loop also lets the host's time timer + Combine sinks fire. let box = UncheckedBox(nil) diff --git a/Sources/aetherctl/DebugOutput.swift b/Sources/aetherctl/DebugOutput.swift new file mode 100644 index 000000000..286a1dc11 --- /dev/null +++ b/Sources/aetherctl/DebugOutput.swift @@ -0,0 +1,39 @@ +import Foundation + +/// Audit OPS-3: debug artefacts go into a private directory under the user's temporary directory, +/// never a fixed name in the shared `/tmp` that another local user could pre-create as a symlink. +/// Audit OPS-107: the directory is the SAME one on every run (`aetherctl-`, mode 0700, checked +/// with `lstat` for being a real directory we own), so the next run overwrites the last one's files +/// instead of leaving another multi-gigabyte `dovitest` output behind. A directory that fails the +/// check (a symlink, someone else's) is not used: the run falls back to a fresh `mkdtemp` one. +let debugOutputDirectory: String = { + let temporary = FileManager.default.temporaryDirectory + let stable = temporary.appendingPathComponent("aetherctl-\(getuid())").path + if privateOwnedDirectory(stable) { return stable } + + let template = temporary.appendingPathComponent("aetherctl-XXXXXX").path + var buffer = Array(template.utf8CString) + if let made = mkdtemp(&buffer) { + return String(cString: made) + } + let fallback = temporary.appendingPathComponent("aetherctl-\(UUID().uuidString)").path + try? FileManager.default.createDirectory(atPath: fallback, withIntermediateDirectories: false, + attributes: [.posixPermissions: 0o700]) + return fallback +}() + +/// Creates `path` (mode 0700) when it is absent and reports whether what stands there is a real +/// directory owned by this user that nobody else can enter. +private func privateOwnedDirectory(_ path: String) -> Bool { + if mkdir(path, 0o700) != 0, errno != EEXIST { return false } + var info = stat() + guard lstat(path, &info) == 0, + (info.st_mode & S_IFMT) == S_IFDIR, + info.st_uid == getuid() else { return false } + if info.st_mode & 0o077 != 0, chmod(path, 0o700) != 0 { return false } + return true +} + +func debugOutputPath(_ name: String) -> String { + (debugOutputDirectory as NSString).appendingPathComponent(name) +} diff --git a/Sources/aetherctl/DiscInspect.swift b/Sources/aetherctl/DiscInspect.swift index e33f1ab11..3852fb55f 100644 --- a/Sources/aetherctl/DiscInspect.swift +++ b/Sources/aetherctl/DiscInspect.swift @@ -8,7 +8,7 @@ import AetherEngine /// does detection bail?". Exit 0 when the image is recognized as playable, else 1. func runDiscInspect(url: URL, dump: Bool = false) -> Int32 { EngineLog.handler = { print($0) } - print("aetherctl disc-inspect: \(url.absoluteString)") + print(EngineLog.redacted("aetherctl disc-inspect: \(url.absoluteString)")) print("") let d = AetherEngine.inspectDisc(url: url, verbose: dump) diff --git a/Sources/aetherctl/DoviTest.swift b/Sources/aetherctl/DoviTest.swift index afded3c42..e4817929d 100644 --- a/Sources/aetherctl/DoviTest.swift +++ b/Sources/aetherctl/DoviTest.swift @@ -3,10 +3,13 @@ import AetherEngine // MARK: - dovitest -/// Validate DoviRpuConverter.convertPacketToProfile81 against dovi_tool ground truth. Walks the HEVC stream, converts DV P7 -> P8.1, writes Annex-B to /tmp/aetherctl-dovitest.hevc. -func runDoviTest(url: URL) -> Int32 { - let outputPath = "/tmp/aetherctl-dovitest.hevc" - print("aetherctl dovitest: \(url.absoluteString)") +/// Validate DoviRpuConverter.convertPacketToProfile81 against dovi_tool ground truth. Walks the HEVC stream, converts DV P7 -> P8.1, writes Annex-B to `outputPath`, by default aetherctl-dovitest.hevc in the per-user private temporary directory (overwritten by the next run). +func runDoviTest(url: URL, outputPath: String? = nil) -> Int32 { + let outputPath = outputPath ?? debugOutputPath("aetherctl-dovitest.hevc") + let rpuPath = outputPath.hasSuffix(".hevc") + ? String(outputPath.dropLast(".hevc".count)) + ".rpu" + : outputPath + ".rpu" + print(EngineLog.redacted("aetherctl dovitest: \(url.absoluteString)")) print("output: \(outputPath)") print("") @@ -35,14 +38,14 @@ func runDoviTest(url: URL) -> Int32 { if result.failures > 0 { print("VERDICT: dovitest had \(result.failures) converter failure(s).") print(" Validate the surviving RPUs against dovi_tool, then debug:") - print(" dovi_tool extract-rpu -i \(outputPath) -o /tmp/host.rpu") - print(" dovi_tool info -i /tmp/host.rpu -f 0") + print(" dovi_tool extract-rpu -i \(outputPath) -o \(rpuPath)") + print(" dovi_tool info -i \(rpuPath) -f 0") return 3 } print("VERDICT: converted \(result.conversions) packet(s) to Profile 8.1.") print(" Validate against dovi_tool ground truth:") - print(" dovi_tool extract-rpu -i \(outputPath) -o /tmp/host.rpu") - print(" dovi_tool info -i /tmp/host.rpu -f 0 | grep -iE 'dovi_profile|disable_residual|rpu_data_crc32'") + print(" dovi_tool extract-rpu -i \(outputPath) -o \(rpuPath)") + print(" dovi_tool info -i \(rpuPath) -f 0 | grep -iE 'dovi_profile|disable_residual|rpu_data_crc32'") return 0 } diff --git a/Sources/aetherctl/Extract.swift b/Sources/aetherctl/Extract.swift index ff36b4f58..a9bff6c1a 100644 --- a/Sources/aetherctl/Extract.swift +++ b/Sources/aetherctl/Extract.swift @@ -29,7 +29,7 @@ private func writePNG(_ image: CGImage, to path: String) -> Bool { func runExtract(url: URL, at seconds: Double, mode: FrameMode, loops: Int, maxWidth: Int) -> Int32 { EngineLog.handler = { print($0) } - print("aetherctl extract: \(url.absoluteString) at=\(seconds)s mode=\(mode) loops=\(loops)") + print(EngineLog.redacted("aetherctl extract: \(url.absoluteString) at=\(seconds)s mode=\(mode) loops=\(loops)")) print("") let extractor = FrameExtractor(url: url, httpHeaders: [:]) @@ -49,7 +49,7 @@ func runExtract(url: URL, at seconds: Double, mode: FrameMode, loops: Int, maxWi if let image { produced += 1 if i == 0 { - let out = "/tmp/aetherctl-extract-\(mode).png" + let out = debugOutputPath("aetherctl-extract-\(mode).png") if writePNG(image, to: out) { print("Wrote \(image.width)x\(image.height) -> \(out)") } else { diff --git a/Sources/aetherctl/HLSLiveRepro.swift b/Sources/aetherctl/HLSLiveRepro.swift index 03692b3ac..868e082f7 100644 --- a/Sources/aetherctl/HLSLiveRepro.swift +++ b/Sources/aetherctl/HLSLiveRepro.swift @@ -23,8 +23,12 @@ func runHLSLiveRepro(args: [String]) -> Int32 { return 64 } let paths = segList.split(separator: ",").map(String.init) - let seconds = takeIntFlag("--seconds", from: &rest) ?? 40 - let segSeconds = takeIntFlag("--segment-seconds", from: &rest) ?? 5 + guard !paths.isEmpty else { + print("ERROR: hlslive requires at least one segment in --segments") + return 64 + } + let seconds = takeIntFlag("--seconds", in: 0...Int(maxRunSeconds), from: &rest) ?? 40 + let segSeconds = takeIntFlag("--segment-seconds", in: 1...3600, from: &rest) ?? 5 let discFlag = takeStringFlag("--disc", from: &rest) var slices: [[UInt8]] = [] diff --git a/Sources/aetherctl/PktDump.swift b/Sources/aetherctl/PktDump.swift index f90f4c677..b96ce9474 100644 --- a/Sources/aetherctl/PktDump.swift +++ b/Sources/aetherctl/PktDump.swift @@ -9,7 +9,7 @@ import AetherEngine /// restartReopen confirms the skipped find_stream_info pass as the judder source. func runPktDump(url: URL, at seconds: Double, count: Int, profileName: String) -> Int32 { EngineLog.handler = { print($0) } - print("aetherctl pktdump: \(url.absoluteString)") + print(EngineLog.redacted("aetherctl pktdump: \(url.absoluteString)")) print(" profile=\(profileName) at=\(seconds)s count=\(count)") print("") do { diff --git a/Sources/aetherctl/PlaybackCmd.swift b/Sources/aetherctl/PlaybackCmd.swift index 0eae838cb..61e3f4823 100644 --- a/Sources/aetherctl/PlaybackCmd.swift +++ b/Sources/aetherctl/PlaybackCmd.swift @@ -98,8 +98,8 @@ enum LoadOptionChange { /// and optionally activate an embedded subtitle track (`--subs `) /// and log every overlay cue that arrives. Repro harness for "loads but never /// plays" reports and for live teletext end-to-end validation (#107). -func runPlay(url: URL, seconds: Double, live: Bool, nativeHLS: Bool = false, liveIngest: Bool = false, fastZap: Bool = false, liveStartImmediately: Bool = true, dvrWindow: Double?, subsPick: String?, hostCalls: [String], audioStats: Bool = false, seekEvery: Double? = nil, seekPattern: [Double] = [], seekCount: Int? = nil, startPosition: Double? = nil, mallocCensus: Bool = false, forceSoftware: Bool = false, - censusThresholdMB: Int? = nil, censusHz: Double? = nil, frameTimes: Bool = false, presentTimes: Bool = false, pictureProbe: Bool = false, pictureOrigin: Double = 0, +func runPlay(url: URL, seconds: Double, live: Bool, nativeHLS: Bool = false, liveIngest: Bool = false, fastZap: Bool = false, liveStartImmediately: Bool = true, dvrWindow: Double?, subsPick: String?, hostCalls: [String], audioStats: Bool = false, seekEvery: Double? = nil, seekPattern: [Double] = [], seekCount: Int? = nil, startPosition: Double? = nil, mallocCensus: Bool = false, forceSoftware: Bool = false, softwareEscalation: Bool = true, + censusThresholdMB: Int? = nil, censusHz: Double? = nil, frameTimes: Bool = false, presentTimes: Bool = false, servedURL: Bool = false, pictureProbe: Bool = false, pictureOrigin: Double = 0, sidecars: [ExternalSubtitleTrack] = [], audioSwitch: AudioSwitchRequest? = nil, teletextPage: Int? = nil, teletextSwitch: TeletextPageSwitchRequest? = nil, audioDelayMs: Int = 0, audioDelaySwitches: [AudioDelaySwitchRequest] = [], @@ -122,16 +122,17 @@ func runPlay(url: URL, seconds: Double, live: Bool, nativeHLS: Bool = false, liv // rather than the process-global test hook it used before. The hook forces every session on the // engine, so it could never exercise the thing a host actually calls. if forceSoftware { print("[aetherctl] decode path: preferredDecodePath=.software (#461)") } + if !softwareEscalation { print("[aetherctl] escalatesToSoftwarePath=false (AE#629)") } if let audioSwitch { print("[aetherctl] audio switch: selectAudioTrack(index: \(audioSwitch.index)) " + "\(audioSwitch.delayMilliseconds) ms after the load returns") } - print("aetherctl play: \(url.absoluteString) (seconds=\(seconds) live=\(live) nativeHLS=\(nativeHLS) liveIngest=\(liveIngest) dvrWindow=\(dvrWindow.map { String($0) } ?? "nil") subs=\(subsPick ?? "off") hostCalls=\(hostCalls.isEmpty ? "none" : hostCalls.joined(separator: "+")) audioStats=\(audioStats) seekEvery=\(seekEvery.map { String($0) } ?? "off") seekCount=\(seekCount.map { String($0) } ?? "unbounded") seekPattern=\(seekPattern.isEmpty ? "off" : seekPattern.map { String($0) }.joined(separator: "/")) startPosition=\(startPosition.map { String($0) } ?? "0"))") + print(EngineLog.redacted("aetherctl play: \(url.absoluteString) (seconds=\(seconds) live=\(live) nativeHLS=\(nativeHLS) liveIngest=\(liveIngest) dvrWindow=\(dvrWindow.map { String($0) } ?? "nil") subs=\(subsPick ?? "off") hostCalls=\(hostCalls.isEmpty ? "none" : hostCalls.joined(separator: "+")) audioStats=\(audioStats) seekEvery=\(seekEvery.map { String($0) } ?? "off") seekCount=\(seekCount.map { String($0) } ?? "unbounded") seekPattern=\(seekPattern.isEmpty ? "off" : seekPattern.map { String($0) }.joined(separator: "/")) startPosition=\(startPosition.map { String($0) } ?? "0"))")) print("") // CFRunLoopRun, not a blocking semaphore: AetherEngine is @MainActor, so parking the main thread would deadlock the executor. let box = UncheckedBox(nil) Task { @MainActor in - box.value = await playSmokeTest(url: url, seconds: seconds, live: live, forceSoftware: forceSoftware, nativeHLS: nativeHLS, liveIngest: liveIngest, fastZap: fastZap, liveStartImmediately: liveStartImmediately, dvrWindow: dvrWindow, subsPick: subsPick, hostCalls: hostCalls, audioStats: audioStats, seekEvery: seekEvery, seekPattern: seekPattern, seekCount: seekCount, startPosition: startPosition, frameTimes: frameTimes, presentTimes: presentTimes, pictureProbe: pictureProbe, pictureOrigin: pictureOrigin, sidecars: sidecars, audioSwitch: audioSwitch, teletextPage: teletextPage, teletextSwitch: teletextSwitch, audioDelayMs: audioDelayMs, audioDelaySwitches: audioDelaySwitches, pausedMount: pausedMount, optionCorrection: optionCorrection, sequentialOrigin: sequentialOrigin, maxConcurrentRequests: maxConcurrentRequests, heldConnection: heldConnection, declaredDuration: declaredDuration, httpHeaders: httpHeaders, deinterlaceFieldRate: deinterlaceFieldRate, assertDolbyVision: assertDolbyVision, preserveASSMarkup: preserveASSMarkup, dolbyVisionHandling: dolbyVisionHandling, record: record) + box.value = await playSmokeTest(url: url, seconds: seconds, live: live, forceSoftware: forceSoftware, softwareEscalation: softwareEscalation, nativeHLS: nativeHLS, liveIngest: liveIngest, fastZap: fastZap, liveStartImmediately: liveStartImmediately, dvrWindow: dvrWindow, subsPick: subsPick, hostCalls: hostCalls, audioStats: audioStats, seekEvery: seekEvery, seekPattern: seekPattern, seekCount: seekCount, startPosition: startPosition, frameTimes: frameTimes, presentTimes: presentTimes, servedURL: servedURL, pictureProbe: pictureProbe, pictureOrigin: pictureOrigin, sidecars: sidecars, audioSwitch: audioSwitch, teletextPage: teletextPage, teletextSwitch: teletextSwitch, audioDelayMs: audioDelayMs, audioDelaySwitches: audioDelaySwitches, pausedMount: pausedMount, optionCorrection: optionCorrection, sequentialOrigin: sequentialOrigin, maxConcurrentRequests: maxConcurrentRequests, heldConnection: heldConnection, declaredDuration: declaredDuration, httpHeaders: httpHeaders, deinterlaceFieldRate: deinterlaceFieldRate, assertDolbyVision: assertDolbyVision, preserveASSMarkup: preserveASSMarkup, dolbyVisionHandling: dolbyVisionHandling, record: record) CFRunLoopStop(CFRunLoopGetMain()) } CFRunLoopRun() @@ -152,6 +153,8 @@ func runPlay(url: URL, seconds: Double, live: Bool, nativeHLS: Bool = false, liv private func networkTelemetryFragment(_ telemetry: LiveTelemetry?) -> String { guard let telemetry else { return "" } var out = "" + if let inst = telemetry.instantBitrateMbps { out += String(format: " inst=%.2fMbps", inst) } + if let avg = telemetry.averageBitrateMbps { out += String(format: " avg=%.2fMbps", avg) } if let mbps = telemetry.networkThroughputMbps { out += String(format: " net=%.2fMbps", mbps) } if let rx = telemetry.networkTransferredBytes { out += String(format: " rx=%.1fMB", Double(rx) / 1_048_576) } out += String(format: " origin=%.1fMB", Double(telemetry.demuxerBytesFetched) / 1_048_576) @@ -450,7 +453,7 @@ private func seekIntentDrill( } @MainActor -private func playSmokeTest(url: URL, seconds: Double, live: Bool, forceSoftware: Bool = false, nativeHLS: Bool = false, liveIngest: Bool = false, fastZap: Bool = false, liveStartImmediately: Bool = true, dvrWindow: Double?, subsPick: String?, hostCalls: [String], audioStats: Bool, seekEvery: Double? = nil, seekPattern: [Double] = [], seekCount: Int? = nil, startPosition: Double? = nil, frameTimes: Bool = false, presentTimes: Bool = false, pictureProbe: Bool = false, pictureOrigin: Double = 0, sidecars: [ExternalSubtitleTrack] = [], audioSwitch: AudioSwitchRequest? = nil, teletextPage: Int? = nil, teletextSwitch: TeletextPageSwitchRequest? = nil, audioDelayMs: Int = 0, audioDelaySwitches: [AudioDelaySwitchRequest] = [], pausedMount: Bool = false, optionCorrection: LoadOptionCorrectionRequest? = nil, sequentialOrigin: Bool = false, maxConcurrentRequests: Int? = nil, heldConnection: Bool = false, declaredDuration: Double? = nil, httpHeaders: [String: String] = [:], deinterlaceFieldRate: DeinterlaceFieldRate = .field, assertDolbyVision: Bool = false, preserveASSMarkup: Bool = false, dolbyVisionHandling: DolbyVisionHandling = .automatic, record: URL? = nil) async -> Int32 { +private func playSmokeTest(url: URL, seconds: Double, live: Bool, forceSoftware: Bool = false, softwareEscalation: Bool = true, nativeHLS: Bool = false, liveIngest: Bool = false, fastZap: Bool = false, liveStartImmediately: Bool = true, dvrWindow: Double?, subsPick: String?, hostCalls: [String], audioStats: Bool, seekEvery: Double? = nil, seekPattern: [Double] = [], seekCount: Int? = nil, startPosition: Double? = nil, frameTimes: Bool = false, presentTimes: Bool = false, servedURL: Bool = false, pictureProbe: Bool = false, pictureOrigin: Double = 0, sidecars: [ExternalSubtitleTrack] = [], audioSwitch: AudioSwitchRequest? = nil, teletextPage: Int? = nil, teletextSwitch: TeletextPageSwitchRequest? = nil, audioDelayMs: Int = 0, audioDelaySwitches: [AudioDelaySwitchRequest] = [], pausedMount: Bool = false, optionCorrection: LoadOptionCorrectionRequest? = nil, sequentialOrigin: Bool = false, maxConcurrentRequests: Int? = nil, heldConnection: Bool = false, declaredDuration: Double? = nil, httpHeaders: [String: String] = [:], deinterlaceFieldRate: DeinterlaceFieldRate = .field, assertDolbyVision: Bool = false, preserveASSMarkup: Bool = false, dolbyVisionHandling: DolbyVisionHandling = .automatic, record: URL? = nil) async -> Int32 { let engine: AetherEngine do { engine = try AetherEngine() @@ -526,6 +529,17 @@ private func playSmokeTest(url: URL, seconds: Double, live: Bool, forceSoftware: } .store(in: &cancellables) + // AE#658: what the engine's own decoder produced; silent on the native path, where AVPlayer decodes. + engine.$decodedVideoFormat + .compactMap { $0 } + .removeDuplicates() + .sink { d in + print(" DECODED \(d.frame.pixelFormat ?? "?") depth=\(d.frame.bitDepth.map(String.init) ?? "?") " + + "primaries=\(d.frame.colorPrimaries ?? "-") transfer=\(d.frame.transfer ?? "-") " + + "matrix=\(d.frame.matrix ?? "-") range=\(d.frame.range ?? "-") -> \(d.pixelBufferLabel)") + } + .store(in: &cancellables) + let options = LoadOptions( suppressDisplayCriteria: true, httpHeaders: httpHeaders, @@ -567,12 +581,31 @@ private func playSmokeTest(url: URL, seconds: Double, live: Bool, forceSoftware: // separates "the deinterlaced path loses frames" from "it emits 2.5x as many of them" could // only be asked of a reporter. send_frame halves the output rate and changes nothing else. deinterlaceFieldRate: deinterlaceFieldRate, - preferredDecodePath: forceSoftware ? .software : .automatic + preferredDecodePath: forceSoftware ? .software : .automatic, + escalatesToSoftwarePath: softwareEscalation ) + // AE#629: the rescue a host used to see only as a route change. + let escalationSub = engine.softwarePathEscalations.sink { event in + let failure = event.absorbedFailure + print(String(format: "ESCALATION at %.2fs duringStartup=%@ absorbed=%@/%@ %@", + event.positionSeconds, String(event.duringStartup), + failure.underlyingDomain ?? "-", failure.underlyingCode.map(String.init) ?? "-", + failure.kind.rawValue)) + } + defer { escalationSub.cancel() } + // `reloadnext`: every published state, because what a host reacts to at an episode seam is a + // transition that lasts a millisecond and never shows up in the once-a-second status line. + let stateTraceStart = Date() + let stateTrace = hostCalls.contains("reloadnext") ? engine.$state.sink { state in + print(String(format: " STATE %@ t=%.3f", String(describing: state), + Date().timeIntervalSince(stateTraceStart))) + } : nil + defer { stateTrace?.cancel() } // #311: installed BEFORE the load on purpose. The engine holds it and arms the host it builds, // which is the documented usage and the part a host would otherwise have to re-do per load. let frameProbe = frameTimes ? FrameTimeProbe() : nil let presentProbe = presentTimes ? PresentedFrameProbe() : nil + var servedURLItem: ObjectIdentifier? let picture = pictureProbe ? PictureProbe(sourceOrigin: pictureOrigin) : nil if let frameProbe { engine.setSoftwareVideoFrameTimeObserver { [weak frameProbe] frame in @@ -603,6 +636,13 @@ private func playSmokeTest(url: URL, seconds: Double, live: Bool, forceSoftware: liveOptions.dvrWindowSeconds = 1800 try await engine.load(url: url, options: liveOptions) } + // The host's episode seam: the next load on the same engine while the first is still playing, + // with no stop() in between, so the native host is reused. + if hostCalls.contains("reloadnext") { + try await Task.sleep(for: .seconds(4)) + print(" HOSTCALL reload in place") + _ = try await engine.load(url: url, startPosition: startPosition, options: options) + } } catch { print("LOAD FAILED: \(error)") return 1 @@ -666,7 +706,7 @@ private func playSmokeTest(url: URL, seconds: Double, live: Bool, forceSoftware: // exists (the foreground retune's hold-paused policy). Resumed at tick 8. print(" HOSTCALL pause() right after load") engine.pause() - case "reloadlive", "seekback", "overlapseek", "ratehold-tail", "pauseseek", "pausehold", "still", "stallclock": + case "reloadlive", "reloadnext", "seekback", "overlapseek", "ratehold-tail", "pauseseek", "pausehold", "still", "stallclock", "pausereload", "playreload", "extplayreload": break // reloadlive handled at load time, seekback/overlapseek/pauseseek in the telemetry loop case "nativesubs": break // Sodalite#156, read at load time into LoadOptions.prepareNativeSubtitles @@ -674,7 +714,7 @@ private func playSmokeTest(url: URL, seconds: Double, live: Bool, forceSoftware: || call.hasPrefix("subson") || call.hasPrefix("nativerender"): break // #433 / Sodalite#156, all in the telemetry loop; `@N` picks the tick default: - print(" HOSTCALL unknown '\(call)' (use play,extractor,setrate,ratehold,pausestart,reloadlive,seekback,seekfar,overlapseek,pauseseek,pausehold,still,stallclock,nativesubs,nativerender,subsoff,subson)") + print(" HOSTCALL unknown '\(call)' (use play,extractor,setrate,ratehold,pausestart,reloadlive,reloadnext,seekback,seekfar,overlapseek,pauseseek,pausehold,still,stallclock,pausereload,playreload,extplayreload,nativesubs,nativerender,subsoff,subson)") } } defer { if let frameExtractor { Task { await frameExtractor.shutdown() } } } @@ -686,6 +726,9 @@ private func playSmokeTest(url: URL, seconds: Double, live: Bool, forceSoftware: var stallClockAtStall: Double? var stallClockBeforeResume: Double? var stallClockAtEnd: Double? + // AE#626: the clock just after the forced stage-2 reload, and at the last tick. + var recoveryReloadClockAfter: Double? + var recoveryReloadClockAtEnd: Double? var monitor: AudioContinuityMonitor? var tapTask: Task? if audioStats { @@ -717,7 +760,7 @@ private func playSmokeTest(url: URL, seconds: Double, live: Bool, forceSoftware: for (delay, group) in Dictionary(grouping: audioDelaySwitches, by: \.delayMilliseconds) .sorted(by: { $0.key < $1.key }) { Task { @MainActor in - try? await Task.sleep(nanoseconds: UInt64(max(0, delay)) * 1_000_000) + try? await Task.sleep(nanoseconds: sleepNanoseconds(milliseconds: delay)) for audioDelaySwitch in group { print(" HOSTCALL setAudioDelay(\(audioDelaySwitch.milliseconds) ms) at " + "+\(delay) ms " @@ -730,7 +773,7 @@ private func playSmokeTest(url: URL, seconds: Double, live: Bool, forceSoftware: if let teletextSwitch { Task { @MainActor in - try? await Task.sleep(nanoseconds: UInt64(max(0, teletextSwitch.delayMilliseconds)) * 1_000_000) + try? await Task.sleep(nanoseconds: sleepNanoseconds(milliseconds: teletextSwitch.delayMilliseconds)) let target = teletextSwitch.page.map(String.init) ?? "auto" print(" HOSTCALL setTeletextPage(\(target)) at +\(teletextSwitch.delayMilliseconds) ms " + "(was \(engine.teletextPage.map(String.init) ?? "auto"))") @@ -744,7 +787,7 @@ private func playSmokeTest(url: URL, seconds: Double, live: Bool, forceSoftware: // exactly the pair a host's recovery ladder has to tell apart. if let optionCorrection { Task { @MainActor in - try? await Task.sleep(nanoseconds: UInt64(max(0, optionCorrection.delayMilliseconds)) * 1_000_000) + try? await Task.sleep(nanoseconds: sleepNanoseconds(milliseconds: optionCorrection.delayMilliseconds)) let labels = optionCorrection.changes.map(\.label).joined(separator: ", ") let before = engine.currentTime print(" HOSTCALL reloadAtCurrentPosition(applying: \(labels)) at +\(optionCorrection.delayMilliseconds) ms " @@ -783,7 +826,7 @@ private func playSmokeTest(url: URL, seconds: Double, live: Bool, forceSoftware: if let audioSwitch { let mon = monitor Task { @MainActor in - try? await Task.sleep(nanoseconds: UInt64(max(0, audioSwitch.delayMilliseconds)) * 1_000_000) + try? await Task.sleep(nanoseconds: sleepNanoseconds(milliseconds: audioSwitch.delayMilliseconds)) print(" HOSTCALL selectAudioTrack(index: \(audioSwitch.index)) " + "at +\(audioSwitch.delayMilliseconds) ms (was \(engine.activeAudioTrackIndex.map(String.init) ?? "none"))") engine.selectAudioTrack(index: audioSwitch.index) @@ -902,6 +945,14 @@ private func playSmokeTest(url: URL, seconds: Double, live: Bool, forceSoftware: presentProbe.attachIfNeeded(engine.currentAVPlayerItem) line += " pres=\(presentProbe.drainTick())" } + if servedURL, let item = engine.currentAVPlayerItem, ObjectIdentifier(item) != servedURLItem, + let asset = item.asset as? AVURLAsset { + servedURLItem = ObjectIdentifier(item) + // The engine's loopback URL stays raw; on the remote route with no stand-in this is the + // origin URL itself, which is redacted like the banner. + let engine = engineEndpoint(mounted: asset.url, source: url) + FileHandle.standardOutput.write(Data(" SERVED \(printableURL(asset.url, engine: engine))\n".utf8)) + } // AE#441: the live rewind surfaces a host actually scales its strip on. Sampling them needed a // patched copy of this CLI before, which is how an over-promising lower bound stayed unseen. if engine.isLive { @@ -990,26 +1041,44 @@ private func playSmokeTest(url: URL, seconds: Double, live: Bool, forceSoftware: if hostCalls.contains("still"), [15, 20, 25].contains(tick) { // The third aim is the live EDGE itself, not the playhead: a target a fraction past the // newest packet is the clamp case, and it is where a live viewer sits most. + // AE#605: a VOD session aims behind the playhead (retained history), ahead of it (the + // forward buffer) and far past what is retained, where the right answer is a MISS: the + // frame there exists, the cache just does not hold it, and the one before it is wrong. + let isLive = engine.isLive let target: Double let label: String - switch tick { - case 15: + switch (tick, isLive) { + case (15, true): target = max(0, engine.currentTime - 20) label = "playhead-20" - case 20: + case (20, true): target = max(0, engine.currentTime - 5) label = "playhead-5" - default: + case (_, true): target = engine.seekableLiveRange?.upperBound ?? engine.currentTime label = "edge" + case (15, false): + target = max(0, engine.currentTime - 10.5) + label = "playhead-10.5" + case (20, false): + target = engine.currentTime + 4.5 + label = "playhead+4.5" + default: + target = min(engine.currentTime + 600, max(0, engine.duration - 1)) + label = "past-frontier" + } + if tick == 15 { + print(" HOSTCALL supportsCacheBackedStills -> \(engine.supportsCacheBackedStills)") } let started = Date() - let image = await engine.liveScrubThumbnail(atSessionSeconds: target, maxWidth: 320) + let image = isLive + ? await engine.liveScrubThumbnail(atSessionSeconds: target, maxWidth: 320) + : await engine.scrubThumbnail(atSeconds: target, maxWidth: 320) let ms = Int(Date().timeIntervalSince(started) * 1000) stillAttempts += 1 if let image { stillHits += 1 - let path = "/tmp/aetherctl-still-\(tick).png" + let path = debugOutputPath("aetherctl-still-\(tick).png") let written = writeStillPNG(image, to: path) print(String(format: " HOSTCALL still(at: %.2f, %@) -> %dx%d in %d ms %@", target, label, image.width, image.height, ms, @@ -1080,20 +1149,53 @@ private func playSmokeTest(url: URL, seconds: Double, live: Bool, forceSoftware: // staged on macOS; its outcome can. if hostCalls.contains("stallclock") { if tick == 4 { + #if DEBUG stallClockStalled = engine.stallRendererClockForTesting() stallClockAtStall = engine.currentTime print(String(format: " HOSTCALL AE#549 stopped the master clock behind the host's back at %.2f%@", engine.currentTime, stallClockStalled ? "" : " -- NO renderer clock on this backend")) + #else + print(" HOSTCALL stallclock needs a DEBUG build (its engine hooks are compiled out of Release): " + + "swift build --product aetherctl") + #endif } if tick == 6 { stallClockBeforeResume = engine.currentTime } if tick == 7 { + #if DEBUG + let stalledRate = engine.rendererClockRateForTesting ?? -1 + #else + let stalledRate: Float = -1 + #endif print(String(format: " HOSTCALL play() on the stalled clock (playhead %.2f, rate %.2f)", - engine.currentTime, engine.rendererClockRateForTesting ?? -1)) + engine.currentTime, stalledRate)) engine.play() } if tick >= 9 { stallClockAtEnd = engine.currentTime } } + // AE#626: force the #93/#65 stage-2 reload on a session the host paused (`pausereload`) or + // left playing (`playreload`), and watch whether the fresh item keeps that transport. + let pauseReload = hostCalls.contains("pausereload") + let externalPlayReload = hostCalls.contains("extplayreload") + if pauseReload || externalPlayReload || hostCalls.contains("playreload") { + if pauseReload || externalPlayReload, tick == 6 { + print(" HOSTCALL pause()") + engine.pause() + } + // The resume AVKit's transport bar or a remote command makes: straight on the player, + // past the engine, so the engine's intent still reads paused while the player runs. + if externalPlayReload, tick == 8 { + print(" HOSTCALL play() straight on the AVPlayer, as AVKit does") + engine.currentAVPlayer?.play() + } + if tick == 10 { + print(String(format: " HOSTCALL AE#626 forcing the stage-2 reload at %.2f (state=%@)", + engine.currentTime, String(describing: engine.state))) + engine.forceStalledConsumerReloadForTesting() + } + if tick == 13 { recoveryReloadClockAfter = engine.currentTime } + if tick >= 13 { recoveryReloadClockAtEnd = engine.currentTime } + } if hostCalls.contains("pausestart"), tick == 8 { print(String(format: " HOSTCALL play() after the start pause (playhead %.2f)", engine.currentTime)) engine.play() @@ -1204,7 +1306,11 @@ private func playSmokeTest(url: URL, seconds: Double, live: Bool, forceSoftware: print(" HOSTCALL setNativeSubtitleRendering(true)") engine.setNativeSubtitleRendering(true) } - if let t = nativeRenderTick, tick == t + 2 { await reportLegibleSelection(engine, "after render on") } + if let t = nativeRenderTick, tick == t + 2 { + await reportLegibleSelection(engine, "after render on") + await reportServedVTT(engine, "after render on", around: engine.currentTime, sessionStart: startPosition ?? 0, + source: url) + } if let subsOffTick, tick == subsOffTick { print(" HOSTCALL subtitles off") engine.clearSubtitle() @@ -1222,7 +1328,11 @@ private func playSmokeTest(url: URL, seconds: Double, live: Bool, forceSoftware: // seconds rather than a runloop turn. Reading at +1 caught the off correctly and reported // every on as a failure, which is the harness lying in the more expensive direction. if let t = subsOffTick, tick == t + 1 { await reportLegibleSelection(engine, "after off") } - if let t = subsOnTick, tick == t + 4 { await reportLegibleSelection(engine, "after on") } + if let t = subsOnTick, tick == t + 4 { + await reportLegibleSelection(engine, "after on") + await reportServedVTT(engine, "after on", around: engine.currentTime, sessionStart: startPosition ?? 0, + source: url) + } } let finalTime = engine.currentTime @@ -1231,6 +1341,16 @@ private func playSmokeTest(url: URL, seconds: Double, live: Bool, forceSoftware: // one when the question is what a host's picker ends up showing. let finalSubtitleTracks = engine.subtitleTracks let finalActiveSubtitle = engine.activeSubtitleTrackIndex + // The settled stats-panel identity. The SOURCE line at load prints before the remote-HLS bypass has + // read anything back from AVPlayer's item, so it reads empty there by construction. + let settledSource = "SOURCE codec=\(engine.sourceVideoCodecName ?? "nil") " + + "\(engine.sourceVideoWidth)x\(engine.sourceVideoHeight) " + + "fmt=\(engine.sourceVideoFormat) stream=\(engine.sourceVideoStreamFormat.map { String(describing: $0) } ?? "nil")" + let settledAudio = engine.audioTracks.map { + "#\($0.id) \($0.name) codec=\($0.codec) ch=\($0.channels) sr=\($0.sampleRate)" + + "\($0.profile.map { " profile=\($0)" } ?? "")\($0.isAtmos ? " atmos" : "")" + }.joined(separator: ", ") + let settledActiveAudio = engine.activeAudioTrackIndex if record != nil { await engine.stopRecording() print(" RECORD final state: \(engine.recordingState)") @@ -1273,6 +1393,8 @@ private func playSmokeTest(url: URL, seconds: Double, live: Bool, forceSoftware: print("subtitle tracks (* = external): \(listed)") print("active subtitle: \(finalActiveSubtitle.map(String.init) ?? "none")") } + print("settled \(settledSource)") + print("audio tracks: \(settledAudio.isEmpty ? "none" : settledAudio) active=\(settledActiveAudio.map(String.init) ?? "none")") print("final t=\(String(format: "%.2f", finalTime))s state=\(String(describing: endState)) cues=\(cueCount)") let closingWindow = await MainActor.run { lastCues } print("WINDOW \(closingWindow.count) cues in the last published window") @@ -1312,6 +1434,24 @@ private func playSmokeTest(url: URL, seconds: Double, live: Bool, forceSoftware: return 4 } } + if hostCalls.contains("pausereload") || hostCalls.contains("playreload") + || hostCalls.contains("extplayreload") { + guard let after = recoveryReloadClockAfter, let atEnd = recoveryReloadClockAtEnd else { + print("VERDICT: AE#626 drill inconclusive (run it for at least 20 s)") + return 5 + } + let moved = atEnd - after + print(String(format: "AE#626 after the reload: %.2f, at the last tick %.2f (%+.2f s), state=%@", + after, atEnd, moved, String(describing: endState))) + if hostCalls.contains("pausereload"), moved > 0.25 { + print("VERDICT: AE#626 reproduced (the reload of a paused session played it)") + return 4 + } + if hostCalls.contains("playreload") || hostCalls.contains("extplayreload"), moved <= 0.25 { + print("VERDICT: AE#626 regression (the reload of a playing session came back stopped)") + return 4 + } + } if hostCalls.contains("stallclock") { guard stallClockStalled, let atStall = stallClockAtStall, let beforeResume = stallClockBeforeResume, let atEnd = stallClockAtEnd else { @@ -1432,3 +1572,153 @@ private func reportLegibleSelection(_ engine: AetherEngine, _ label: String) asy print(" LEGIBLE \(label): selected=\(name) options=\(group.options.count) " + "engineActive=\(engine.activeSubtitleTrackIndex.map(String.init) ?? "nil")") } + +/// Sodalite#156 round 2: what the RECEIVER would actually get, which is not the same question as +/// whether a selection is held. +/// +/// The first version of this harness asked `currentMediaSelection` and called a run green when it +/// read back the language the host picked. On the device the same transition produced a caption box +/// with no text in it, so the selection was held and the payload behind it was empty. AVKit fetches +/// the whole forward window in one burst at selection and never re-fetches a segment it already has, +/// which makes the payload at that instant the only thing that matters. +/// +/// So this fetches the served WebVTT the way a receiver does: master -> the selected rendition's +/// media playlist -> the segments covering the playhead, and counts CUE TEXT, not segments. +private enum Self_VTT {} +extension AetherEngine { + /// `HH:MM:SS.mmm` or `MM:SS.mmm` as seconds. + static func vttSeconds(_ s: String) -> Double? { + let parts = s.split(separator: ":").map(String.init) + guard !parts.isEmpty, let last = Double(parts.last!) else { return nil } + var total = last + if parts.count >= 2 { total += (Double(parts[parts.count - 2]) ?? 0) * 60 } + if parts.count >= 3 { total += (Double(parts[parts.count - 3]) ?? 0) * 3600 } + return total + } +} + +/// The engine server behind a mounted item, or nil. An item's URL is either the source itself (the +/// remote route with no stand-in) or the engine's loopback stand-in, so a loopback URL that is not +/// the source is the engine's. +private func engineEndpoint(mounted: URL, source: URL) -> URL? { + let host = mounted.host?.lowercased() + guard mounted != source, host == "127.0.0.1" || host == "localhost" || host == "::1" else { return nil } + return mounted +} + +/// `url` for stdout. A URL on the engine's own server prints raw, because its path token is the only +/// way in and this is a local harness; anything else, an origin on localhost included, can carry a +/// credential, so it is redacted the way the banner is. +private func printableURL(_ url: URL, engine: URL?) -> String { + let onEngine = engine.map { url.scheme == $0.scheme && url.host == $0.host && url.port == $0.port } ?? false + return onEngine ? url.absoluteString : EngineLog.redacted(url.absoluteString) +} + +@MainActor +private func reportServedVTT(_ engine: AetherEngine, _ label: String, around playhead: Double, + sessionStart: Double, source: URL) async { + guard let item = engine.currentAVPlayerItem, + let asset = item.asset as? AVURLAsset else { + print(" VTT \(label): no item") + return + } + let master = asset.url + let engine = engineEndpoint(mounted: master, source: source) + func get(_ url: URL) async -> String? { + guard let (data, _) = try? await URLSession.shared.data(from: url) else { return nil } + return String(data: data, encoding: .utf8) + } + guard let masterBody = await get(master) else { + print(" VTT \(label): master unreachable at \(printableURL(master, engine: engine))") + return + } + // The rendition AVPlayer is on, named by the legible selection so the harness follows the same + // pick the receiver does rather than guessing an ordinal. + var wantedName: String? + if let group = try? await item.asset.loadMediaSelectionGroup(for: .legible) { + wantedName = item.currentMediaSelection.selectedMediaOption(in: group)?.displayName + } + let renditions: [(name: String, uri: String)] = masterBody + .split(separator: "\n") + .filter { $0.hasPrefix("#EXT-X-MEDIA:TYPE=SUBTITLES") } + .compactMap { line in + func field(_ key: String) -> String? { + guard let r = line.range(of: "\(key)=\"") else { return nil } + let rest = line[r.upperBound...] + guard let end = rest.firstIndex(of: "\"") else { return nil } + return String(rest[.." counts cues without parsing. + let c = body.components(separatedBy: "-->").count - 1 + cues += c + if c > 0 { nonEmpty += 1 } + // Sodalite#156: a POPULATED segment whose cues sit outside the window the playlist declares for + // it is as invisible as an empty one, and it reads identically in every count above. The device + // capture ruled the empty case out (nothing served empty, 22 segments fetched, still a blank + // box), and irregular keyframes make segments as much as 7.5 s long against a nominal 4, so the + // declared window and the cue times are exactly the pair worth comparing. + let end = offsets.first(where: { $0.start > start })?.start ?? (start + 4) + for line in body.split(separator: "\n") where line.contains("-->") { + let parts = line.split(separator: " ") + guard let t = parts.first.map(String.init), let cueStart = AetherEngine.vttSeconds(t) else { continue } + if cueStart < start - 1 || cueStart > end + 1 { + misplaced += 1 + if firstMismatch == nil { + firstMismatch = String(format: "%@ declares [%.1f,%.1f) but carries a cue at %.1f", + name, start, end, cueStart) + } + } + break + } + } + print(" VTT \(label): rendition=\(pick.name) segments=\(checked) nonEmpty=\(nonEmpty) cues=\(cues)" + + (misplaced > 0 ? " MISPLACED=\(misplaced) \(firstMismatch ?? "")" : "") + + " picked=\(offsets[startIndex...].prefix(2).map(\.name).joined(separator: ",")) " + + "of=\(offsets.count) relative=\(String(format: "%.1f", relative)) playhead=\(String(format: "%.1f", playhead))" + + (checked > 0 && nonEmpty == 0 ? " <- a caption box with nothing in it" : "")) +} diff --git a/Sources/aetherctl/Probe.swift b/Sources/aetherctl/Probe.swift index ab9be45ce..079a3a347 100644 --- a/Sources/aetherctl/Probe.swift +++ b/Sources/aetherctl/Probe.swift @@ -5,10 +5,11 @@ import AetherEngine func runProbe(url: URL, detecting: ProbeDetail = []) -> Int32 { EngineLog.handler = { print($0) } - print("aetherctl probe: \(url.absoluteString)") + print(EngineLog.redacted("aetherctl probe: \(url.absoluteString)")) if !detecting.isEmpty { var passes: [String] = [] if detecting.contains(.hdr10Plus) { passes.append("hdr10plus (packet scan)") } + if detecting.contains(.hdrVivid) { passes.append("hdr-vivid (packet scan)") } if detecting.contains(.atmos) { passes.append("atmos (bounded decode)") } print("detail passes: \(passes.joined(separator: ", "))") } @@ -29,6 +30,11 @@ func runProbe(url: URL, detecting: ProbeDetail = []) -> Int32 { print("Duration: \(duration)s") print("Video: codec=\(codec) resolution=\(res) fps=\(rate)") print(" format: \(probe.videoFormat)") + if let f = probe.videoStreamFormat { + print(" pixels: \(f.pixelFormat ?? "-") depth=\(f.bitDepth.map { "\($0)-bit" } ?? "-") profile=\(f.profile ?? "-")") + print(" colour: primaries=\(f.colorPrimariesLabel ?? "-") transfer=\(f.transferLabel ?? "-") " + + "matrix=\(f.matrixLabel ?? "-") range=\(f.rangeLabel ?? "-")") + } if probe.isDolbyVision { print(" HDR/DV: Dolby Vision signaled") } @@ -36,6 +42,9 @@ func runProbe(url: URL, detecting: ProbeDetail = []) -> Int32 { // A negative here means "not seen inside the scan budget", never "proven absent". print(" HDR10+: \(probe.carriesHDR10PlusMetadata ? "ST 2094-40 metadata seen" : "not seen")") } + if detecting.contains(.hdrVivid) { + print(" HDR Vivid: \(probe.carriesHDRVividMetadata ? "CUVA metadata seen" : "not seen")") + } print("") if probe.audioTracks.isEmpty { @@ -47,6 +56,8 @@ func runProbe(url: URL, detecting: ProbeDetail = []) -> Int32 { let atmos = track.isAtmos ? " [Atmos]" : "" let def = track.isDefault ? " (default)" : "" print(" [\(track.id)] codec=\(track.codec) channels=\(track.channels) lang=\(lang)\(atmos)\(def)") + print(" rate=\(track.sampleRate) Hz bits=\(track.bitsPerSample) fmt=\(track.sampleFormat ?? "-") " + + "layout=\(track.channelLayout ?? "-") profile=\(track.profile ?? "-")") print(" title=\(track.name)") } } diff --git a/Sources/aetherctl/SMBTestCommand.swift b/Sources/aetherctl/SMBTestCommand.swift index 6ce49b6fb..1d8d8b748 100644 --- a/Sources/aetherctl/SMBTestCommand.swift +++ b/Sources/aetherctl/SMBTestCommand.swift @@ -23,6 +23,26 @@ private func smbTestRun(_ args: [String]) async -> Int32 { let total = reader.seek(offset: 0, whence: 65536) // AVSEEK_SIZE print("connected: \(u.path) size=\(total) bytes") + // The random-seek offsets are drawn first so the sequential pass can record the bytes that + // belong at each of them: the seeked reads are then compared against what the file holds, not + // against a second read of the same path (audit OPS-105). + struct Probe { let offset: Int64; let length: Int; var reference: [UInt8] } + var rng = SystemRandomNumberGenerator() + var probes: [Probe] = [] + if total > 0 { + for _ in 0.. Int32 { while true { let n = buf.withUnsafeMutableBufferPointer { reader.read($0.baseAddress, size: Int32(chunk)) } if n <= 0 { break } - readBytes += Int64(n) + let chunkEnd = readBytes + Int64(n) + // Skip probes that ended before this chunk; one that crosses into it stays for its tail. + while cursor < byOffset.count { + let probe = probes[byOffset[cursor]] + guard probe.offset + Int64(probe.length) <= readBytes else { break } + cursor += 1 + } + for i in byOffset[cursor...] { + if probes[i].offset >= chunkEnd { break } + let lo = max(probes[i].offset, readBytes) + let hi = min(probes[i].offset + Int64(probes[i].length), chunkEnd) + guard lo < hi else { continue } + for position in lo.. 0 ? Double(readBytes) / 1_048_576.0 / seqElapsed : 0 @@ -42,18 +78,23 @@ private func smbTestRun(_ args: [String]) async -> Int32 { return 1 } - // Random-seek correctness: two reads at each random offset must agree (deterministic content). - var rng = SystemRandomNumberGenerator() - for _ in 0.. Int32 { } } +/// Reads until `length` bytes arrived, so a legitimate short read is not taken for a failure; a read that +/// returns 0 (EOF) or -1 (error) ends it short, and the caller sees the shortfall. private func smbReadAt(_ reader: SMBIOReader, _ offset: Int64, _ length: Int) -> Data { _ = reader.seek(offset: offset, whence: Int32(SEEK_SET)) + var out = Data() var buf = [UInt8](repeating: 0, count: length) - let n = buf.withUnsafeMutableBufferPointer { reader.read($0.baseAddress, size: Int32(length)) } - return Data(buf.prefix(Int(max(n, 0)))) + while out.count < length { + let n = buf.withUnsafeMutableBufferPointer { reader.read($0.baseAddress, size: Int32(length - out.count)) } + if n <= 0 { break } + out.append(contentsOf: buf.prefix(Int(n))) + } + return out } private func smbParseIntFlag(_ args: [String], _ flag: String) -> Int? { diff --git a/Sources/aetherctl/SWDecode.swift b/Sources/aetherctl/SWDecode.swift index 32a20324e..0a0195b43 100644 --- a/Sources/aetherctl/SWDecode.swift +++ b/Sources/aetherctl/SWDecode.swift @@ -5,7 +5,7 @@ import AetherEngine func runSWDecode(url: URL, maxPackets: Int) -> Int32 { EngineLog.handler = { print($0) } - print("aetherctl swdecode: \(url.absoluteString) (maxPackets=\(maxPackets))") + print(EngineLog.redacted("aetherctl swdecode: \(url.absoluteString) (maxPackets=\(maxPackets))")) print("") let result: SoftwareDecodeProbeResult @@ -30,6 +30,9 @@ func runSWDecode(url: URL, maxPackets: Int) -> Int32 { if let fmt = result.firstFramePixelFormat { print("First frame pixfmt: \(fmt)") print("First frame size: \(result.firstFrameWidth)x\(result.firstFrameHeight)") + if let color = result.firstFrameColor { + print("First frame color: \(color)") + } } else { print("First frame: (none decoded)") } diff --git a/Sources/aetherctl/SeekTest.swift b/Sources/aetherctl/SeekTest.swift index 5c71b747a..a61a33cc6 100644 --- a/Sources/aetherctl/SeekTest.swift +++ b/Sources/aetherctl/SeekTest.swift @@ -21,7 +21,8 @@ private func seekTestRun(url: URL, seeks: Int, gapMs: Int, settleSeconds: Double let parkCount = UncheckedBox(0) // AE#528: a park whose PARK line says stuck=0s is backpressure behind a consumer that is still // fetching (a viewer scrubbing through resident content), and the breaker deliberately does not - // fire there. Counted apart so the verdict cannot read a healthy park as an unrecovered wedge. + // fire there. AE#649 widened it to idle=0s, which also covers a consumer that is quiet but still + // rendering. Counted apart so the verdict cannot read a healthy park as an unrecovered wedge. let liveConsumerParkCount = UncheckedBox(0) // #65 fix signals: did the VOD wedge breaker fire and recover (Piece A producer re-anchor + Piece B // engine seek-deadline clock reconcile)? A wedge that is BROKEN + re-anchored is the fix engaging. @@ -73,7 +74,7 @@ private func seekTestRun(url: URL, seeks: Int, gapMs: Int, settleSeconds: Double // "[HLSSegmentProducer] #65 backpressure PARK ...". Count abnormal parks (VOD wedge signature). if line.contains("#65 backpressure PARK") { parkCount.value += 1 - if line.contains("stuck=0s") { liveConsumerParkCount.value += 1 } + if line.contains("idle=0s") { liveConsumerParkCount.value += 1 } } // Fix engaging: the wedge breaker exited the pump, the host re-anchored, and/or the seek deadline reconciled. if line.contains("#65 backpressure WEDGE BROKEN") { wedgeBrokenCount.value += 1 } @@ -83,7 +84,7 @@ private func seekTestRun(url: URL, seeks: Int, gapMs: Int, settleSeconds: Double print("") print("=== SEEKTEST (issue #35 rapid-seek burst) ===") - print(" url=\(url.absoluteString) seeks=\(seeks) gapMs=\(gapMs) settle=\(settleSeconds)s") + print(EngineLog.redacted(" url=\(url.absoluteString) seeks=\(seeks) gapMs=\(gapMs) settle=\(settleSeconds)s")) let engine: AetherEngine do { @@ -162,13 +163,17 @@ private func seekTestRun(url: URL, seeks: Int, gapMs: Int, settleSeconds: Double } let sawSeeking = probes.contains { $0.seeking } let endedCleared = !(probes.last?.seeking ?? true) + // A line printed "<-- FAIL" has to reach the exit code too: nothing else a script can gate on (audit OPS-104). + var failures: [String] = [] print("") print("=== #37/#38 PROBE (single backward seek, concurrent sampler) ===") print(String(format: " preSeek=%.1f target=%.1f tol=%.1f samples=%d", preSeekCt, probeLo, tol, probes.count)) print(" #37 clock bounce back through pre-seek after reaching target: " + (bounceAfterTarget ? "YES <-- FAIL" : "no <-- PASS")) + if bounceAfterTarget { failures.append("#37 clock bounce") } print(" #38 isSeeking observed in-flight=\(sawSeeking ? "yes" : "NO") ended-cleared=\(endedCleared ? "yes" : "NO") " + ((sawSeeking && endedCleared) ? "<-- PASS" : "<-- FAIL")) + if !(sawSeeking && endedCleared) { failures.append("#38 isSeeking window") } // #38 follow-up: the falling edge alone cannot say what happened; the probe seek must produce a // .began and a matching .landed, and that landing must name a position at the target. let probeSeekEvents = seekEvents.value.filter { $0.origin == .programmatic } @@ -183,6 +188,7 @@ private func seekTestRun(url: URL, seeks: Int, gapMs: Int, settleSeconds: Double print(" #38 event pair: \(probeBegan.map { "began@\(String(format: "%.1f", $0.target))" } ?? "NONE") -> " + (probeTerminator.map { "\($0)" } ?? "NO TERMINATOR") + " " + (probeLandedAtTarget ? "<-- PASS" : "<-- FAIL")) + if !probeLandedAtTarget { failures.append("#38 seek event pair") } struct Sample { let wall: Double; let ct: Double; let src: Double; let playing: Bool } var samples: [Sample] = [] @@ -259,10 +265,23 @@ private func seekTestRun(url: URL, seeks: Int, gapMs: Int, settleSeconds: Double let finalCt = samples.last?.ct ?? engine.currentTime let settleError = abs(finalCt - finalTarget) - let fullRestart = tally.value["fullRestart"] ?? 0 - let coalesced = tally.value["coalesced"] ?? 0 - let settleAdvance = tally.value["settleAdvance"] ?? 0 - let abandon = tally.value["abandon"] ?? 0 + // The log handler keeps appending from engine threads until `engine.stop()` below, and it mutates + // these under `handlerLock`: read them once under the same lock and report from the copy (audit OPS-104). + let counters: (tally: [String: Int], publishedShifts: [Double], gateOpenShifts: [Int], + ledgerCount: Int, maxLedgerDriftAbs: Double, parkCount: Int, + liveConsumerParkCount: Int, wedgeBrokenCount: Int, reanchorCount: Int, + seekReconcileCount: Int) = { + handlerLock.lock() + defer { handlerLock.unlock() } + return (tally.value, publishedShifts.value, gateOpenShifts.value, ledgerCount.value, + maxLedgerDriftAbs.value, parkCount.value, liveConsumerParkCount.value, + wedgeBrokenCount.value, reanchorCount.value, seekReconcileCount.value) + }() + + let fullRestart = counters.tally["fullRestart"] ?? 0 + let coalesced = counters.tally["coalesced"] ?? 0 + let settleAdvance = counters.tally["settleAdvance"] ?? 0 + let abandon = counters.tally["abandon"] ?? 0 print("") print("=== SEEKTEST RESULTS ===") @@ -284,8 +303,8 @@ private func seekTestRun(url: URL, seeks: Int, gapMs: Int, settleSeconds: Double print(" full restarts for the same burst; 'abandoned' should trend to 0.") // #65 discriminator: did the producer shift VARY across the burst's restart epochs? - let pubShifts = publishedShifts.value - let rawShifts = gateOpenShifts.value + let pubShifts = counters.publishedShifts + let rawShifts = counters.gateOpenShifts let distinctPub = Set(pubShifts.map { ($0 * 1000).rounded() / 1000 }) let distinctRaw = Set(rawShifts) print("") @@ -295,11 +314,11 @@ private func seekTestRun(url: URL, seeks: Int, gapMs: Int, settleSeconds: Double "[" + pubShifts.map { String(format: "%.3f", $0) }.joined(separator: ", ") + "]", distinctPub.count)) print(" clockLead settle = \(String(format: "%.2f", settleClockLead))s (headless ~0 by design; #65 is presented-vs-clock, invisible to ct-src)") - print(" ledger segments opened = \(ledgerCount.value) maxContentDrift = \(String(format: "%.3f", maxLedgerDriftAbs.value))s (the POSITIVE Root-B signal)") - print(" abnormal backpressure parks (VOD wedge signature) = \(parkCount.value)" - + " (of those, \(liveConsumerParkCount.value) with the consumer still fetching, AE#528)") - print(" #65 FIX signals: wedge breaks=\(wedgeBrokenCount.value) producer re-anchors=\(reanchorCount.value) seek-deadline reconciles=\(seekReconcileCount.value)") - let fixEngaged = wedgeBrokenCount.value > 0 || reanchorCount.value > 0 || seekReconcileCount.value > 0 + print(" ledger segments opened = \(counters.ledgerCount) maxContentDrift = \(String(format: "%.3f", counters.maxLedgerDriftAbs))s (the POSITIVE Root-B signal)") + print(" abnormal backpressure parks (VOD wedge signature) = \(counters.parkCount)" + + " (of those, \(counters.liveConsumerParkCount) with the consumer still fetching, AE#528)") + print(" #65 FIX signals: wedge breaks=\(counters.wedgeBrokenCount) producer re-anchors=\(counters.reanchorCount) seek-deadline reconciles=\(counters.seekReconcileCount)") + let fixEngaged = counters.wedgeBrokenCount > 0 || counters.reanchorCount > 0 || counters.seekReconcileCount > 0 if fixEngaged { print(" >> #65 FIX ENGAGED: the VOD wedge breaker / seek-deadline reconcile fired during the burst. A") print(" non-zero break+re-anchor with playback resuming afterward (clock advancing, no sustained PARK") @@ -309,28 +328,29 @@ private func seekTestRun(url: URL, seeks: Int, gapMs: Int, settleSeconds: Double print(" >> INCONCLUSIVE: 0 producer restarts. The file was fully produced before the burst, so") print(" every seek hit the cache and the cross-epoch cascade never fired. Re-run with a LONGER") print(" file (seek targets must land beyond the producer write head) to exercise #65.") - } else if maxLedgerDriftAbs.value >= 0.5 { + } else if counters.maxLedgerDriftAbs >= 0.5 { print(" >> ROOT B CONFIRMED (content-vs-clock): a segment was muxed with source content offset") - print(" \(String(format: "%.2f", maxLedgerDriftAbs.value))s from its planned/EXTINF position, so the presented frame leads the") + print(" \(String(format: "%.2f", counters.maxLedgerDriftAbs))s from its planned/EXTINF position, so the presented frame leads the") print(" clock by that much. Fix the content-drift source (off-plan seek landing / uniform-plan stride),") print(" NOT a seam port. Grep '#65 ledger' for the exact seg/epoch.") } else if distinctRaw.count > 1 || distinctPub.count > 1 { print(" >> ROOT A (cross-epoch shift divergence): the producer published MORE THAN ONE shift across") print(" the burst. Buffered bytes from a superseded epoch fold with the latest scalar -> picture") print(" leads the clock. The live seam-history port is the fix.") - } else if parkCount.value > 0, parkCount.value == liveConsumerParkCount.value, !fixEngaged { - print(" >> PARKED BEHIND A LIVE CONSUMER (not a wedge): \(parkCount.value) park(s), every one of them") - print(" logged stuck=0s, so the consumer kept declaring new fetch targets the whole time. The") - print(" breaker staying quiet here is AE#528 working; a wedge is a park whose stuck= climbs.") - } else if parkCount.value > 0 { + } else if counters.parkCount > 0, counters.parkCount == counters.liveConsumerParkCount, !fixEngaged { + print(" >> PARKED BEHIND A LIVE CONSUMER (not a wedge): \(counters.parkCount) park(s), every one of them") + print(" logged idle=0s, so the consumer kept declaring fetch targets or kept rendering the whole") + print(" time. The breaker staying quiet here is AE#528 working; a wedge is a park whose idle= climbs.") + } else if counters.parkCount > 0 { if fixEngaged { - print(" >> PRODUCER WEDGE DETECTED AND BROKEN: \(parkCount.value) abnormal park(s) but the breaker fired") - print(" (\(wedgeBrokenCount.value) break(s), \(reanchorCount.value) re-anchor(s), \(seekReconcileCount.value) seek reconcile(s)).") + print(" >> PRODUCER WEDGE DETECTED AND BROKEN: \(counters.parkCount) abnormal park(s) but the breaker fired") + print(" (\(counters.wedgeBrokenCount) break(s), \(counters.reanchorCount) re-anchor(s), \(counters.seekReconcileCount) seek reconcile(s)).") print(" This is the #65 fix recovering the livelock; verify on device that playback actually resumes.") } else { - print(" >> PRODUCER WEDGE (UNRECOVERED): invariant shift AND ~zero ledger drift, but \(parkCount.value) abnormal") + print(" >> PRODUCER WEDGE (UNRECOVERED): invariant shift AND ~zero ledger drift, but \(counters.parkCount) abnormal") print(" backpressure park(s) and the breaker did NOT fire. The 6s symptom is the frozen-clock/stall") print(" artifact. Either the park stayed under the break threshold or the fix is not in this build.") + failures.append("producer wedge unrecovered") } } else { print(" >> NO ENGINE-LEVEL DIVERGENCE REPRODUCED: invariant shift, ledger drift ~0, no wedge. The headless") @@ -358,6 +378,7 @@ private func seekTestRun(url: URL, seeks: Int, gapMs: Int, settleSeconds: Double + "late-landings=\(lateLandings.count)") let unpaired = begun.subtracting(terminated).sorted() print(" unpaired begans: " + (unpaired.isEmpty ? "none <-- PASS" : "\(unpaired) <-- FAIL")) + if !unpaired.isEmpty { failures.append("unpaired seek begans \(unpaired)") } // AE#534: began -> first terminal, per seek, in issue order. Quote the median and the MAX: the // read this measures is an XPC round trip to a media server that is least likely to answer during // a seek, so a mean would bury exactly the case the question is about. @@ -378,6 +399,10 @@ private func seekTestRun(url: URL, seeks: Int, gapMs: Int, settleSeconds: Double } for event in events.suffix(12) { print(" \(event)") } print("") + if !failures.isEmpty { + print("VERDICT: seektest FAIL: " + failures.joined(separator: "; ")) + return 1 + } print("VERDICT: seektest DONE (comparison harness; compare tallies old vs new build)") return 0 } diff --git a/Sources/aetherctl/SegVerify.swift b/Sources/aetherctl/SegVerify.swift index 94e6d9af5..67646af9b 100644 --- a/Sources/aetherctl/SegVerify.swift +++ b/Sources/aetherctl/SegVerify.swift @@ -14,7 +14,7 @@ func runSegVerify(url: URL, from: Int, count: Int, dvModeAvailable: Bool, dolbyVisionHandling: DolbyVisionHandling = .automatic, dumpDir: String? = nil) -> Int32 { setvbuf(stdout, nil, _IONBF, 0) // unbuffered: progressive output survives a long-running run - print("segverify: starting engine for \(url.absoluteString)") + print(EngineLog.redacted("segverify: starting engine for \(url.absoluteString)")) let engine = HLSVideoEngine(url: url, dvModeAvailable: dvModeAvailable, forceDolbyVisionOnNonDVDisplay: forceDVWithoutDisplay, dolbyVisionHandling: dolbyVisionHandling, diff --git a/Sources/aetherctl/Serve.swift b/Sources/aetherctl/Serve.swift index 9e6bc72ae..0c92881b3 100644 --- a/Sources/aetherctl/Serve.swift +++ b/Sources/aetherctl/Serve.swift @@ -7,7 +7,7 @@ func runServe(url: URL, dvModeAvailable: Bool, forceDVWithoutDisplay: Bool = fal dolbyVisionHandling: DolbyVisionHandling = .automatic, nativeSubsIndex: Int? = nil, startPosition: Double? = nil, audioDelayMs: Int = 0, objectAudioRendering: ObjectAudioRendering = .off, - audioStreamIndex: Int32? = nil) -> Never { + audioStreamIndex: Int32? = nil, iFrames: Bool = false) -> Never { EngineLog.handler = { line in let timestamp = ISO8601DateFormatter.string( from: Date(), @@ -25,7 +25,8 @@ func runServe(url: URL, dvModeAvailable: Bool, forceDVWithoutDisplay: Bool = fal if audioDelayMs != 0 { flagSuffix += " [--audio-delay \(audioDelayMs)]" } if case .apac(let layout) = objectAudioRendering { flagSuffix += " [--atmos-bed \(layout.rawValue)]" } if let audioStreamIndex { flagSuffix += " [--audio-index \(audioStreamIndex)]" } - print("aetherctl serve: \(url.absoluteString)\(flagSuffix)") + if iFrames { flagSuffix += " [--iframes]" } + print(EngineLog.redacted("aetherctl serve: \(url.absoluteString)\(flagSuffix)")) print("") let engine = HLSVideoEngine( @@ -48,6 +49,10 @@ func runServe(url: URL, dvModeAvailable: Bool, forceDVWithoutDisplay: Bool = fal if nativeSubsIndex != nil { engine.requestNativeSubtitleTrack() } + // AE#682: the LoadOptions.serveIFramePlaylist path. Must precede start(). + if iFrames { + engine.requestIFramePlaylist() + } let playbackURL: URL do { playbackURL = try engine.start() @@ -102,7 +107,7 @@ func runValidate(url: URL, dvModeAvailable: Bool, forceDVWithoutDisplay: Bool = var flagSuffix = dvModeAvailable ? "" : " [--no-dv]" if forceDVWithoutDisplay { flagSuffix += " [--force-dv]" } if dolbyVisionHandling == .baseLayerOnly { flagSuffix += " [--dv-base-layer]" } - print("aetherctl validate: \(url.absoluteString)\(flagSuffix)") + print(EngineLog.redacted("aetherctl validate: \(url.absoluteString)\(flagSuffix)")) print("") let engine = HLSVideoEngine( diff --git a/Sources/aetherctl/main.swift b/Sources/aetherctl/main.swift index 4964998b8..dcdadfa9d 100644 --- a/Sources/aetherctl/main.swift +++ b/Sources/aetherctl/main.swift @@ -66,11 +66,11 @@ func printUsage() { aetherctl: standalone AetherEngine repro harness Usage: - aetherctl probe [--detect-hdr10plus] [--detect-atmos] - aetherctl serve [--no-dv] [--force-dv] [--dv-base-layer] [--start-position S] [--atmos-bed 7.1.4] + aetherctl probe [--detect-hdr10plus] [--detect-hdr-vivid] [--detect-atmos] + aetherctl serve [--no-dv] [--force-dv] [--dv-base-layer] [--start-position S] [--iframes] [--atmos-bed 7.1.4] aetherctl validate [--no-dv] [--force-dv] [--dv-base-layer] aetherctl swdecode [--frames N] - aetherctl play [--seconds N] [--live] [--fast-zap] [--live-start-immediately] [--dvr-window N] [--subs ] + aetherctl play [--seconds N] [--live] [--no-sw-escalation] [--fast-zap] [--live-start-immediately] [--dvr-window N] [--subs ] [--assert-dv] [--dv-base-layer] [--start-position S] [--switch-audio [@ms]] [--teletext-page N] [--switch-teletext-page [@ms]] @@ -79,7 +79,7 @@ func printUsage() { [--drop-audio] [--sequential-origin] [--declared-duration S] [--max-concurrent-requests N] - [--audio-stats] [--host-calls play,extractor,setrate,pausestart,reloadlive,seekback,seekfar,pauseseek] + [--audio-stats] [--host-calls play,extractor,setrate,pausestart,reloadlive,seekback,seekfar,pauseseek,pausereload,playreload,extplayreload,reloadnext] (full load+play session smoke test; --subs activates the first matching embedded subtitle track and logs overlay cues; --audio-stats taps decoded PCM and prints per-second audio lead @@ -106,7 +106,7 @@ func printUsage() { aetherctl segverify [--from N] [--count K] [--no-dv] [--force-dv] [--dv-base-layer] [--dump ] (#92: SW-decode each segment in isolation; framesDecoded==0 => not independent) aetherctl disc-inspect - aetherctl dovitest + aetherctl dovitest [--out PATH.hevc] aetherctl extract [--at ] [--snapshot] [--width ] [--loops ] aetherctl audio [--seconds N] aetherctl audiotap [--duration S] [--out PATH.wav] [--remote | --software] @@ -118,8 +118,6 @@ func printUsage() { never EOF, unknown size; prints physFP and its slope against that rate) aetherctl live [--seconds N] [--seed ] [--dvr-window N] [--serve-only] [--measure-rss] [--report-cache-bytes] [--rewind-test] [--reload-test] [--sw] [--drop-after N] [--discontinuity-at N] [--realtime] [--realtime-rate X] [--fast-zap] [--preroll N] [--rewind-hold N] [--gen-highbitrate-seed] [--freeze-after N] [--unfreeze-after N] [--rewind-before-freeze N] [--force-recovery-reload-at N] [--live-only] [--no-blocking-reload] [--force-master] - [--freeze-after N] [--unfreeze-after N] [--rewind-before-freeze N] [--force-recovery-reload-at N] - [--no-blocking-reload] aetherctl dvr [--path native|sw|both] [--seconds N] [--dvr-window N] aetherctl dualsubs --primary --secondary [--seek ] aetherctl hlsfixture [--port N] [--segment-seconds N] [--target-duration N] [--window N] @@ -216,13 +214,16 @@ func printUsage() { mediastreamvalidator / mp4dump / ffprobe from another terminal: - curl -i http://127.0.0.1:/master.m3u8 - curl -o /tmp/init.mp4 http://127.0.0.1:/init.mp4 - curl -o /tmp/seg0.mp4 http://127.0.0.1:/seg0.mp4 - mediastreamvalidator http://127.0.0.1:/master.m3u8 + curl -i http://127.0.0.1://master.m3u8 + curl -o /tmp/init.mp4 http://127.0.0.1://init.mp4 + curl -o /tmp/seg0.mp4 http://127.0.0.1://seg0.mp4 + mediastreamvalidator http://127.0.0.1://master.m3u8 mp4dump --verbosity 1 /tmp/init.mp4 ffprobe -v debug /tmp/seg0.mp4 - open 'http://127.0.0.1:/master.m3u8' + open 'http://127.0.0.1://master.m3u8' + + The server answers only paths that start with its + per-session token; copy / from the printed URL. Ctrl-C to tear down. @@ -234,7 +235,9 @@ func printUsage() { packet's Dolby Vision RPU from Profile 7 to Profile 8.1 (and drop the enhancement layer) via DoviRpuConverter, and write the result to - /tmp/aetherctl-dovitest.hevc in Annex-B form. Feed + aetherctl-dovitest.hevc (Annex-B) in a private + per-user temporary directory (printed, reused and + overwritten by the next run), or to --out PATH. Feed that to `dovi_tool extract-rpu` + `info` to validate the rewritten RPU against ground truth. @@ -257,7 +260,8 @@ func printUsage() { to --width. Snapshot mode (--snapshot) decodes frame-accurately at full resolution. Use --loops N with `leaks --atExit` to detect memory leaks. - Writes the first frame to /tmp/aetherctl-extract-.png. + Writes the first frame to aetherctl-extract-.png + in a private per-user temporary directory, printed. audio Load a source through the engine's audio-only path (LoadOptions.audioOnly=true), play for ~10 seconds, @@ -320,7 +324,7 @@ if first == "--help" || first == "-h" || first == "help" { if first == "dvr" { var rest = Array(args.dropFirst(2)) let path = takeStringFlag("--path", from: &rest) ?? "both" - let seconds = takeDoubleFlag("--seconds", from: &rest) ?? 120.0 + let seconds = takeDoubleFlag("--seconds", in: 0...maxRunSeconds, from: &rest) ?? 120.0 let dvrWin = takeDoubleFlag("--dvr-window", from: &rest) ?? 60.0 guard ["native", "sw", "both"].contains(path) else { print("ERROR: --path must be native, sw, or both (got '\(path)')") @@ -333,8 +337,8 @@ if first == "dvr" { // #92 verifier: SW-decode each segment in isolation; framesDecoded==0 => not independently decodable. if first == "segverify" { var rest = Array(args.dropFirst(2)) - let fromIdx = takeIntFlag("--from", from: &rest) ?? 0 - let count = takeIntFlag("--count", from: &rest) ?? 12 + let fromIdx = takeIntFlag("--from", in: 0...1_000_000, from: &rest) ?? 0 + let count = takeIntFlag("--count", in: 0...1_000_000, from: &rest) ?? 12 let noDV = takeFlag("--no-dv", from: &rest) let forceDV = takeFlag("--force-dv", from: &rest) let dvBaseLayer = takeFlag("--dv-base-layer", from: &rest) @@ -354,7 +358,7 @@ if first == "segverify" { // Rapid-seek burst repro (issue #35). if first == "seektest" { var rest = Array(args.dropFirst(2)) - let seeks = takeIntFlag("--seeks", from: &rest) ?? 40 + let seeks = takeIntFlag("--seeks", in: 0...1_000_000, from: &rest) ?? 40 let gapMs = takeIntFlag("--gap-ms", from: &rest) ?? 60 let settle = takeDoubleFlag("--settle", from: &rest) ?? 5.0 let throttleKbps = takeIntFlag("--throttle-kbps", from: &rest) @@ -374,8 +378,8 @@ if first == "seektest" { // SW-path background-audio keepalive harness (iOS background audio on the software decode path). if first == "bgaudio" { var rest = Array(args.dropFirst(2)) - let fg = takeDoubleFlag("--fg", from: &rest) ?? 3.0 - let bg = takeDoubleFlag("--bg", from: &rest) ?? 6.0 + let fg = takeDoubleFlag("--fg", in: 0...maxRunSeconds, from: &rest) ?? 3.0 + let bg = takeDoubleFlag("--bg", in: 0...maxRunSeconds, from: &rest) ?? 6.0 guard let urlArg = rest.first(where: { !$0.hasPrefix("--") }) else { print("ERROR: bgaudio requires a argument") print("Usage: aetherctl bgaudio [--fg N] [--bg N] ") @@ -388,7 +392,7 @@ if first == "bgaudio" { if first == "smbtest" { var rest = Array(args.dropFirst(2)) - let reads = takeIntFlag("--reads", from: &rest) ?? 64 + let reads = takeIntFlag("--reads", in: 0...1_000_000, from: &rest) ?? 64 guard let urlArg = rest.first(where: { !$0.hasPrefix("--") }) else { print("ERROR: smbtest requires a argument") exit(64) @@ -441,14 +445,15 @@ if first == "disc-inspect" { // DV P7 -> 8.1 converter validation harness. if first == "dovitest" { var rest = Array(args.dropFirst(2)) + let outPath = takeStringFlag("--out", from: &rest) guard let urlArg = rest.first(where: { !$0.hasPrefix("--") }) else { print("ERROR: dovitest requires a argument") - print("Usage: aetherctl dovitest ") + print("Usage: aetherctl dovitest [--out PATH.hevc] ") exit(64) } rest.removeAll { $0 == urlArg } rejectStrayFlags(rest, subcommand: "dovitest") - exit(runDoviTest(url: parseSourceURL(urlArg))) + exit(runDoviTest(url: parseSourceURL(urlArg), outputPath: outPath)) } // #93 post-recovery judder: raw video packet timing per demuxer open profile. @@ -471,7 +476,7 @@ if first == "pktdump" { if first == "audiotap" { var rest = Array(args.dropFirst(2)) let duration = takeDoubleFlag("--duration", from: &rest) ?? 30 - let outPath = takeStringFlag("--out", from: &rest) ?? "/tmp/audiotap.wav" + let outPath = takeStringFlag("--out", from: &rest) ?? debugOutputPath("audiotap.wav") let remote = rest.contains("--remote") rest.removeAll { $0 == "--remote" } let software = rest.contains("--software") @@ -500,7 +505,7 @@ if first == "hlslive" { if first == "live" { var rest = Array(args.dropFirst(2)) - let seconds = takeDoubleFlag("--seconds", from: &rest) ?? 20.0 + let seconds = takeDoubleFlag("--seconds", in: 0...maxRunSeconds, from: &rest) ?? 20.0 let dvrWindow = takeDoubleFlag("--dvr-window", from: &rest) let seed = takeStringFlag("--seed", from: &rest) let serveOnly = takeFlag("--serve-only", from: &rest) @@ -590,7 +595,7 @@ if first == "live" { if first == "play" { var rest = Array(args.dropFirst(2)) - let seconds = takeDoubleFlag("--seconds", from: &rest) ?? 30.0 + let seconds = takeDoubleFlag("--seconds", in: 0...maxRunSeconds, from: &rest) ?? 30.0 let live = takeFlag("--live", from: &rest) // AE#293: the nativeRemoteHLS bypass, the path the #168 carriage watchdog and the carriage probe // live on. Pair with --live; without it the m3u8 goes to the raw live path, which rejects it. @@ -633,6 +638,7 @@ if first == "play" { let seekCount = takeIntFlag("--seek-count", from: &rest) let mallocCensus = takeFlag("--malloc-census", from: &rest) let playForceSW = takeFlag("--sw", from: &rest) + let playSoftwareEscalation = !takeFlag("--no-sw-escalation", from: &rest) // AE#493: `LoadOptions.panelPresentsDolbyVision`, the host assertion. macOS has no per-mode display // capability API, so DV is unclaimable from inside the engine and a Mac run routes every DV source // as its HDR10 base layer until the host says otherwise. @@ -690,6 +696,8 @@ if first == "play" { // per-frame boundaries and the clock a host would pace an overlay against are both observable. let frameTimes = takeFlag("--frame-times", from: &rest) let presentTimes = takeFlag("--present-times", from: &rest) + // AE#684: print the loopback URL of every native item, so its init and segments can be fetched. + let servedURL = takeFlag("--served-url", from: &rest) let pictureProbe = takeFlag("--picture-probe", from: &rest) // AE#534: the source axis's origin, for a container whose timeline does not start at zero. // The picture states a frame index, which an -output_ts_offset remux does not move. @@ -871,8 +879,8 @@ if first == "play" { } done.wait() } - exit(runPlay(url: parseSourceURL(urlArg), seconds: seconds, live: live, nativeHLS: nativeHLS, liveIngest: liveIngest, fastZap: playFastZap, liveStartImmediately: liveStartImmediately, dvrWindow: dvrWindow, subsPick: subsPick, hostCalls: hostCalls, audioStats: audioStats, seekEvery: seekEvery, seekPattern: seekPattern, seekCount: seekCount, startPosition: playStartPosition, mallocCensus: mallocCensus, forceSoftware: playForceSW, - censusThresholdMB: censusThresholdMB, censusHz: censusHz, frameTimes: frameTimes, presentTimes: presentTimes, pictureProbe: pictureProbe, pictureOrigin: pictureOrigin, sidecars: sidecars, + exit(runPlay(url: parseSourceURL(urlArg), seconds: seconds, live: live, nativeHLS: nativeHLS, liveIngest: liveIngest, fastZap: playFastZap, liveStartImmediately: liveStartImmediately, dvrWindow: dvrWindow, subsPick: subsPick, hostCalls: hostCalls, audioStats: audioStats, seekEvery: seekEvery, seekPattern: seekPattern, seekCount: seekCount, startPosition: playStartPosition, mallocCensus: mallocCensus, forceSoftware: playForceSW, softwareEscalation: playSoftwareEscalation, + censusThresholdMB: censusThresholdMB, censusHz: censusHz, frameTimes: frameTimes, presentTimes: presentTimes, servedURL: servedURL, pictureProbe: pictureProbe, pictureOrigin: pictureOrigin, sidecars: sidecars, audioSwitch: audioSwitch, teletextPage: teletextPage, teletextSwitch: teletextSwitch, audioDelayMs: audioDelayMs, audioDelaySwitches: audioDelaySwitches, @@ -905,6 +913,7 @@ if ["probe", "serve", "validate", "swdecode", "extract", "audio", "customio"].co // instead of only through a host. Each costs reads past find_stream_info; the bare `probe` does neither. var probeDetail: ProbeDetail = [] if takeFlag("--detect-hdr10plus", from: &rest) { probeDetail.insert(.hdr10Plus) } + if takeFlag("--detect-hdr-vivid", from: &rest) { probeDetail.insert(.hdrVivid) } if takeFlag("--detect-atmos", from: &rest) { probeDetail.insert(.atmos) } let inMemory = takeFlag("--memory", from: &rest) let forwardOnly = takeFlag("--forward-only", from: &rest) @@ -964,10 +973,12 @@ if ["probe", "serve", "validate", "swdecode", "extract", "audio", "customio"].co let switchAudioFlag = takeFlag("--switch-audio", from: &rest) let selectSubsFlag = takeFlag("--select-subs", from: &rest) let extractFlag = takeFlag("--extract", from: &rest) - let secondsFlag = takeDoubleFlag("--seconds", from: &rest) + let secondsFlag = takeDoubleFlag("--seconds", in: 0...maxRunSeconds, from: &rest) let audioSeconds = secondsFlag ?? 10 // --native-subs: diagnostics affordance for mov_text subtitle track (#55); serve only. let nativeSubsIndex = takeIntFlag("--native-subs", from: &rest) + // AE#682: list an I-frame rendition in the master; serve only. + let serveIFrames = takeFlag("--iframes", from: &rest) // --throttle-kbps: slow-CDN simulation; starves the producer below real-time to provoke rebuffers. let throttleKbps = takeIntFlag("--throttle-kbps", from: &rest) // --start-position: anchor the first producer at a resume position like load(startPosition:) (#99); serve only. @@ -1010,7 +1021,7 @@ if ["probe", "serve", "validate", "swdecode", "extract", "audio", "customio"].co nativeSubsIndex: nativeSubsIndex, startPosition: startPosition, audioDelayMs: serveAudioDelayMs, objectAudioRendering: serveObjectAudioRendering, // `--audio-index` (parsed once above, shared with customio): serve that source stream. - audioStreamIndex: customAudioIndex) + audioStreamIndex: customAudioIndex, iFrames: serveIFrames) case "validate": exit(runValidate(url: url, dvModeAvailable: dvModeAvailable, forceDVWithoutDisplay: forceDV, dolbyVisionHandling: dvHandling)) diff --git a/Tests/AetherEngineTests/ASSOverrideBlockScanTests.swift b/Tests/AetherEngineTests/ASSOverrideBlockScanTests.swift new file mode 100644 index 000000000..9649841f1 --- /dev/null +++ b/Tests/AetherEngineTests/ASSOverrideBlockScanTests.swift @@ -0,0 +1,182 @@ +import Testing +import Foundation +@testable import AetherEngine + +/// Audit SUB-101 / SUB-110: `cleanASSBody` stripped override blocks with `\{[^}]*\}`, which is +/// quadratic on a cue made of unclosed `{` (20k braces took 2.6 s, doubling costs 4x) and ran on the +/// pump thread under the subtitle tap lock. `edgeTrimmed` shifted the run array once per leading +/// blank run. Both are one pass now. +struct ASSOverrideBlockScanTests { + + /// The implementation `cleanASSBody` had before the forward scan, kept as the oracle. + private func legacyCleanASSBody(_ raw: String) -> String? { + var s = raw + s = s.replacingOccurrences(of: "\\N", with: "\n") + s = s.replacingOccurrences(of: "\\n", with: "\n") + s = s.replacingOccurrences(of: "\\h", with: " ") + s = s.replacingOccurrences(of: "\\{[^}]*\\}", with: "", options: .regularExpression) + let trimmed = s.trimmingCharacters(in: .whitespacesAndNewlines) + return trimmed.isEmpty ? nil : trimmed + } + + private struct SplitMix64: RandomNumberGenerator { + var state: UInt64 + mutating func next() -> UInt64 { + state &+= 0x9E37_79B9_7F4A_7C15 + var z = state + z = (z ^ (z >> 30)) &* 0xBF58_476D_1CE4_E5B9 + z = (z ^ (z >> 27)) &* 0x94D0_49BB_1331_11EB + return z ^ (z >> 31) + } + } + + private static let normalLines: [String] = [ + "plain text", + "", + " ", + "{\\an8}Top of the screen", + "{\\an8}{\\b1}Top{\\b0}\\Nline two", + "{\\pos(192,240)}Positioned{\\i1} italic{\\i0}", + "{\\1c&H00FF00&}green{\\r} back", + "{a}{b}{c}adjacent blocks", + "{a}{b}", + "{}empty block", + "x{}y{}z", + "{a{b}c}nested open brace", + "{{a}b", + "{a}{b", + "a{b", + "{", + "}", + "}{", + "a}b{c}d", + "{\\k20}ka{\\k25}ra{\\k30}o{\\k15}ke", + "line one\\Nline two\\nline three\\hspaced", + "{\\fnArial\\fs32\\b1}\\NLeading break", + "trailing {\\i1}", + " {\\an2} padded ", + "multi\nline {\\b1}text\r\nwith {\\i1}CRLF", + "{multi\nline\nblock}after", + "\u{00E9}\u{00E8} {\\i1}caf\u{00E9}{\\i0} \u{4E2D}\u{6587} \u{1F600} end", + "{\u{4E2D}\u{6587}}\u{1F600}", + "e\u{0301}{\\b1}x", + "{\u{0301}}x", + "{\\N}x", + "\\N{\\N}\\N", + "{\\b1}{\\i1}{\\u1}{\\s1}", + ] + + @Test("the forward scan matches the old regex on ordinary lines") + func matchesRegexOnOrdinaryLines() { + for line in Self.normalLines { + #expect(SubtitleRectText.cleanASSBody(line) == legacyCleanASSBody(line), "line: \(line.debugDescription)") + } + } + + @Test("the forward scan matches the old regex on generated lines full of braces") + func matchesRegexOnGeneratedLines() { + let alphabet: [String] = ["{", "}", "{", "}", "\\", "N", "n", "h", "a", "b", " ", "\n", "\u{00E9}", + "\u{1F600}", "\u{0301}", "\\N", "{\\b1}", ","] + var rng = SplitMix64(state: 0xA55_0C1E) + for _ in 0..<6000 { + let length = Int.random(in: 0...40, using: &rng) + var line = "" + for _ in 0..?, size: Int32) -> Int32 { + guard let buffer, size > 0 else { return -1 } + condition.lock() + defer { condition.unlock() } + if let blockAt, position >= blockAt { + parked = true + while !released { condition.wait() } + return 0 + } + var n = min(Int(size), 64, bytes.count - position) + if let blockAt { n = min(n, blockAt - position) } + guard n > 0 else { return 0 } + bytes.withUnsafeBytes { raw in _ = memcpy(buffer, raw.baseAddress! + position, n) } + position += n + return Int32(n) + } + + func seek(offset: Int64, whence: Int32) -> Int64 { + condition.withLock { + switch whence { + case 65536: return Int64(bytes.count) + case SEEK_SET: position = Int(max(0, min(offset, Int64(bytes.count)))) + case SEEK_CUR: position = max(0, min(position + Int(offset), bytes.count)) + case SEEK_END: position = max(0, min(bytes.count + Int(offset), bytes.count)) + default: return -1 + } + return Int64(position) + } + } + + func close() {} + var discImageProbeEnabled: Bool { false } + } + + private final class ReadOutcome: @unchecked Sendable { + enum Kind: Sendable { case endOfFile, threw } + private let lock = NSLock() + private var _kind: Kind? + var kind: Kind? { lock.withLock { _kind } } + func finish(_ kind: Kind) { lock.withLock { _kind = kind } } + } + + /// The SimpleBlock boundary the Issue234 fixture's open never reads past, so the parked read + /// comes back as a clean end of file rather than a torn element. + private static let parkOffset = 2237 + + private func fixture() throws -> Data { + try #require(Data(base64Encoded: Issue234SideReaderSeekAnchorTests.fixtureBase64, + options: .ignoreUnknownCharacters)) + } + + private func drain(_ demuxer: Demuxer, into outcome: ReadOutcome) { + Thread.detachNewThread { + do { + while let read = try demuxer.readPacket() { + var packet: UnsafeMutablePointer? = read + trackedPacketFree(&packet) + } + outcome.finish(.endOfFile) + } catch { + outcome.finish(.threw) + } + } + } + + @Test("a failed read is the source's end only while nobody has closed the demuxer") + func readFailureClassification() { + #expect(Demuxer.readFailureCode(FFmpegErr.eof, closeRequested: false) == nil, + "a real end of file must stay the end of the source") + #expect(Demuxer.readFailureCode(FFmpegErr.eof, closeRequested: true) == FFmpegErr.exit, + "an end of file on a closed demuxer is the abort, not the source's") + for code in [FFmpegErr.eio, FFmpegErr.invalidData, FFmpegErr.exit, -1] { + #expect(Demuxer.readFailureCode(code, closeRequested: false) == code) + #expect(Demuxer.readFailureCode(code, closeRequested: true) == code) + } + } + + @Test("a read that the close aborts throws instead of reporting the end of the source", + .timeLimit(.minutes(1))) + func abortedReadThrows() async throws { + let reader = ParkingReader(try fixture()) + let demuxer = Demuxer() + try demuxer.open(reader: reader, formatHint: "matroska") + defer { demuxer.close() } + // Runs before the close: a wait the time limit cancels must not leave the parked read + // holding the access lock that close() waits on. + defer { reader.release() } + reader.arm(blockAt: Self.parkOffset) + + let outcome = ReadOutcome() + drain(demuxer, into: outcome) + try await waitFor { reader.isParked } + demuxer.markClosed() + reader.release() + try await waitFor { outcome.kind != nil } + + #expect(outcome.kind == .threw, "the aborted read was reported as the end of the source") + } + + @Test("the end of a source nobody closed is still the end of the source", + .timeLimit(.minutes(1))) + func realEndOfFileIsStillNil() async throws { + let demuxer = Demuxer() + try demuxer.open(reader: ParkingReader(try fixture()), formatHint: "matroska") + defer { demuxer.close() } + + let outcome = ReadOutcome() + drain(demuxer, into: outcome) + try await waitFor { outcome.kind != nil } + + #expect(outcome.kind == .endOfFile) + } +} diff --git a/Tests/AetherEngineTests/AnnexBSampleConverterTests.swift b/Tests/AetherEngineTests/AnnexBSampleConverterTests.swift new file mode 100644 index 000000000..19df958c1 --- /dev/null +++ b/Tests/AetherEngineTests/AnnexBSampleConverterTests.swift @@ -0,0 +1,73 @@ +// Annex-B HEVC samples converted by the session muxer instead of movenc (PR #703, MovieClaw P38). +// +// movenc's own conversion under `hvc1` drops every in-band VPS/SPS/PPS, so a stream that sends a new +// PPS mid-title has its later slices decoded against the stale one. The converter must keep every NAL. +import Foundation +import Testing +import AetherLibavcodec +@testable import AetherEngine + +@Suite("Annex-B samples keep their in-band parameter sets (PR #703)") +struct AnnexBSampleConverterTests { + + private let aud: [UInt8] = [0x46, 0x01, 0x50] + private let vps: [UInt8] = [0x40, 0x01, 0x0C] + private let pps: [UInt8] = [0x44, 0x01, 0xC0, 0x72] + /// An IDR slice carrying an emulation-prevention byte, which must pass through untouched. + private let idr: [UInt8] = [0x28, 0x01, 0xAF, 0x00, 0x00, 0x03, 0x01, 0x80] + + private func lengthPrefixed(_ nals: [[UInt8]]) -> [UInt8] { + nals.flatMap { nal -> [UInt8] in + let n = nal.count + return [UInt8(n >> 24 & 0xFF), UInt8(n >> 16 & 0xFF), UInt8(n >> 8 & 0xFF), UInt8(n & 0xFF)] + nal + } + } + + /// Run the packet-level conversion on a fresh packet holding `bytes`; nil when it declined. + private func convert(_ bytes: [UInt8]) -> [UInt8]? { + var packet: UnsafeMutablePointer? = av_packet_alloc() + defer { av_packet_free(&packet) } + guard let pkt = packet, av_new_packet(pkt, Int32(bytes.count)) == 0 else { return nil } + bytes.withUnsafeBytes { _ = memcpy(pkt.pointee.data, $0.baseAddress, bytes.count) } + guard AnnexBSampleConverter.convertToLengthPrefixed(pkt) else { return nil } + return Array(UnsafeBufferPointer(start: pkt.pointee.data, count: Int(pkt.pointee.size))) + } + + @Test("Every NAL survives, parameter sets included, and start-code zero padding is dropped") + func keepsParameterSets() { + let annexB: [UInt8] = [0, 0, 0, 1] + aud + [0, 0, 1] + vps + [0, 0, 0, 1] + pps + + [0, 0, 1] + idr + [0, 0] + #expect(convert(annexB) == lengthPrefixed([aud, vps, pps, idr])) + } + + @Test("A sample with no start code is declined and left as it was") + func lengthPrefixedInputDeclined() { + #expect(convert(lengthPrefixed([idr])) == nil) + } + + @Test("A NAL in the 256 to 511 byte band is converted once, not split at its own length bytes") + func midSizedNAL() { + // Its length prefix would read 00 00 01 xx, but the input here is Annex B, so the only start + // codes are the real ones and the payload contains none. + let slice: [UInt8] = [0x02, 0x01] + [UInt8](repeating: 0x55, count: 300) + let annexB: [UInt8] = [0, 0, 0, 1] + pps + [0, 0, 0, 1] + slice + #expect(convert(annexB) == lengthPrefixed([pps, slice])) + } + + @Test("The converted packet's buffer is a new one, the source buffer is not written") + func sourceBufferUntouched() { + let annexB: [UInt8] = [0, 0, 1] + pps + [0, 0, 1] + idr + var source: UnsafeMutablePointer? = av_packet_alloc() + var shared: UnsafeMutablePointer? = av_packet_alloc() + defer { av_packet_free(&source); av_packet_free(&shared) } + guard let src = source, let ref = shared, av_new_packet(src, Int32(annexB.count)) == 0 else { + Issue.record("packet allocation failed"); return + } + annexB.withUnsafeBytes { _ = memcpy(src.pointee.data, $0.baseAddress, annexB.count) } + #expect(av_packet_ref(ref, src) == 0) + #expect(AnnexBSampleConverter.convertToLengthPrefixed(ref)) + #expect(Array(UnsafeBufferPointer(start: src.pointee.data, count: Int(src.pointee.size))) == annexB) + #expect(Array(UnsafeBufferPointer(start: ref.pointee.data, count: Int(ref.pointee.size))) + == lengthPrefixed([pps, idr])) + } +} diff --git a/Tests/AetherEngineTests/AtmosConfirmationForkTests.swift b/Tests/AetherEngineTests/AtmosConfirmationForkTests.swift new file mode 100644 index 000000000..9dcdad116 --- /dev/null +++ b/Tests/AetherEngineTests/AtmosConfirmationForkTests.swift @@ -0,0 +1,51 @@ +import Testing +import Foundation +@testable import AetherEngine + +/// Audit DEC-109: `startAtmosConfirmation` forked the host's custom reader only to test the fork for +/// nil and dropped it, and the `IOReader` contract says the engine owns and closes every independent +/// reader it is handed. +@Suite("Atmos confirmation reader probe (DEC-109)") +struct AtmosConfirmationForkTests { + + private final class Fork: IOReader, @unchecked Sendable { + private let lock = NSLock() + private var _closed = false + var isClosed: Bool { lock.withLock { _closed } } + func read(_ buffer: UnsafeMutablePointer?, size: Int32) -> Int32 { 0 } + func seek(offset: Int64, whence: Int32) -> Int64 { -1 } + func close() { lock.withLock { _closed = true } } + } + + private final class Host: IOReader, @unchecked Sendable { + private let lock = NSLock() + private let forkable: Bool + private var _forks: [Fork] = [] + var forks: [Fork] { lock.withLock { _forks } } + init(forkable: Bool) { self.forkable = forkable } + func read(_ buffer: UnsafeMutablePointer?, size: Int32) -> Int32 { 0 } + func seek(offset: Int64, whence: Int32) -> Int64 { -1 } + func close() {} + func makeIndependentReader() -> IOReader? { + guard forkable else { return nil } + let fork = Fork() + lock.withLock { _forks.append(fork) } + return fork + } + } + + @Test("the probe closes the fork it built") + func probeClosesItsFork() { + let host = Host(forkable: true) + #expect(AetherEngine.canForkCustomReader(host)) + #expect(host.forks.count == 1) + let allClosed = host.forks.allSatisfy { $0.isClosed } + #expect(allClosed, "a fork built to be nil-tested must not outlive the test") + } + + @Test("a one-shot reader, or none at all, cannot be forked") + func unforkableReader() { + #expect(!AetherEngine.canForkCustomReader(Host(forkable: false))) + #expect(!AetherEngine.canForkCustomReader(nil)) + } +} diff --git a/Tests/AetherEngineTests/AtmosDetectionOptionsTests.swift b/Tests/AetherEngineTests/AtmosDetectionOptionsTests.swift index 35ffecd2a..59089d978 100644 --- a/Tests/AetherEngineTests/AtmosDetectionOptionsTests.swift +++ b/Tests/AetherEngineTests/AtmosDetectionOptionsTests.swift @@ -75,6 +75,14 @@ struct AtmosDetectionOptionsTests { #expect(AetherEngine.atmosForeignPacketFuse(maxPackets: Int.max) == Int.max) } + @Test("the input byte budget covers interleaved UHD video and saturates instead of trapping") + func inputByteBudgetIsBoundedAndSaturates() { + #expect(AetherEngine.atmosInputByteBudget(maxBytes: 8 * 1024 * 1024) == 128 * 1024 * 1024) + #expect(AetherEngine.atmosInputByteBudget(maxBytes: 0) == 64 * 1024 * 1024) + #expect(AetherEngine.atmosInputByteBudget(maxBytes: -1) == 64 * 1024 * 1024) + #expect(AetherEngine.atmosInputByteBudget(maxBytes: .max) == .max) + } + @Test("no default audio stream (-1) surfaces unchanged when no override is given") func noAudioStreamPropagatesAsNegativeOne() { let options = AtmosDetectionOptions() diff --git a/Tests/AetherEngineTests/AudioBridgeLiveBytesTests.swift b/Tests/AetherEngineTests/AudioBridgeLiveBytesTests.swift new file mode 100644 index 000000000..0a8bc6e1e --- /dev/null +++ b/Tests/AetherEngineTests/AudioBridgeLiveBytesTests.swift @@ -0,0 +1,137 @@ +import Testing +import Foundation +import AetherLibavcodec +import AetherLibavutil +@testable import AetherEngine + +/// Audit DEC-104 / Vpipeline-101: the 1 Hz telemetry read `swrCtx` and `encoderCtx` without the +/// bridge's lock and called `swr_get_delay` on them, while the pump thread frees and replaces both +/// mid-stream (a decoded format change rebuilds the resampler, a restart after EOF rebuilds the +/// encoder). The read is a use-after-free in a window a few instructions wide, so the defect is pinned +/// as a stress run that is meant to be read under TSan, plus the figures the snapshot must agree on. +@Suite("AudioBridge live bytes snapshot (DEC-104, Vpipeline-101)") +struct AudioBridgeLiveBytesTests { + + private final class Flag: @unchecked Sendable { + private let lock = NSLock() + private var done = false + func set() { lock.withLock { done = true } } + var isSet: Bool { lock.withLock { done } } + } + + private final class Tally: @unchecked Sendable { + private let lock = NSLock() + private var polls = 0 + private var insane = 0 + func record(sane: Bool) { lock.withLock { polls += 1; if !sane { insane += 1 } } } + var snapshot: (polls: Int, insane: Int) { lock.withLock { (polls, insane) } } + } + + private func makeCodecpar() -> UnsafeMutablePointer { + let par = avcodec_parameters_alloc()! + par.pointee.codec_type = AVMEDIA_TYPE_AUDIO + par.pointee.codec_id = AV_CODEC_ID_MP3 + par.pointee.sample_rate = 48_000 + par.pointee.format = AV_SAMPLE_FMT_FLTP.rawValue + av_channel_layout_default(&par.pointee.ch_layout, 2) + return par + } + + /// One silent MPEG-1 Layer III frame (128 kbps). 44.1 kHz and 48 kHz frames alternate, so the + /// decoder answers with a different rate every packet and the bridge rebuilds its resampler each time. + private func makeFrame(rate48k: Bool, pts: Int64) -> UnsafeMutablePointer? { + let size = rate48k ? 384 : 417 + guard let pkt = trackedPacketAlloc(), av_new_packet(pkt, Int32(size)) >= 0 else { return nil } + memset(pkt.pointee.data, 0, size) + pkt.pointee.data[0] = 0xFF + pkt.pointee.data[1] = 0xFB + pkt.pointee.data[2] = rate48k ? 0x94 : 0x90 + pkt.pointee.pts = pts + pkt.pointee.dts = pts + return pkt + } + + private func free(_ packets: [UnsafeMutablePointer]) { + for p in packets { + var pp: UnsafeMutablePointer? = p + trackedPacketFree(&pp) + } + } + + @Test("a poller reading liveBytes through resampler and encoder swaps sees only sane figures") + func pollerSurvivesContextSwaps() throws { + let codecpar = makeCodecpar() + defer { + var p: UnsafeMutablePointer? = codecpar + avcodec_parameters_free(&p) + } + let bridge = try AudioBridge(srcCodecpar: codecpar, srcTimeBase: AVRational(num: 1, den: 1000), + mode: .surroundCompat) + defer { bridge.close() } + + let finished = Flag() + let tally = Tally() + let poller = Thread { + while !finished.isSet { + let live = bridge.liveBytes + let sane = live.fifoSamples >= 0 + && live.swrDelaySamples >= 0 && live.swrDelaySamples < 1 << 20 + && live.fifoBytes >= 0 && live.swrDelayBytes >= 0 + && live.totalBytes == live.fifoBytes + live.swrDelayBytes + tally.record(sane: sane) + } + } + poller.start() + + var produced = 0 + for round in 0..<40 { + for i in 0..<40 { + guard let pkt = makeFrame(rate48k: i % 2 == 0, pts: Int64(round * 40 + i) * 26) else { continue } + let out = (try? bridge.feed(packet: pkt)) ?? [] + produced += out.count + free(out) + var p: UnsafeMutablePointer? = pkt + trackedPacketFree(&p) + } + free(bridge.flush()) + bridge.startSegment() + } + finished.set() + while !poller.isFinished { usleep(1000) } + + #expect(bridge.feedStats.framesDecoded > 0, "the fixture must reach the resampler for the swaps to happen") + let result = tally.snapshot + #expect(result.polls > 0) + #expect(result.insane == 0) + } + + @Test("the snapshot follows the bridge through feed, restart and close") + func snapshotTracksOperations() throws { + let codecpar = makeCodecpar() + defer { + var p: UnsafeMutablePointer? = codecpar + avcodec_parameters_free(&p) + } + let bridge = try AudioBridge(srcCodecpar: codecpar, srcTimeBase: AVRational(num: 1, den: 1000), + mode: .surroundCompat) + for i in 0..<3 { + guard let pkt = makeFrame(rate48k: true, pts: Int64(i) * 24) else { continue } + free((try? bridge.feed(packet: pkt)) ?? []) + var p: UnsafeMutablePointer? = pkt + trackedPacketFree(&p) + } + let afterFeed = bridge.liveBytes + #expect(afterFeed.fifoSamples > 0, "three MP3 frames leave a partial encoder frame in the FIFO") + #expect(afterFeed.fifoSamples == bridge.fifoSampleCount) + #expect(afterFeed.fifoBytes > 0) + + bridge.startSegment() + #expect(bridge.liveBytes.fifoSamples == 0) + #expect(bridge.fifoSampleCount == 0) + + bridge.close() + let closed = bridge.liveBytes + #expect(closed.fifoSamples == 0 && closed.fifoBytes == 0) + #expect(closed.swrDelaySamples == 0 && closed.swrDelayBytes == 0) + } +} diff --git a/Tests/AetherEngineTests/AudioOnlyClockAnchorTests.swift b/Tests/AetherEngineTests/AudioOnlyClockAnchorTests.swift new file mode 100644 index 000000000..01b3c783b --- /dev/null +++ b/Tests/AetherEngineTests/AudioOnlyClockAnchorTests.swift @@ -0,0 +1,169 @@ +import Testing +import Foundation +import CoreMedia +@testable import AetherEngine + +/// Audit DEC-102: the audio-only FFmpeg host anchored its clock at the load position and ignored the +/// first sample's timestamp. A source whose timestamps start far past that (an Ogg radio stream's granule +/// position, an audio-only TS that joined mid-broadcast) then stamped every buffer hours ahead of a clock +/// that started at 0, the back-pressure gate parked the loop against the gap, and the session played +/// silence. The software video host resolved the same question with `SWClockAnchorPolicy`; this host now +/// shares it. +@Suite("Audio-only clock anchor (DEC-102)", .serialized) +struct AudioOnlyClockAnchorTests { + + // MARK: - The decision + + @Test("a first sample an hour past the load anchor moves the anchor and records the session zero") + func farFirstSampleAnchorsAtTheSample() { + let first = CMTime(seconds: 3600, preferredTimescale: 90000) + let resolved = AudioPlaybackHost.clockAnchor(initialClockTime: .zero, firstPTS: first) + #expect(resolved.anchor.seconds == 3600) + #expect(resolved.sessionZeroSeconds == 3600) + #expect(AudioPlaybackHost.publishedTime(raw: 3600, sessionZeroSeconds: resolved.sessionZeroSeconds) == 0) + #expect(AudioPlaybackHost.publishedTime(raw: 3612.5, sessionZeroSeconds: resolved.sessionZeroSeconds) == 12.5) + } + + @Test("a resume whose first sample is where it was asked to be keeps the load anchor verbatim") + func alignedResumeKeepsTheAnchor() { + let initial = CMTime(seconds: 1000, preferredTimescale: 90000) + let resolved = AudioPlaybackHost.clockAnchor( + initialClockTime: initial, firstPTS: CMTime(seconds: 1000.4, preferredTimescale: 90000)) + #expect(resolved.anchor == initial) + #expect(resolved.sessionZeroSeconds == 0) + #expect(AudioPlaybackHost.publishedTime(raw: 1003, sessionZeroSeconds: 0) == 1003) + } + + @Test("an invalid first timestamp keeps the load anchor") + func invalidFirstSampleKeepsTheAnchor() { + let resolved = AudioPlaybackHost.clockAnchor(initialClockTime: .zero, firstPTS: .invalid) + #expect(resolved.anchor == .zero) + #expect(resolved.sessionZeroSeconds == 0) + } + + @Test("a seek target on the published axis round-trips through the source axis") + func seekTargetRoundTrips() { + let zero = 3600.0 + let source = SWClockAnchorPolicy.sourceSeconds(forSession: 42, sessionZeroSeconds: zero) + #expect(source == 3642) + #expect(AudioPlaybackHost.publishedTime(raw: source, sessionZeroSeconds: zero) == 42) + } + + // MARK: - End to end, on a real demuxer + + @MainActor + @Test("an audio-only TS that starts an hour in is anchored at its first sample, and seeks stay on its axis") + func hourOffsetTransportStream() async throws { + let stream = Self.makeMP2TransportStream(startPTSSeconds: 3600, frames: 200) + let demuxer = Demuxer() + try demuxer.open(reader: DataIOReader(data: stream), formatHint: "mpegts") + let host = AudioPlaybackHost() + try await host.load(demuxer: demuxer, startPosition: nil, audioSourceStreamIndex: nil) + defer { host.stop() } + + host.play() + try await waitFor { host.isClockArmedForTesting } + + #expect(abs(host.sessionZeroForTesting - 3600) < 1, "zero \(host.sessionZeroForTesting)") + let clock = try #require(host.clockSecondsForTesting) + #expect(clock >= 3599 && clock < 3700, "the clock has to sit where the samples are, not at 0: \(clock)") + + await host.seek(to: 1.0) + let seeked = try #require(host.clockSecondsForTesting) + #expect(abs(seeked - 3601) < 2, "a seek to 1.0 s of the published axis is 3601 s of the source: \(seeked)") + #expect(host.currentTime == 1.0) + } + + // MARK: - Fixture: MPEG-1 Layer II in MPEG-TS + + /// `frames` silent 48 kHz mono Layer II frames (24 ms each, zero allocation bits decode to silence), + /// one PES packet each, the first stamped at `startPTSSeconds`. No PCR: the demuxer takes audio + /// timestamps from the PES headers. + private static func makeMP2TransportStream(startPTSSeconds: Double, frames: Int) -> Data { + let pmtPID: UInt16 = 0x1000, audioPID: UInt16 = 0x0100 + var counters: [UInt16: UInt8] = [:] + var out = Data() + + var pat = Data([0x00, 0x01, 0xC1, 0x00, 0x00, 0x00, 0x01]) + pat.append(contentsOf: [UInt8(0xE0 | (pmtPID >> 8)), UInt8(pmtPID & 0xFF)]) + out.append(psi(pid: 0, section: section(tableID: 0x00, body: pat), counters: &counters)) + + var pmt = Data([0x00, 0x01, 0xC1, 0x00, 0x00]) + pmt.append(contentsOf: [UInt8(0xE0 | (audioPID >> 8)), UInt8(audioPID & 0xFF), 0xF0, 0x00]) + pmt.append(contentsOf: [0x03, UInt8(0xE0 | (audioPID >> 8)), UInt8(audioPID & 0xFF), 0xF0, 0x00]) + out.append(psi(pid: pmtPID, section: section(tableID: 0x02, body: pmt), counters: &counters)) + + var frame = Data(count: 192) + frame.replaceSubrange(0..<4, with: [0xFF, 0xFD, 0x44, 0xC0]) + let startTicks = Int64(startPTSSeconds * 90_000) + for i in 0.. Data { + let length = body.count + 4 + var s = Data([tableID, UInt8(0xB0 | ((length >> 8) & 0x0F)), UInt8(length & 0xFF)]) + s.append(body) + var crc: UInt32 = 0xFFFF_FFFF + for byte in s { + crc ^= UInt32(byte) << 24 + for _ in 0..<8 { crc = (crc & 0x8000_0000) != 0 ? (crc << 1) ^ 0x04C1_1DB7 : crc << 1 } + } + for shift in stride(from: 24, through: 0, by: -8) { s.append(UInt8((crc >> UInt32(shift)) & 0xFF)) } + return s + } + + private static func psi(pid: UInt16, section: Data, counters: inout [UInt16: UInt8]) -> Data { + var payload = Data([0x00]) + payload.append(section) + payload.append(contentsOf: [UInt8](repeating: 0xFF, count: 184 - payload.count)) + return packet(pid: pid, pusi: true, payload: payload, counters: &counters) + } + + private static func pes(pid: UInt16, payload: Data, pts: Int64, counters: inout [UInt16: UInt8]) -> Data { + var packetData = Data([0x00, 0x00, 0x01, 0xC0]) + let length = 3 + 5 + payload.count + packetData.append(contentsOf: [UInt8(length >> 8), UInt8(length & 0xFF), 0x80, 0x80, 0x05]) + packetData.append(contentsOf: [ + UInt8(0x21 | ((pts >> 29) & 0x0E)), + UInt8((pts >> 22) & 0xFF), + UInt8(0x01 | ((pts >> 14) & 0xFE)), + UInt8((pts >> 7) & 0xFF), + UInt8(0x01 | ((pts << 1) & 0xFE)), + ]) + packetData.append(payload) + var out = Data() + var offset = 0 + var first = true + while offset < packetData.count { + let take = min(184, packetData.count - offset) + out.append(packet(pid: pid, pusi: first, payload: packetData.subdata(in: offset..<(offset + take)), + counters: &counters)) + offset += take + first = false + } + return out + } + + private static func packet(pid: UInt16, pusi: Bool, payload: Data, counters: inout [UInt16: UInt8]) -> Data { + let counter = counters[pid, default: 0] + var p = Data([0x47, UInt8((pusi ? 0x40 : 0x00) | Int(pid >> 8)), UInt8(pid & 0xFF)]) + let stuffing = 184 - payload.count + if stuffing == 0 { + p.append(0x10 | counter) + } else { + p.append(0x30 | counter) + p.append(UInt8(stuffing - 1)) + if stuffing >= 2 { + p.append(0x00) + p.append(contentsOf: [UInt8](repeating: 0xFF, count: stuffing - 2)) + } + } + p.append(payload) + counters[pid] = (counter &+ 1) & 0x0F + return p + } +} diff --git a/Tests/AetherEngineTests/AudioOutputFlushEpochTests.swift b/Tests/AetherEngineTests/AudioOutputFlushEpochTests.swift new file mode 100644 index 000000000..9efda83d7 --- /dev/null +++ b/Tests/AetherEngineTests/AudioOutputFlushEpochTests.swift @@ -0,0 +1,131 @@ +import Foundation +import AVFoundation +import CoreMedia +import Testing +@testable import AetherEngine + +/// Audit DEC-106: both FFmpeg hosts decided "this buffer still belongs to the current position" with a +/// seek-generation read and then called `AudioOutput.enqueue`, so a seek whose renderer flush landed +/// between the two parked a pre-seek buffer in the fresh queue (silence until the clock reached its +/// stamp, and on the software host a false rebuffer). The check has to be made under the lock `flush` +/// takes, against an epoch the flush retires. +@Suite("AudioOutput flush epoch (DEC-106)") +struct AudioOutputFlushEpochTests { + + private func makeBuffer(pts: Double, samples: Int = 1024, sampleRate: Int32 = 48000) -> CMSampleBuffer? { + var asbd = AudioStreamBasicDescription( + mSampleRate: Float64(sampleRate), + mFormatID: kAudioFormatLinearPCM, + mFormatFlags: kAudioFormatFlagIsFloat | kAudioFormatFlagIsPacked, + mBytesPerPacket: 4, mFramesPerPacket: 1, mBytesPerFrame: 4, + mChannelsPerFrame: 1, mBitsPerChannel: 32, mReserved: 0) + var format: CMAudioFormatDescription? + guard CMAudioFormatDescriptionCreate(allocator: kCFAllocatorDefault, + asbd: &asbd, + layoutSize: 0, layout: nil, + magicCookieSize: 0, magicCookie: nil, + extensions: nil, + formatDescriptionOut: &format) == noErr, + let format else { return nil } + var block: CMBlockBuffer? + guard CMBlockBufferCreateWithMemoryBlock(allocator: kCFAllocatorDefault, + memoryBlock: nil, + blockLength: samples * 4, + blockAllocator: kCFAllocatorDefault, + customBlockSource: nil, + offsetToData: 0, + dataLength: samples * 4, + flags: kCMBlockBufferAssureMemoryNowFlag, + blockBufferOut: &block) == noErr, + let block else { return nil } + var timing = CMSampleTimingInfo(duration: CMTime(value: 1, timescale: sampleRate), + presentationTimeStamp: CMTime(seconds: pts, preferredTimescale: sampleRate), + decodeTimeStamp: .invalid) + var out: CMSampleBuffer? + guard CMSampleBufferCreate(allocator: kCFAllocatorDefault, + dataBuffer: block, dataReady: true, + makeDataReadyCallback: nil, refcon: nil, + formatDescription: format, + sampleCount: CMItemCount(samples), + sampleTimingEntryCount: 1, sampleTimingArray: &timing, + sampleSizeEntryCount: 0, sampleSizeArray: nil, + sampleBufferOut: &out) == noErr else { return nil } + return out + } + + @Test("a buffer decided on before a flush is refused after it") + func staleBufferIsRefused() throws { + let output = AudioOutput() + let buffer = try #require(makeBuffer(pts: 100)) + let captured = output.epoch + output.flush() + #expect(output.enqueue(sampleBuffer: buffer, ifEpoch: captured) == false) + } + + @Test("a buffer decided on after the flush is accepted") + func freshBufferIsAccepted() throws { + let output = AudioOutput() + output.flush() + let buffer = try #require(makeBuffer(pts: 100)) + #expect(output.enqueue(sampleBuffer: buffer, ifEpoch: output.epoch)) + } + + @Test("every flush retires the epoch, and stop counts as one") + func flushAndStopMoveTheEpoch() { + let output = AudioOutput() + let start = output.epoch + output.flush() + let afterFlush = output.epoch + #expect(afterFlush != start) + output.stop() + #expect(output.epoch != afterFlush) + } + + private final class Returned: @unchecked Sendable { + private let lock = NSLock() + private var done = false + func set() { lock.withLock { done = true } } + var isSet: Bool { lock.withLock { done } } + } + + private final class BufferBox: @unchecked Sendable { + let buffer: CMSampleBuffer + init(_ buffer: CMSampleBuffer) { self.buffer = buffer } + } + + /// AE#549's observer takes the output's lock on whatever thread AVFoundation posts from, and `enqueue` + /// now holds that lock across `renderer.enqueue`. A renderer that posts from inside `enqueue` (the + /// 27 SDKs move the same condition into the enqueue call itself) would make the feed thread wait on a + /// lock it already holds, and every later `flush()` / `stop()` / `seekClock()` with it. + @Test("an automatic-flush notification posted from inside enqueue does not hang the feed", + .timeLimit(.minutes(1))) + func automaticFlushPostedInsideEnqueueDoesNotDeadlock() async throws { + let output = AudioOutput() + let box = BufferBox(try #require(makeBuffer(pts: 1))) + output.afterRendererEnqueueForTesting = { [unowned output] in + NotificationCenter.default.post(name: .AVSampleBufferAudioRendererWasFlushedAutomatically, + object: output.renderer) + } + let returned = Returned() + let feed = Thread { + output.enqueue(sampleBuffer: box.buffer) + returned.set() + } + feed.start() + + let finished = try await waitFor(upTo: .seconds(10)) { returned.isSet } + #expect(finished, "the feed thread is parked on a lock it holds itself") + guard finished else { return } + + try await waitFor { output.automaticFlushCount == 1 } + #expect(output.automaticFlushCount == 1, "the flush still happens, serialized with enqueueing") + } + + @Test("an unconditional enqueue is untouched by the epoch") + func unconditionalEnqueueStillEnqueues() throws { + let output = AudioOutput() + output.flush() + let buffer = try #require(makeBuffer(pts: 1)) + #expect(output.enqueue(sampleBuffer: buffer)) + } +} diff --git a/Tests/AetherEngineTests/AudioSelectionOwnershipTests.swift b/Tests/AetherEngineTests/AudioSelectionOwnershipTests.swift new file mode 100644 index 000000000..83e837e1c --- /dev/null +++ b/Tests/AetherEngineTests/AudioSelectionOwnershipTests.swift @@ -0,0 +1,53 @@ +import XCTest +@testable import AetherEngine + +@MainActor +final class AudioSelectionOwnershipTests: XCTestCase { + private func fixture() throws -> AetherEngine { + let engine = try AetherEngine() + engine.loadedURL = URL(string: "https://example.invalid/controlled.ts")! + engine.audioTracks = [1, 2].map { + TrackInfo(id: $0, name: "Track \($0)", codec: "aac", language: nil, isDefault: $0 == 1) + } + engine.activeAudioTrackIndex = 1 + return engine + } + + func testSameAudioDoesNotScheduleRebuild() throws { + let engine = try fixture() + defer { engine.stop() } + engine.selectAudioTrack(index: 1) + XCTAssertNil(engine.audioSelectionTask) + } + + func testRapidSelectionReturningToCurrentAudioCoalescesBeforeIO() async throws { + let engine = try fixture() + defer { engine.stop() } + let generation = engine.loadGeneration + engine.selectAudioTrack(index: 2) + engine.selectAudioTrack(index: 1) + await engine.audioSelectionTask?.value + XCTAssertEqual(engine.loadGeneration, generation) + XCTAssertEqual(engine.activeAudioTrackIndex, 1) + XCTAssertNil(engine.audioSelectionTask) + } + + func testStopInvalidatesQueuedAudioWorkBeforeItCanReopenSource() async throws { + let engine = try fixture() + engine.selectAudioTrack(index: 2) + let task = try XCTUnwrap(engine.audioSelectionTask) + engine.stop() + let stoppedGeneration = engine.loadGeneration + await task.value + XCTAssertEqual(engine.loadGeneration, stoppedGeneration) + XCTAssertNil(engine.loadedURL) + XCTAssertEqual(engine.state, .idle) + } + + func testLiveURLDoesNotRequireHostReload() throws { + let engine = try fixture() + defer { engine.stop() } + engine.isLive = true + XCTAssertNil(engine.sessionReloadRefusal) + } +} diff --git a/Tests/AetherEngineTests/BackgroundDecisionWaitsForLoadTests.swift b/Tests/AetherEngineTests/BackgroundDecisionWaitsForLoadTests.swift new file mode 100644 index 000000000..ddf7ac0b8 --- /dev/null +++ b/Tests/AetherEngineTests/BackgroundDecisionWaitsForLoadTests.swift @@ -0,0 +1,139 @@ +import Foundation +import Testing +@testable import AetherEngine + +/// Audit LIF-101: the CORE-2 owed background decision settled on any move out of `.loading`, and a +/// native load passes through `.paused` (the item's readiness, landing inside the tvOS play gate) +/// seconds before it returns. The tvOS teardown that followed bumped the generation under the load, +/// and the host's `load()` threw the `CancellationError` that means "a newer load or stop took +/// over". Sodalite reads it exactly that way: no observers, no start report, a spinner forever. +/// +/// The lifecycle itself is iOS / tvOS only; the owed funnel is not, so these drive it through +/// `noteDidEnterBackground()` and a policy seam that performs the same synchronous teardown. +// Serialized: each test parks a load's reader on a cooperative-pool thread until the test releases +// it, and the release itself needs a pool thread. Run in parallel, these filled CI's four-thread +// pool and stalled the whole swift-testing process for about 100 s. +@Suite("A background decision owed during a load waits for the load to return", .serialized, .timeLimit(.minutes(2))) +@MainActor +struct BackgroundDecisionWaitsForLoadTests { + + /// A fixture served from memory whose first read parks until the test lets it through, so a + /// load can be held in flight for as long as a test needs. + final class HeldDataReader: IOReader, @unchecked Sendable { + private let data: DataIOReader + private let condition = NSCondition() + private var released = false + private var arrivals = 0 + + init(_ bytes: Data) { data = DataIOReader(data: bytes) } + + var entered: Bool { condition.withLock { arrivals > 0 } } + func release() { condition.withLock { released = true; condition.broadcast() } } + + func read(_ buffer: UnsafeMutablePointer?, size: Int32) -> Int32 { + condition.lock() + arrivals += 1 + while !released { condition.wait() } + condition.unlock() + return data.read(buffer, size: size) + } + func seek(offset: Int64, whence: Int32) -> Int64 { data.seek(offset: offset, whence: whence) } + func close() { release() } + func cancel() { release() } + func makeIndependentReader() -> IOReader? { nil } + var discImageProbeEnabled: Bool { false } + } + + /// The states the owed decision was made in, one per decision. + final class PolicyRuns { + var states: [PlaybackState] = [] + } + + private static func installPolicy(on engine: AetherEngine) -> PolicyRuns { + let runs = PolicyRuns() + engine.owedBackgroundPolicyForTesting = { engine in + runs.states.append(engine.state) + engine.releaseVideoPipelineForBackground() + } + return runs + } + + private static func fixture() throws -> Data { try ProbeTestFixtures.hdr10Plus() } + + @Test("A load in flight holds the decision whatever state it passes through") + func settleMatrix() { + #expect(!AetherEngine.backgroundDecisionSettles(state: .paused, loadInFlight: true)) + #expect(!AetherEngine.backgroundDecisionSettles(state: .playing, loadInFlight: true)) + #expect(AetherEngine.backgroundDecisionSettles(state: .paused, loadInFlight: false)) + #expect(AetherEngine.backgroundDecisionSettles(state: .playing, loadInFlight: false)) + #expect(!AetherEngine.backgroundDecisionSettles(state: .loading, loadInFlight: false)) + #expect(!AetherEngine.backgroundDecisionSettles(state: .seeking, loadInFlight: false)) + } + + @Test("load() returns normally when the app backgrounds mid-load, and the teardown follows it (audit LIF-101)") + func loadReturnsThenTheTeardownRuns() async throws { + let engine = try AetherEngine() + defer { engine.stop() } + let runs = Self.installPolicy(on: engine) + let reader = HeldDataReader(try Self.fixture()) + let load = Task { @MainActor in + try await engine.load(source: .custom(reader, formatHint: "mp4")) + } + try await waitFor { reader.entered } + + engine.noteDidEnterBackground() + // The readiness waypoint `.loading -> .paused` the native load passes through mid-flight. + engine.state = .paused + let decidedMidLoad = try await waitFor(upTo: .milliseconds(300)) { !runs.states.isEmpty } + #expect(!decidedMidLoad) + reader.release() + + let probe = try await load.value + #expect(probe != nil) + try await waitFor { !runs.states.isEmpty } + #expect(runs.states == [.playing]) + #expect(engine.state == .paused) + #expect(engine.loadedURL != nil) + } + + @Test("With no load in flight an owed decision still settles on the state edge (audit CORE-2)") + func owedSeekStillSettlesOnTheState() async throws { + let engine = try AetherEngine() + defer { engine.stop() } + _ = try await engine.load(source: .custom(DataIOReader(data: try Self.fixture()), formatHint: "mp4")) + let runs = Self.installPolicy(on: engine) + engine.state = .seeking + + engine.noteDidEnterBackground() + engine.state = .playing + + try await waitFor { !runs.states.isEmpty } + #expect(runs.states == [.playing]) + #expect(engine.state == .paused) + } + + @Test("A PiP window closed in the background during a load tears the session down once it settles (audit LIF-105)") + func pictureInPictureClosedMidLoadIsOwed() async throws { + let engine = try AetherEngine() + defer { engine.stop() } + _ = try await engine.load(source: .custom(DataIOReader(data: try Self.fixture()), formatHint: "mp4")) + let runs = Self.installPolicy(on: engine) + // Backgrounded with a PiP window keeping the session alive, so nothing was owed. + engine.noteDidEnterBackground() + + let reader = HeldDataReader(try Self.fixture()) + let load = Task { @MainActor in + try await engine.load(source: .custom(reader, formatHint: "mp4")) + } + try await waitFor { reader.entered } + engine.settleBackgroundDecisionAfterPictureInPictureClosed() + let decidedMidLoad = try await waitFor(upTo: .milliseconds(300)) { !runs.states.isEmpty } + #expect(!decidedMidLoad) + reader.release() + + _ = try await load.value + try await waitFor { !runs.states.isEmpty } + #expect(runs.states == [.playing]) + #expect(engine.state == .paused) + } +} diff --git a/Tests/AetherEngineTests/BackgroundKeepaliveTests.swift b/Tests/AetherEngineTests/BackgroundKeepaliveTests.swift index 1972f0e37..451d8041a 100644 --- a/Tests/AetherEngineTests/BackgroundKeepaliveTests.swift +++ b/Tests/AetherEngineTests/BackgroundKeepaliveTests.swift @@ -49,9 +49,24 @@ struct BackgroundKeepaliveTests { #expect(AetherEngine.backgroundAction(isAudioBackend: false, hasSoftwareHost: true, keepVideoAlive: false, pipActive: false, state: .paused) == .teardownVideo) } - @Test("do nothing when idle or loading (nothing to tear down)") - func doNothingWhenNotPlayable() { + /// Audit CORE-2: a load or seek in flight DOES have a pipeline (loopback server, AVIO connection, + /// the item). It is not torn down mid-flight, it is judged once it settles, and that judgement is + /// owed rather than skipped: a load finishing after the TV button used to start playing in the + /// background and cross the suspension whole. + @Test("a load or seek in flight is not torn down mid-flight, the decision is owed until it settles") + func loadOrSeekInFlightOwesTheDecision() { #expect(AetherEngine.backgroundAction(isAudioBackend: false, hasSoftwareHost: false, keepVideoAlive: false, pipActive: false, state: .loading) == .doNothing) + #expect(AetherEngine.backgroundAction(isAudioBackend: false, hasSoftwareHost: false, keepVideoAlive: false, pipActive: false, state: .seeking) == .doNothing) + #expect(AetherEngine.backgroundActionIsOwed(state: .loading)) + #expect(AetherEngine.backgroundActionIsOwed(state: .seeking)) + } + + @Test("a settled or idle session owes nothing") + func settledStatesOweNothing() { + #expect(AetherEngine.backgroundAction(isAudioBackend: false, hasSoftwareHost: false, keepVideoAlive: false, pipActive: false, state: .idle) == .doNothing) + for state: PlaybackState in [.idle, .playing, .paused, .ended, .error("x")] { + #expect(!AetherEngine.backgroundActionIsOwed(state: state)) + } } @Test("tvOS: an active PiP window keeps the video pipeline alive") diff --git a/Tests/AetherEngineTests/BoundedFetchTests.swift b/Tests/AetherEngineTests/BoundedFetchTests.swift new file mode 100644 index 000000000..1fe31020d --- /dev/null +++ b/Tests/AetherEngineTests/BoundedFetchTests.swift @@ -0,0 +1,278 @@ +import Darwin +import Foundation +import Testing +@testable import AetherEngine + +/// Audit NET-112 / FEA-107 / DEC-103 / NET-113: segment, key and rendition bodies were read with +/// `session.data(for:)`, which holds the whole body before anyone can look at its size, and the +/// playlist fetch appended one byte at a time. One bounded fetch now serves all of them. +@Suite(.timeLimit(.minutes(2))) +struct BoundedFetchTests { + + private func makeSession() -> URLSession { + let configuration = URLSessionConfiguration.ephemeral + configuration.timeoutIntervalForRequest = 20 + configuration.timeoutIntervalForResource = 60 + return URLSession(configuration: configuration, delegate: EngineTLS.sessionDelegate, delegateQueue: nil) + } + + private func request(_ origin: LoopbackBodyOrigin, _ path: String = "/body") -> URLRequest { + URLRequest(url: URL(string: "http://127.0.0.1:\(origin.port)\(path)")!) + } + + @Test("an endless body with no declared length is cut off at the cap and the connection is dropped") + func endlessBodyIsCutOff() async throws { + let origin = try #require(LoopbackBodyOrigin(body: .endless(chunkBytes: 16 * 1024))) + defer { origin.stop() } + let session = makeSession() + defer { session.invalidateAndCancel() } + let limit = 512 * 1024 + await #expect(throws: BoundedFetch.Exceeded(limit: limit)) { + _ = try await BoundedFetch.data(for: request(origin), session: session, limit: limit) + } + try await waitFor { origin.clientHungUp } + #expect(origin.bytesWritten < 200 * 1024 * 1024) + } + + @Test("a key body is cut off at 64 bytes and a 16 byte key arrives whole") + func keyBodiesAreBounded() async throws { + let endless = try #require(LoopbackBodyOrigin(body: .endless(chunkBytes: 1024))) + let key = try #require(LoopbackBodyOrigin(body: .fixed(16))) + defer { endless.stop(); key.stop() } + let session = makeSession() + defer { session.invalidateAndCancel() } + await #expect(throws: BoundedFetch.Exceeded(limit: BoundedFetch.keyLimit)) { + _ = try await BoundedFetch.data(for: request(endless), session: session, limit: BoundedFetch.keyLimit) + } + let (data, response) = try await BoundedFetch.data( + for: request(key), session: session, limit: BoundedFetch.keyLimit) + #expect(data.count == 16) + #expect((response as? HTTPURLResponse)?.statusCode == 200) + } + + @Test("a declared length over the cap is refused at the response head") + func declaredLengthIsRefusedAtTheHead() async throws { + let origin = try #require(LoopbackBodyOrigin( + body: .endless(chunkBytes: 16 * 1024), declaredLength: 8 * 1024 * 1024 * 1024)) + defer { origin.stop() } + let session = makeSession() + defer { session.invalidateAndCancel() } + await #expect(throws: BoundedFetch.Exceeded(limit: 1024 * 1024)) { + _ = try await BoundedFetch.data(for: request(origin), session: session, limit: 1024 * 1024) + } + try await waitFor { origin.clientHungUp } + #expect(origin.bytesWritten < 16 * 1024 * 1024) + } + + @Test("a refusal comes back with its status and no body") + func refusalKeepsItsStatus() async throws { + let origin = try #require(LoopbackBodyOrigin(body: .endless(chunkBytes: 4096), status: 404)) + defer { origin.stop() } + let session = makeSession() + defer { session.invalidateAndCancel() } + let (data, response) = try await BoundedFetch.data(for: request(origin), session: session, limit: 1024) + #expect((response as? HTTPURLResponse)?.statusCode == 404) + #expect(data.isEmpty) + } + + @Test("a body within the cap arrives whole and in order") + func bodyWithinTheCapArrivesWhole() async throws { + let size = 3 * 1024 * 1024 + 17 + let origin = try #require(LoopbackBodyOrigin(body: .fixed(size), declaredLength: Int64(size))) + defer { origin.stop() } + let session = makeSession() + defer { session.invalidateAndCancel() } + let (data, _) = try await BoundedFetch.data(for: request(origin), session: session, limit: 4 * 1024 * 1024) + #expect(data.count == size) + #expect(data.prefix(4) == Data([0, 1, 2, 3])) + } + + @Test("a cancelled fetch ends instead of waiting for its body") + func cancellationEndsTheFetch() async throws { + let origin = try #require(LoopbackBodyOrigin(body: .endless(chunkBytes: 4096))) + defer { origin.stop() } + let session = makeSession() + defer { session.invalidateAndCancel() } + let task = Task { [request = request(origin)] in + try await BoundedFetch.data(for: request, session: session, limit: 1 << 30) + } + try await waitFor { origin.bytesWritten > 0 } + task.cancel() + await #expect(throws: (any Error).self) { _ = try await task.value } + try await waitFor { origin.clientHungUp } + } + + @Test("the playlist wrapper keeps its error text") + func playlistWrapperKeepsItsError() async throws { + let origin = try #require(LoopbackBodyOrigin(body: .endless(chunkBytes: 1024))) + defer { origin.stop() } + let session = makeSession() + defer { session.invalidateAndCancel() } + await #expect(throws: HLSIngestError.playlistInvalid(reason: "playlist exceeds 4096 bytes")) { + _ = try await BoundedPlaylistFetch.data(for: request(origin), session: session, limit: 4096) + } + } + + @Test("32 MiB of body accumulates in chunks, not byte by byte") + func largeBodyAccumulatesQuickly() async throws { + let size = 32 * 1024 * 1024 + let origin = try #require(LoopbackBodyOrigin(body: .fixed(size), declaredLength: Int64(size))) + defer { origin.stop() } + let session = makeSession() + defer { session.invalidateAndCancel() } + // Measured against `session.data(for:)` on the same origin in the same run, not against a wall + // clock: a loaded suite stretched an absolute sample to 8 s, while the per-byte accumulation this + // replaced ran about 27x slower than the plain fetch under any load (audit NET-113). + var bestPlain = Duration.seconds(3600) + var bestBounded = Duration.seconds(3600) + for _ in 0..<3 { + let plainStart = ContinuousClock.now + let (plain, _) = try await session.data(for: request(origin)) + bestPlain = min(bestPlain, ContinuousClock.now - plainStart) + #expect(plain.count == size) + let boundedStart = ContinuousClock.now + let (data, _) = try await BoundedPlaylistFetch.data(for: request(origin), session: session, limit: 64 * 1024 * 1024) + bestBounded = min(bestBounded, ContinuousClock.now - boundedStart) + #expect(data.count == size) + } + #expect(bestBounded < bestPlain * 4 + .milliseconds(250), "bounded \(bestBounded) vs plain \(bestPlain)") + } + + @Test("a segment may weigh 20 MB/s of its own duration between 32 MiB and 256 MiB") + func segmentLimitFollowsTheDuration() { + let floor = 32 * 1024 * 1024 + let ceiling = 256 * 1024 * 1024 + #expect(BoundedFetch.segmentLimit(forDuration: 0) == floor) + #expect(BoundedFetch.segmentLimit(forDuration: -4) == floor) + #expect(BoundedFetch.segmentLimit(forDuration: .nan) == floor) + #expect(BoundedFetch.segmentLimit(forDuration: .infinity) == floor) + #expect(BoundedFetch.segmentLimit(forDuration: 1) == floor) + #expect(BoundedFetch.segmentLimit(forDuration: 2) == 40_000_000) + #expect(BoundedFetch.segmentLimit(forDuration: 6) == 120_000_000) + // A 10 s UHD remux segment at 125 MB is inside its cap. + #expect(BoundedFetch.segmentLimit(forDuration: 10) >= 125_000_000) + #expect(BoundedFetch.segmentLimit(forDuration: 600) == ceiling) + } +} + +/// A one-request-per-connection loopback origin whose body can be endless, to see a fetch end where +/// its cap says and to count what the origin managed to write before the client hung up. +private final class LoopbackBodyOrigin: @unchecked Sendable { + enum Body { + case endless(chunkBytes: Int) + case fixed(Int) + } + + let port: UInt16 + private let listenFD: Int32 + private let body: Body + private let status: Int + private let declaredLength: Int64? + private let lock = NSLock() + private var _bytesWritten = 0 + private var _clientHungUp = false + private var _connections: Set = [] + + var bytesWritten: Int { lock.withLock { _bytesWritten } } + var clientHungUp: Bool { lock.withLock { _clientHungUp } } + + init?(body: Body, status: Int = 200, declaredLength: Int64? = nil) { + self.body = body + self.status = status + self.declaredLength = declaredLength + let fd = socket(AF_INET, SOCK_STREAM, 0) + guard fd >= 0 else { return nil } + var one: Int32 = 1 + setsockopt(fd, SOL_SOCKET, SO_REUSEADDR, &one, socklen_t(MemoryLayout.size)) + var address = sockaddr_in() + address.sin_family = sa_family_t(AF_INET) + address.sin_port = 0 + address.sin_addr.s_addr = inet_addr("127.0.0.1") + let bound = withUnsafePointer(to: &address) { + $0.withMemoryRebound(to: sockaddr.self, capacity: 1) { + bind(fd, $0, socklen_t(MemoryLayout.size)) + } + } + guard bound == 0, listen(fd, 8) == 0 else { + close(fd) + return nil + } + var length = socklen_t(MemoryLayout.size) + let named = withUnsafeMutablePointer(to: &address) { + $0.withMemoryRebound(to: sockaddr.self, capacity: 1) { getsockname(fd, $0, &length) } + } + guard named == 0 else { + close(fd) + return nil + } + listenFD = fd + port = UInt16(bigEndian: address.sin_port) + Thread.detachNewThread { [self] in acceptLoop() } + } + + func stop() { + lock.lock() + for fd in _connections { shutdown(fd, SHUT_RDWR) } + lock.unlock() + shutdown(listenFD, SHUT_RDWR) + close(listenFD) + } + + private func acceptLoop() { + while true { + let fd = accept(listenFD, nil, nil) + if fd < 0 { return } + var one: Int32 = 1 + setsockopt(fd, SOL_SOCKET, SO_NOSIGPIPE, &one, socklen_t(MemoryLayout.size)) + lock.withLock { _ = _connections.insert(fd) } + Thread.detachNewThread { [self] in serve(fd) } + } + } + + private func serve(_ fd: Int32) { + defer { + lock.withLock { _ = _connections.remove(fd) } + close(fd) + } + var received = Data() + var byte: UInt8 = 0 + while !received.suffix(4).elementsEqual([13, 10, 13, 10]) { + guard recv(fd, &byte, 1, 0) == 1 else { return } + received.append(byte) + } + var head = "HTTP/1.1 \(status) X\r\nConnection: close\r\nContent-Type: video/mp2t\r\n" + if let declaredLength { head += "Content-Length: \(declaredLength)\r\n" } + head += "\r\n" + guard write(fd, Array(head.utf8)) else { return hangUp() } + switch body { + case .endless(let chunkBytes): + let chunk = [UInt8](repeating: 0x55, count: chunkBytes) + while write(fd, chunk) { lock.withLock { _bytesWritten += chunkBytes } } + hangUp() + case .fixed(let count): + // 64 KiB is a multiple of 256, so a repeated ramp stays one continuous ramp. + let ramp = (0..<64 * 1024).map { UInt8(truncatingIfNeeded: $0) } + var remaining = count + while remaining > 0 { + let size = min(remaining, ramp.count) + guard write(fd, size == ramp.count ? ramp : Array(ramp.prefix(size))) else { return hangUp() } + lock.withLock { _bytesWritten += size } + remaining -= size + } + } + } + + private func hangUp() { + lock.withLock { _clientHungUp = true } + } + + private func write(_ fd: Int32, _ bytes: [UInt8]) -> Bool { + var sent = 0 + while sent < bytes.count { + let n = bytes.withUnsafeBytes { send(fd, $0.baseAddress! + sent, bytes.count - sent, 0) } + if n <= 0 { return false } + sent += n + } + return true + } +} diff --git a/Tests/AetherEngineTests/BoundedPlaylistFetchTests.swift b/Tests/AetherEngineTests/BoundedPlaylistFetchTests.swift new file mode 100644 index 000000000..235170499 --- /dev/null +++ b/Tests/AetherEngineTests/BoundedPlaylistFetchTests.swift @@ -0,0 +1,85 @@ +import Foundation +import Testing +@testable import AetherEngine + +/// The playlist caps in the ingest readers and the carriage probe used to be tested after +/// `session.data(for:)` had buffered the whole body, so they bounded nothing (audit NET-10). These +/// pin that the cap now holds while the body arrives, with and without a declared length. +@Suite("Playlist bodies are capped while they arrive", .serialized) +struct BoundedPlaylistFetchTests { + + private static let limit = 4096 + + @Test("A body within the cap arrives whole") + func withinCap() async throws { + let (data, response) = try await fetch(.init(chunks: 3, chunkBytes: 1000, declaresLength: false)) + #expect((response as? HTTPURLResponse)?.statusCode == 200) + #expect(data.count == 3000) + } + + @Test("A declared length over the cap is refused before the body is read") + func declaredOverCap() async { + await #expect(throws: HLSIngestError.playlistInvalid(reason: "playlist exceeds \(Self.limit) bytes")) { + _ = try await fetch(.init(chunks: 8, chunkBytes: 1000, declaresLength: true)) + } + } + + @Test("A body with no stated length is cut off at the cap") + func undeclaredOverCap() async { + await #expect(throws: HLSIngestError.playlistInvalid(reason: "playlist exceeds \(Self.limit) bytes")) { + _ = try await fetch(.init(chunks: 64, chunkBytes: 1000, declaresLength: false)) + } + } + + @Test("A refusal comes back with its status and no body") + func refusalKeepsItsStatus() async throws { + let (data, response) = try await fetch(.init(chunks: 64, chunkBytes: 1000, declaresLength: false, status: 404)) + #expect((response as? HTTPURLResponse)?.statusCode == 404) + #expect(data.isEmpty) + } + + private func fetch(_ shape: StreamingPlaylistProtocol.Shape) async throws -> (Data, URLResponse) { + StreamingPlaylistProtocol.shape = shape + let configuration = URLSessionConfiguration.ephemeral + configuration.protocolClasses = [StreamingPlaylistProtocol.self] + let session = URLSession(configuration: configuration) + defer { session.invalidateAndCancel() } + return try await BoundedPlaylistFetch.data( + for: URLRequest(url: URL(string: "https://origin.test/live.m3u8")!), session: session, + limit: Self.limit) + } +} + +/// Answers every request with `shape`: a body delivered in chunks, with or without a length. +private final class StreamingPlaylistProtocol: URLProtocol, @unchecked Sendable { + struct Shape { + var chunks: Int + var chunkBytes: Int + var declaresLength: Bool + var status = 200 + } + + private static let lock = NSLock() + nonisolated(unsafe) private static var _shape = Shape(chunks: 0, chunkBytes: 0, declaresLength: false) + static var shape: Shape { + get { lock.withLock { _shape } } + set { lock.withLock { _shape = newValue } } + } + + override class func canInit(with request: URLRequest) -> Bool { true } + override class func canonicalRequest(for request: URLRequest) -> URLRequest { request } + override func stopLoading() {} + + override func startLoading() { + let shape = Self.shape + var fields = ["Content-Type": "application/vnd.apple.mpegurl"] + if shape.declaresLength { fields["Content-Length"] = String(shape.chunks * shape.chunkBytes) } + let response = HTTPURLResponse( + url: request.url!, statusCode: shape.status, httpVersion: "HTTP/1.1", headerFields: fields)! + client?.urlProtocol(self, didReceive: response, cacheStoragePolicy: .notAllowed) + for _ in 0 ..< shape.chunks { + client?.urlProtocol(self, didLoad: Data(repeating: 0x23, count: shape.chunkBytes)) + } + client?.urlProtocolDidFinishLoading(self) + } +} diff --git a/Tests/AetherEngineTests/ByteFIFOTests.swift b/Tests/AetherEngineTests/ByteFIFOTests.swift index 96e9cdbdf..76e9f3fd6 100644 --- a/Tests/AetherEngineTests/ByteFIFOTests.swift +++ b/Tests/AetherEngineTests/ByteFIFOTests.swift @@ -61,6 +61,37 @@ final class ByteFIFOTests: XCTestCase { wait(for: [expectation], timeout: 2) } + // audit NET-8: storage moved from one re-based `Data` to a queue of chunks plus a head offset; + // these pin that a read still spans chunk boundaries and resumes mid-chunk correctly. + func testReadSpansMultipleWrites() { + let fifo = ByteFIFO(capacity: 1024) + XCTAssertTrue(fifo.write(Data([1, 2]))) + XCTAssertTrue(fifo.write(Data([3, 4, 5]))) + var buffer = [UInt8](repeating: 0, count: 10) + let n = buffer.withUnsafeMutableBufferPointer { + fifo.read(into: $0.baseAddress!, maxLength: 10) + } + XCTAssertEqual(n, 5) + XCTAssertEqual(Array(buffer[0..<5]), [1, 2, 3, 4, 5]) + } + + func testPartialReadResumesMidChunk() { + let fifo = ByteFIFO(capacity: 1024) + XCTAssertTrue(fifo.write(Data([1, 2, 3, 4, 5, 6]))) + var first = [UInt8](repeating: 0, count: 3) + let n1 = first.withUnsafeMutableBufferPointer { + fifo.read(into: $0.baseAddress!, maxLength: 3) + } + XCTAssertEqual(n1, 3) + XCTAssertEqual(Array(first[0..<3]), [1, 2, 3]) + var second = [UInt8](repeating: 0, count: 3) + let n2 = second.withUnsafeMutableBufferPointer { + fifo.read(into: $0.baseAddress!, maxLength: 3) + } + XCTAssertEqual(n2, 3) + XCTAssertEqual(Array(second[0..<3]), [4, 5, 6]) + } + func testWriteBlocksAtCapacityUntilRead() { let fifo = ByteFIFO(capacity: 4) XCTAssertTrue(fifo.write(Data([1, 2, 3, 4]))) diff --git a/Tests/AetherEngineTests/CLIDocCoverageTests.swift b/Tests/AetherEngineTests/CLIDocCoverageTests.swift new file mode 100644 index 000000000..17a26ad3a --- /dev/null +++ b/Tests/AetherEngineTests/CLIDocCoverageTests.swift @@ -0,0 +1,56 @@ +import Testing +import Foundation + +/// `docs/cli.md` is the only place a flag of `aetherctl` is explained, and three of them (`--assert-dv`, +/// `--max-concurrent-requests`, `--no-blocking-reload`) lived in the parser for a long time before +/// anyone noticed the page never mentioned them (audit OPS-111). Every quoted `"--flag"` literal in +/// `Sources/aetherctl` is a flag the CLI accepts, so each one has to appear in the page. +/// +/// aetherctl is an executable target that no test target can link, so this reads the sources as text. +@Suite("aetherctl flags are documented") +struct CLIDocCoverageTests { + private static let root = URL(fileURLWithPath: #filePath) + .deletingLastPathComponent().deletingLastPathComponent().deletingLastPathComponent() + + private static var checkoutPresent: Bool { + FileManager.default.fileExists(atPath: root.appendingPathComponent("docs/cli.md").path) + && FileManager.default.fileExists(atPath: root.appendingPathComponent("Sources/aetherctl").path) + } + + /// `--help` is the one literal that is a command, not a flag of a subcommand, and the usage text + /// already documents it. + private static let notFlags: Set = ["--help"] + + @Test("every --flag literal in aetherctl appears in docs/cli.md", + .enabled(if: CLIDocCoverageTests.checkoutPresent, "needs a source checkout")) + func everyFlagIsDocumented() throws { + let sourcesDir = Self.root.appendingPathComponent("Sources/aetherctl") + let files = try FileManager.default.contentsOfDirectory(atPath: sourcesDir.path) + .filter { $0.hasSuffix(".swift") } + #expect(!files.isEmpty) + + let literal = try NSRegularExpression(pattern: #""(--[a-z0-9][a-z0-9-]*)""#) + var flags = Set() + for file in files { + let text = try String(contentsOf: sourcesDir.appendingPathComponent(file), encoding: .utf8) + let whole = NSRange(text.startIndex..., in: text) + for match in literal.matches(in: text, range: whole) { + if let range = Range(match.range(at: 1), in: text) { flags.insert(String(text[range])) } + } + } + // A parser that stopped matching would pass vacuously. + #expect(flags.count > 50, "found only \(flags.count) flag literals, the pattern has drifted") + + let doc = try String(contentsOf: Self.root.appendingPathComponent("docs/cli.md"), encoding: .utf8) + let missing = flags.subtracting(Self.notFlags).filter { !Self.mentions(doc, flag: $0) }.sorted() + #expect(missing.isEmpty, "docs/cli.md does not mention: \(missing.joined(separator: ", "))") + } + + /// A whole-flag match, so `--sw` is not satisfied by `--sweep` or `--sw-escalation`. + private static func mentions(_ doc: String, flag: String) -> Bool { + let escaped = NSRegularExpression.escapedPattern(for: flag) + guard let pattern = try? NSRegularExpression(pattern: "(? 0 } } + var wasReleasedByEngine: Bool { condition.withLock { releasedByEngine } } + + func read(_ buffer: UnsafeMutablePointer?, size: Int32) -> Int32 { + condition.lock() + defer { condition.unlock() } + arrivals += 1 + let deadline = Date().addingTimeInterval(fallback) + while !releasedByEngine, condition.wait(until: deadline) {} + return -1 + } + func seek(offset: Int64, whence: Int32) -> Int64 { -1 } + func close() { condition.withLock { releasedByEngine = true; condition.broadcast() } } + func cancel() { close() } + func makeIndependentReader() -> IOReader? { nil } + var discImageProbeEnabled: Bool { false } + } + + enum Outcome: Equatable { case returned, cancelled, failed(String) } + + final class OutcomeBox { + var outcome: Outcome? + } + + private static func start(_ engine: AetherEngine, _ source: MediaSource, + options: LoadOptions = .init()) -> (Task, OutcomeBox) { + let box = OutcomeBox() + let task = Task { @MainActor in + do { + _ = try await engine.load(source: source, options: options) + box.outcome = .returned + } catch is CancellationError { + box.outcome = .cancelled + } catch { + box.outcome = .failed("\(error)") + } + } + return (task, box) + } + + /// How long a cancelled load may take to end. What it guards is the wait it replaced: a silent + /// origin is only given up on by a timeout, the shortest at open being the 5 s HEAD probe and + /// the ingest's 10 s request timeout, so any ceiling below 5 s tells "the cancel ended it" from + /// "a timeout ended it". 1.5 s was a guess at the quiet-machine time and a loaded runner took 2 s. + static let ceiling: Duration = .seconds(4) + + /// Cancels after the load has been held a moment and reports how long it took to end. The wait + /// runs well past `ceiling`, so a slow end reports its real duration instead of the poll's. + private static func cancelAndTime(_ task: Task, _ box: OutcomeBox, + budget: Duration = .seconds(30)) async throws -> Duration { + try await Task.sleep(for: .milliseconds(200)) + let clock = ContinuousClock() + let cancelledAt = clock.now + task.cancel() + _ = try await waitFor(upTo: budget) { box.outcome != nil } + return clock.now - cancelledAt + } + + private static func fixture() throws -> Data { try ProbeTestFixtures.hdr10Plus() } + + @Test("A custom source blocked in its read ends with CancellationError once its task is cancelled") + func customSourceBlockedInRead() async throws { + let engine = try AetherEngine() + defer { engine.stop() } + let reader = ParkedReader() + let (task, box) = Self.start(engine, .custom(reader, formatHint: "mpegts")) + try await waitFor { reader.entered } + + let elapsed = try await Self.cancelAndTime(task, box) + #expect(box.outcome == .cancelled, "ended \(String(describing: box.outcome)) after \(elapsed)") + #expect(elapsed < Self.ceiling) + #expect(reader.wasReleasedByEngine) + await task.value + #expect(engine.state == .idle) + #expect(engine.loadedURL == nil) + } + + @Test("A URL source whose origin never answers ends with CancellationError once its task is cancelled") + func urlSourceWithSilentOrigin() async throws { + let origin = try ProbeHTTPTestOrigin(data: try Self.fixture()) + origin.holdLaterRequests() + defer { origin.stop() } + let engine = try AetherEngine() + defer { engine.stop() } + let url = try #require(URL(string: "http://127.0.0.1:\(origin.port)/silent.mp4")) + let (task, box) = Self.start(engine, .url(url)) + try await waitFor { !origin.requests.isEmpty } + + let elapsed = try await Self.cancelAndTime(task, box) + #expect(box.outcome == .cancelled, "ended \(String(describing: box.outcome)) after \(elapsed)") + #expect(elapsed < Self.ceiling) + origin.stop() + await task.value + #expect(engine.state == .idle) + } + + @Test("A live HLS ingest whose origin never answers ends with CancellationError once its task is cancelled") + func liveIngestWithSilentOrigin() async throws { + let origin = try ProbeHTTPTestOrigin(data: Data("#EXTM3U\n".utf8)) + origin.holdLaterRequests() + defer { origin.stop() } + let engine = try AetherEngine() + defer { engine.stop() } + let url = try #require(URL(string: "http://127.0.0.1:\(origin.port)/live/index.m3u8")) + let reader = HLSLiveIngestReader(playlistURL: url) + let (task, box) = Self.start(engine, .custom(reader, formatHint: "mpegts"), + options: LoadOptions(isLive: true)) + try await waitFor { !origin.requests.isEmpty } + + let elapsed = try await Self.cancelAndTime(task, box) + #expect(box.outcome == .cancelled, "ended \(String(describing: box.outcome)) after \(elapsed)") + #expect(elapsed < Self.ceiling) + origin.stop() + await task.value + #expect(engine.state == .idle) + } + + @Test("A live playlist URL rerouted onto the ingest ends with CancellationError once its task is cancelled") + func liveURLWithSilentOrigin() async throws { + let origin = try ProbeHTTPTestOrigin(data: Data("#EXTM3U\n".utf8)) + origin.holdLaterRequests() + defer { origin.stop() } + let engine = try AetherEngine() + defer { engine.stop() } + let url = try #require(URL(string: "http://127.0.0.1:\(origin.port)/live/index.m3u8")) + var options = LoadOptions(isLive: true) + options.nativeRemoteHLS = false + let (task, box) = Self.start(engine, .url(url), options: options) + try await waitFor { !origin.requests.isEmpty } + + let elapsed = try await Self.cancelAndTime(task, box) + #expect(box.outcome == .cancelled, "ended \(String(describing: box.outcome)) after \(elapsed)") + #expect(elapsed < Self.ceiling) + origin.stop() + await task.value + #expect(engine.state == .idle) + } + + @Test("A load issued after a cancelled one plays normally") + func loadAfterCancelledLoad() async throws { + let engine = try AetherEngine() + defer { engine.stop() } + let reader = ParkedReader() + let (task, box) = Self.start(engine, .custom(reader, formatHint: "mpegts")) + try await waitFor { reader.entered } + try await Task.sleep(for: .milliseconds(200)) + task.cancel() + await task.value + #expect(box.outcome == .cancelled) + + let probe = try await engine.load(source: .custom(DataIOReader(data: try Self.fixture()), formatHint: "mp4")) + #expect(probe != nil) + #expect(engine.loadedURL != nil) + #expect(engine.state != .idle) + #expect(engine.errorInfo == nil) + } + + @Test("Cancelling a load and starting the next in the same turn leaves the next one alone") + func cancelThenLoadInTheSameTurn() async throws { + let engine = try AetherEngine() + defer { engine.stop() } + let reader = ParkedReader() + let (first, firstBox) = Self.start(engine, .custom(reader, formatHint: "mpegts")) + try await waitFor { reader.entered } + + first.cancel() + let (second, secondBox) = Self.start( + engine, .custom(DataIOReader(data: try Self.fixture()), formatHint: "mp4")) + await first.value + await second.value + #expect(firstBox.outcome == .cancelled) + #expect(secondBox.outcome == .returned) + #expect(engine.loadedURL != nil) + #expect(engine.state != .idle) + } + + @Test("A cancellation that lands after a newer load took over does not touch it") + func staleCancellationSparesTheSuccessor() async throws { + let engine = try AetherEngine() + defer { engine.stop() } + let stale = AetherEngine.LoadAttempt() + stale.generation = engine.loadGeneration + let reader = ParkedReader() + let (task, box) = Self.start(engine, .custom(reader, formatHint: "mpegts")) + try await waitFor { reader.entered } + + engine.abandonCancelledLoad(stale) + let touched = try await waitFor(upTo: .milliseconds(300)) { reader.wasReleasedByEngine } + #expect(!touched) + #expect(engine.state == .loading) + task.cancel() + await task.value + #expect(box.outcome == .cancelled) + } + + @Test("A cancellation that lands after the load returned leaves the session playing") + func lateCancellationSparesTheLoadedSession() async throws { + let engine = try AetherEngine() + defer { engine.stop() } + let attempt = AetherEngine.LoadAttempt() + _ = try await engine.load(source: .custom(DataIOReader(data: try Self.fixture()), formatHint: "mp4")) + attempt.generation = engine.loadGeneration + + engine.abandonCancelledLoad(attempt) + #expect(engine.loadedURL != nil) + #expect(engine.state != .idle) + } + + @Test("A load started on an already cancelled task throws without tearing down the running session") + func alreadyCancelledTaskLeavesTheRunningSession() async throws { + let engine = try AetherEngine() + defer { engine.stop() } + _ = try await engine.load(source: .custom(DataIOReader(data: try Self.fixture()), formatHint: "mp4")) + let generation = engine.loadGeneration + let reader = ParkedReader(fallback: 2) + + let task = Task { @MainActor in + withUnsafeCurrentTask { $0?.cancel() } + _ = try await engine.load(source: .custom(reader, formatHint: "mpegts")) + } + await #expect(throws: CancellationError.self) { try await task.value } + #expect(!reader.entered) + #expect(engine.loadGeneration == generation) + #expect(engine.loadedURL != nil) + } + @Test("A plain cancel publishes no error on its way to idle") + func cancelPublishesNoError() async throws { + let engine = try AetherEngine() + defer { engine.stop() } + var states: [PlaybackState] = [] + let sub = engine.$state.sink { states.append($0) } + defer { sub.cancel() } + let reader = ParkedReader() + let (task, box) = Self.start(engine, .custom(reader, formatHint: "mpegts")) + try await waitFor { reader.entered } + task.cancel() + await task.value + + #expect(box.outcome == .cancelled) + #expect(!states.contains { if case .error = $0 { return true } else { return false } }, "\(states)") + #expect(states.last == .idle) + #expect(engine.errorInfo == nil) + } + + @Test("Cancel-then-load keeps the native player the way a newer load would (#15)") + func cancelKeepsTheNativePlayer() async throws { + let engine = try AetherEngine() + defer { engine.stop() } + _ = try await engine.load(source: .custom(DataIOReader(data: try Self.fixture()), formatHint: "mp4")) + let host = try #require(engine.nativeHost) + let player = try #require(engine.currentAVPlayer) + + let reader = ParkedReader() + let (task, box) = Self.start(engine, .custom(reader, formatHint: "mpegts")) + try await waitFor { reader.entered } + task.cancel() + await task.value + #expect(box.outcome == .cancelled) + #expect(engine.nativeHost === host) + #expect(engine.currentAVPlayer === player) + + _ = try await engine.load(source: .custom(DataIOReader(data: try Self.fixture()), formatHint: "mp4")) + #expect(engine.nativeHost === host) + #expect(engine.currentAVPlayer === player) + } + + @Test("Cancelling a load that follows an engine rebuild ends the rebuild's load and throws (AE#629)") + func cancelledFollowerEndsTheRebuild() async throws { + let engine = try AetherEngine() + defer { engine.stop() } + let hostReader = ParkedReader() + let (hostLoad, hostBox) = Self.start(engine, .custom(hostReader, formatHint: "mpegts")) + try await waitFor { hostReader.entered } + let generation = engine.loadGeneration + + // The escalation's rebuild: armed for the live generation, claimed by its own load's teardown. + let rebuildReader = ParkedReader() + let rebuild = Task { @MainActor in + _ = try await engine.load(source: .custom(rebuildReader, formatHint: "mpegts")) + } + engine.softwarePathRebuild = rebuild + engine.softwarePathTakeoverArm = generation + try await waitFor { rebuildReader.entered } + #expect(engine.softwarePathTakeover?.supersededGeneration == generation) + #expect(hostBox.outcome == nil) + + let elapsed = try await Self.cancelAndTime(hostLoad, hostBox) + #expect(hostBox.outcome == .cancelled, "ended \(String(describing: hostBox.outcome)) after \(elapsed)") + #expect(rebuildReader.wasReleasedByEngine) + await #expect(throws: CancellationError.self) { try await rebuild.value } + #expect(engine.state == .idle) + #expect(engine.errorInfo == nil) + } + @Test("A reroute's startup continuation does not outlive a load that was cancelled before it began (#361)") + func cancelledRerouteWithdrawsItsContinuation() async throws { + let engine = try AetherEngine() + defer { engine.stop() } + let before = engine.startupGeneration + let task = Task { @MainActor in + withUnsafeCurrentTask { $0?.cancel() } + engine.continueStartupAcrossReroute() + _ = try await engine.load(source: .custom(ParkedReader(fallback: 2), formatHint: "mpegts")) + } + await #expect(throws: CancellationError.self) { try await task.value } + + _ = try await engine.load(source: .custom(DataIOReader(data: try Self.fixture()), formatHint: "mp4")) + #expect(engine.startupGeneration == before &+ 1) + } + /// A reader whose `cancel()` only unblocks the read in flight (the IOReader contract) and whose + /// `close()` ends it, so a rebuild can park on it again after the probe that preceded it was aborted. + final class RetainedReader: IOReader, @unchecked Sendable { + private let condition = NSCondition() + private var cancels = 0 + private var arrivals = 0 + private var closed = false + + var arrivalCount: Int { condition.withLock { arrivals } } + var isClosed: Bool { condition.withLock { closed } } + + func read(_ buffer: UnsafeMutablePointer?, size: Int32) -> Int32 { + condition.lock() + defer { condition.unlock() } + let epoch = cancels + arrivals += 1 + let deadline = Date().addingTimeInterval(10) + while !closed, cancels == epoch, condition.wait(until: deadline) {} + return -1 + } + func seek(offset: Int64, whence: Int32) -> Int64 { -1 } + func close() { condition.withLock { closed = true; condition.broadcast() } } + func cancel() { condition.withLock { cancels += 1; condition.broadcast() } } + func makeIndependentReader() -> IOReader? { nil } + var discImageProbeEnabled: Bool { false } + } + + /// The custom-source rebuild runs `reloadWithAudioOverride` on the retained reader with its probe + /// detached, which cancelling the rebuild's Task does not reach. Modelled here by that shape. + @Test("Cancelling a follower ends a retained-reader rebuild that Task cancellation cannot reach (AE#629)") + func cancelledFollowerEndsARetainedReaderRebuild() async throws { + let engine = try AetherEngine() + defer { engine.stop() } + let reader = RetainedReader() + let (hostLoad, hostBox) = Self.start(engine, .custom(reader, formatHint: "mpegts")) + try await waitFor { reader.arrivalCount > 0 } + let generation = engine.loadGeneration + + let rebuild = Task { @MainActor in + engine.claimSoftwarePathTakeover() + engine.stopInternal(resetDisplayCriteria: false, keepCustomReader: true) + let gen = engine.loadGeneration + await Task.detached { + var byte: UInt8 = 0 + _ = reader.read(&byte, size: 1) + }.value + try engine.checkLoadCurrent(gen) + } + engine.softwarePathRebuild = rebuild + engine.softwarePathTakeoverArm = generation + try await waitFor { engine.softwarePathTakeover != nil && reader.arrivalCount > 1 } + + let elapsed = try await Self.cancelAndTime(hostLoad, hostBox) + #expect(hostBox.outcome == .cancelled, "ended \(String(describing: hostBox.outcome)) after \(elapsed)") + #expect(elapsed < Self.ceiling) + #expect(reader.isClosed) + await #expect(throws: CancellationError.self) { try await rebuild.value } + await hostLoad.value + #expect(engine.state == .idle) + } + @Test("Cancelling a follower ends a rebuild whose load() rerouted into a nested load (AE#629, AE#678)") + func cancelledFollowerEndsANestedRerouteRebuild() async throws { + let origin = try ProbeHTTPTestOrigin(data: Data("#EXTM3U\n".utf8)) + origin.holdLaterRequests() + defer { origin.stop() } + let engine = try AetherEngine() + defer { engine.stop() } + let hostReader = ParkedReader() + let (hostLoad, hostBox) = Self.start(engine, .custom(hostReader, formatHint: "mpegts")) + try await waitFor { hostReader.entered } + let generation = engine.loadGeneration + + let url = try #require(URL(string: "http://127.0.0.1:\(origin.port)/live/index.m3u8")) + var options = LoadOptions(isLive: true) + options.nativeRemoteHLS = false + let rebuild = Task { @MainActor in + _ = try await engine.load(source: .url(url), options: options) + } + engine.softwarePathRebuild = rebuild + engine.softwarePathTakeoverArm = generation + try await waitFor { !origin.requests.isEmpty } + let takeover = try #require(engine.softwarePathTakeover) + #expect(engine.loadGeneration != takeover.rebuildGeneration) + + let elapsed = try await Self.cancelAndTime(hostLoad, hostBox) + #expect(hostBox.outcome == .cancelled, "ended \(String(describing: hostBox.outcome)) after \(elapsed)") + #expect(elapsed < Self.ceiling) + await #expect(throws: CancellationError.self) { try await rebuild.value } + origin.stop() + await hostLoad.value + #expect(engine.state == .idle) + } + + /// Serves the fixture, and once armed parks any read from the head of the file (where a reopen + /// starts) until the engine cancels or closes it. + final class HeadGatedReader: IOReader, @unchecked Sendable { + private let bytes: Data + private let condition = NSCondition() + private var position = 0 + private var armed = false + private var parkedAtHead = 0 + + init(_ bytes: Data) { self.bytes = bytes } + + func arm() { condition.withLock { armed = true } } + var parkedCount: Int { condition.withLock { parkedAtHead } } + + func read(_ buffer: UnsafeMutablePointer?, size: Int32) -> Int32 { + guard let buffer, size > 0 else { return -1 } + condition.lock() + defer { condition.unlock() } + if armed, position == 0 { + parkedAtHead += 1 + let parked = parkedAtHead + let deadline = Date().addingTimeInterval(10) + while armed, parkedAtHead == parked, condition.wait(until: deadline) {} + return -1 + } + let n = min(Int(size), bytes.count - position) + guard n > 0 else { return -1 } + bytes.withUnsafeBytes { raw in _ = memcpy(buffer, raw.baseAddress! + position, n) } + position += n + return Int32(n) + } + func seek(offset: Int64, whence: Int32) -> Int64 { + condition.withLock { + switch whence { + case 65536: return Int64(bytes.count) + case SEEK_SET: position = Int(max(0, min(offset, Int64(bytes.count)))) + case SEEK_CUR: position = max(0, min(position + Int(offset), bytes.count)) + case SEEK_END: position = max(0, min(bytes.count + Int(offset), bytes.count)) + default: return -1 + } + return Int64(position) + } + } + func close() { condition.withLock { armed = false; condition.broadcast() } } + func cancel() { condition.withLock { parkedAtHead += 1; condition.broadcast() } } + func makeIndependentReader() -> IOReader? { nil } + var discImageProbeEnabled: Bool { false } + } + + @Test("A custom-source reload a stop() supersedes mid-reopen throws CancellationError and publishes no error") + func supersededCustomReloadIsACancellation() async throws { + let engine = try AetherEngine() + defer { engine.stop() } + let reader = HeadGatedReader(try Self.fixture()) + _ = try await engine.load(source: .custom(reader, formatHint: "mp4")) + reader.arm() + let parkedBefore = reader.parkedCount + + let reload = Task { @MainActor in try await engine.reloadAtCurrentPosition() } + try await waitFor { reader.parkedCount > parkedBefore } + engine.stop() + + await #expect(throws: CancellationError.self) { try await reload.value } + #expect(engine.state == .idle) + #expect(engine.errorInfo == nil) + } +} diff --git a/Tests/AetherEngineTests/ChunkedBodyDecoderBoundsTests.swift b/Tests/AetherEngineTests/ChunkedBodyDecoderBoundsTests.swift new file mode 100644 index 000000000..527b5acc5 --- /dev/null +++ b/Tests/AetherEngineTests/ChunkedBodyDecoderBoundsTests.swift @@ -0,0 +1,133 @@ +import Testing +import Foundation +@testable import AetherEngine + +/// Audit DMX-111: the held connection's chunked decoder waited for a CRLF it would never be sent. +/// A size or trailer line with no end made `takeLine` return nil forever, so the caller kept feeding +/// it, the buffer grew at wire rate and every call rescanned all of it. The blank-line and trailer +/// states consumed forever without producing a byte. A line and the framing around a body are +/// bounded now, and a chunk's own body is not. +@Suite("Chunked decoder bounds") +struct ChunkedBodyDecoderBoundsTests { + + private let maxLine = ChunkedBodyDecoder.maxLineBytes + private let maxFraming = ChunkedBodyDecoder.maxFramingBytes + + /// Feeds `wire` in `piece`-sized steps, taking after every step, until `take` throws or the wire + /// is spent. Returns the error, if any, how many bytes had been fed when it came, and the body. + private func run(_ decoder: ChunkedBodyDecoder, _ wire: Data, piece: Int = 1024) + -> (error: ChunkedBodyDecoder.ChunkedError?, fed: Int, body: Data) { + var fed = 0 + var body = Data() + while fed < wire.count { + let end = min(fed + piece, wire.count) + decoder.feed(wire.subdata(in: fed..;`, so the extension pads it to the length under test. + func wire(lineBytes: Int) -> Data { + let head = "5;" + let padding = String(repeating: "e", count: lineBytes - head.count) + return Data((head + padding + "\r\nhello\r\n0\r\n\r\n").utf8) + } + for piece in [1, 7, 1024, 1 << 20] { + let ok = run(ChunkedBodyDecoder(), wire(lineBytes: maxLine), piece: piece) + #expect(ok.error == nil, "a line of \(maxLine) bytes in pieces of \(piece)") + #expect(String(decoding: ok.body, as: UTF8.self) == "hello") + + let tooLong = run(ChunkedBodyDecoder(), wire(lineBytes: maxLine + 1), piece: piece) + #expect(tooLong.error == .lineTooLong, "a line of \(maxLine + 1) bytes in pieces of \(piece)") + } + } + + @Test("the bounds are on framing, not on the body a chunk carries") + func bodyIsNotBounded() { + let payload = Data(repeating: 0x42, count: 3 * 1024 * 1024) + let wire = Data("300000\r\n".utf8) + payload + Data("\r\n0\r\n\r\n".utf8) + let decoder = ChunkedBodyDecoder() + let result = run(decoder, wire, piece: 64 * 1024) + #expect(result.error == nil) + #expect(result.body == payload) + #expect(decoder.isComplete) + } + + @Test("framing that adds up past the bound across many small chunks is fine, because body resets it") + func manySmallChunks() { + let chunkCount = 4 * maxFraming / 6 + var wire = Data() + for _ in 0.. 2 * maxFraming) + let decoder = ChunkedBodyDecoder() + let result = run(decoder, wire) + #expect(result.error == nil) + #expect(result.body.count == chunkCount) + #expect(decoder.isComplete) + } + + @Test("a realistic trailer still ends the body") + func realisticTrailer() { + let trailer = (0..<8).map { "X-Field-\($0): value-\($0)\r\n" }.joined() + let wire = Data(("5\r\nhello\r\n0\r\n" + trailer + "\r\n").utf8) + let decoder = ChunkedBodyDecoder() + let result = run(decoder, wire) + #expect(result.error == nil) + #expect(String(decoding: result.body, as: UTF8.self) == "hello") + #expect(decoder.isComplete) + } +} diff --git a/Tests/AetherEngineTests/ClockTickEngineChangeTests.swift b/Tests/AetherEngineTests/ClockTickEngineChangeTests.swift new file mode 100644 index 000000000..a083cb478 --- /dev/null +++ b/Tests/AetherEngineTests/ClockTickEngineChangeTests.swift @@ -0,0 +1,47 @@ +import Combine +import Testing +@testable import AetherEngine + +/// `player.clock` exists so the ~10 Hz ticks never fire `objectWillChange` on the engine (docs/api.md, +/// Time). `playlistShiftSeconds` is published on the engine, and the tick assigned it on every pass +/// whether it had moved or not, so every SwiftUI view observing the engine re-rendered at 10 Hz again. +@Suite("The clock tick leaves the engine's objectWillChange quiet") +@MainActor +struct ClockTickEngineChangeTests { + + @Test("A shift that did not move is not published again on the next tick (audit PERF-106)") + func steadyShiftIsNotRepublished() throws { + let engine = try AetherEngine() + engine.setPresentationAxis(.anchored(shiftSeconds: 12.5)) + engine.applyNativeHostClockTick(1.0) + #expect(engine.playlistShiftSeconds == 12.5) + + var changes = 0 + let sub = engine.objectWillChange.sink { changes += 1 } + defer { sub.cancel() } + for tick in 1...20 { + engine.applyNativeHostClockTick(1.0 + Double(tick) * 0.1) + } + + #expect(changes == 0) + #expect(engine.playlistShiftSeconds == 12.5) + } + + @Test("A shift that did move is still published") + func movedShiftIsPublished() throws { + let engine = try AetherEngine() + var map = PresentationAxisMap.anchored(shiftSeconds: 12.5) + map.appendSeam(shiftSeconds: 30, activatingAtItemSeconds: 5) + engine.setPresentationAxis(map) + engine.applyNativeHostClockTick(1.0) + #expect(engine.playlistShiftSeconds == 12.5) + + var changes = 0 + let sub = engine.objectWillChange.sink { changes += 1 } + defer { sub.cancel() } + engine.applyNativeHostClockTick(6.0) + + #expect(changes == 1) + #expect(engine.playlistShiftSeconds == 30) + } +} diff --git a/Tests/AetherEngineTests/ColorAttachmentsTests.swift b/Tests/AetherEngineTests/ColorAttachmentsTests.swift new file mode 100644 index 000000000..0f32ee28a --- /dev/null +++ b/Tests/AetherEngineTests/ColorAttachmentsTests.swift @@ -0,0 +1,67 @@ +import CoreVideo +import Testing +import AetherLibavutil +@testable import AetherEngine + +struct ColorAttachmentsTests { + + private func described( + _ primaries: AVColorPrimaries = AVCOL_PRI_UNSPECIFIED, + _ transfer: AVColorTransferCharacteristic = AVCOL_TRC_UNSPECIFIED, + _ matrix: AVColorSpace = AVCOL_SPC_UNSPECIFIED + ) -> ColorDescription { + ColorDescription(primaries: primaries, transfer: transfer, matrix: matrix, range: AVCOL_RANGE_MPEG) + } + + @Test("AE#654: an untagged stream is presented as BT.709 in all three, whatever its size") + func untaggedIsBT709() { + let tags = ColorAttachments.presented(.unspecified) + #expect(tags == ColorAttachments.Tags( + primaries: kCVImageBufferColorPrimaries_ITU_R_709_2, + transfer: kCVImageBufferTransferFunction_ITU_R_709_2, + matrix: kCVImageBufferYCbCrMatrix_ITU_R_709_2)) + } + + @Test("AE#654: a lone BT.601 matrix gets SMPTE-C primaries, as VideoToolbox fills it") + func bt601MatrixImpliesSMPTEC() { + let tags = ColorAttachments.presented(described(AVCOL_PRI_UNSPECIFIED, AVCOL_TRC_UNSPECIFIED, AVCOL_SPC_SMPTE170M)) + #expect(tags.primaries == kCVImageBufferColorPrimaries_SMPTE_C) + #expect(tags.transfer == kCVImageBufferTransferFunction_ITU_R_709_2) + #expect(tags.matrix == kCVImageBufferYCbCrMatrix_ITU_R_601_4) + } + + @Test("AE#654: SD primaries without a matrix get the BT.601 matrix, not BT.709") + func sdPrimariesImplyBT601Matrix() { + #expect(ColorAttachments.presented(described(AVCOL_PRI_BT470BG)).matrix == kCVImageBufferYCbCrMatrix_ITU_R_601_4) + #expect(ColorAttachments.presented(described(AVCOL_PRI_SMPTE170M)).matrix == kCVImageBufferYCbCrMatrix_ITU_R_601_4) + #expect(ColorAttachments.presented(described(AVCOL_PRI_BT2020)).matrix == kCVImageBufferYCbCrMatrix_ITU_R_2020) + } + + @Test("AE#654: a tagged SD stream keeps its tags; before, none of them had a CoreVideo mapping") + func taggedSDIsNotGuessed() { + let pal = ColorAttachments.presented(described(AVCOL_PRI_BT470BG, AVCOL_TRC_SMPTE170M, AVCOL_SPC_BT470BG)) + #expect(pal.primaries == kCVImageBufferColorPrimaries_EBU_3213) + #expect(pal.transfer == kCVImageBufferTransferFunction_ITU_R_709_2) + #expect(pal.matrix == kCVImageBufferYCbCrMatrix_ITU_R_601_4) + } + + @Test("AE#654: a declared HDR description passes through untouched") + func hdrUntouched() { + let tags = ColorAttachments.presented(described(AVCOL_PRI_BT2020, AVCOL_TRC_SMPTE2084, AVCOL_SPC_BT2020_NCL)) + #expect(tags == ColorAttachments.Tags( + primaries: kCVImageBufferColorPrimaries_ITU_R_2020, + transfer: kCVImageBufferTransferFunction_SMPTE_ST_2084_PQ, + matrix: kCVImageBufferYCbCrMatrix_ITU_R_2020)) + } + + @Test("A still's colour space is the one CoreVideo manages the displayed buffer in") + func stillColorSpaceFollowsPlayback() throws { + let untagged = try #require(ColorAttachments.colorSpace(for: ColorAttachments.presented(.unspecified))) + #expect(untagged.name as String? == "kCGColorSpaceCoreMedia709") + let sd = ColorAttachments.presented(described(AVCOL_PRI_SMPTE170M, AVCOL_TRC_SMPTE170M, AVCOL_SPC_SMPTE170M)) + let sdSpace = try #require(ColorAttachments.colorSpace(for: sd)) + #expect(sdSpace != untagged) + let sdr2020 = ColorAttachments.presented(described(AVCOL_PRI_BT2020, AVCOL_TRC_BT2020_10, AVCOL_SPC_BT2020_NCL)) + #expect(ColorAttachments.colorSpace(for: sdr2020) != nil) + } +} diff --git a/Tests/AetherEngineTests/CompositionOffsetRepairFirstSeekTests.swift b/Tests/AetherEngineTests/CompositionOffsetRepairFirstSeekTests.swift new file mode 100644 index 000000000..122a11542 --- /dev/null +++ b/Tests/AetherEngineTests/CompositionOffsetRepairFirstSeekTests.swift @@ -0,0 +1,268 @@ +import Foundation +import AetherLibavcodec +import Testing +@testable import AetherEngine + +/// #699 follow-up: the software host seeks to a resume position before its first read. The repair +/// verdict used to be reached at that first read, so it sampled the landing instead of the head, met +/// an open-GOP keyframe whose picture order is not 0, refused, and the session played in decode +/// order. A seek before the first read now settles the verdict at the head first. +@Suite("Composition-offset repair when the first move is a seek (#699)") +struct CompositionOffsetRepairFirstSeekTests { + + private struct Timestamps: Equatable, CustomStringConvertible { + var pts: Int64 + var dts: Int64 + var description: String { "pts=\(pts) dts=\(dts)" } + } + + /// Opens the fixture, seeks before reading anything (the resume shape), and returns the video + /// timestamps from the landing on. + private static func videoTimestamps( + base64: String, seekingTo seconds: Double, afterReading readFirst: Int = 0 + ) throws -> [Timestamps] { + let data = try #require(Data(base64Encoded: base64, options: .ignoreUnknownCharacters)) + let demuxer = Demuxer() + try demuxer.open(reader: DataIOReader(data: data), formatHint: "mp4") + defer { demuxer.close() } + for _ in 0..? = packet + trackedPacketFree(&owned) + } + return result + } + + @Test("a resume onto the CRA carries the healthy twin's timestamps") + func resumeOntoCRAIsRepaired() throws { + let healthy = try Self.videoTimestamps(base64: Self.healthyFixtureBase64, seekingTo: 0.6) + let repaired = try Self.videoTimestamps(base64: Self.missingFixtureBase64, seekingTo: 0.6) + // The landing is the CRA at 0.6 s (7680 ticks), decoded ahead of its leading pictures. + #expect(healthy.first == Timestamps(pts: 7680, dts: 6144)) + #expect(healthy.contains { $0.pts != $0.dts }) + #expect(repaired == healthy) + } + + /// The native path decides at the head and seeks later. Its landing used to be anchored as if + /// the CRA were the first picture displayed, so the axis sat one slot early per leading picture. + @Test("a seek after playback started counts the landing's leading pictures") + func seekAfterReadingCountsLeadingPictures() throws { + let healthy = try Self.videoTimestamps(base64: Self.healthyFixtureBase64, seekingTo: 0.6, afterReading: 8) + let repaired = try Self.videoTimestamps(base64: Self.missingFixtureBase64, seekingTo: 0.6, afterReading: 8) + #expect(healthy.first == Timestamps(pts: 7680, dts: 6144)) + #expect(repaired == healthy) + } + + @Test("the healthy twin is untouched by the head sample a first seek now takes") + func healthyTwinSurvivesTheHeadSample() throws { + let fromHead = try Self.videoTimestamps(base64: Self.healthyFixtureBase64, seekingTo: 0) + #expect(fromHead.count == 60) + #expect(fromHead.first == Timestamps(pts: 0, dts: -512)) + } + + /// 64x64 HEVC Main 10, 50 fps, 60 frames, four B pictures per hierarchical mini-GOP, and an + /// open-GOP CRA with leading pictures at frame 30. + /// + /// ffmpeg -f lavfi -i 'testsrc=s=64x64:r=50:d=1.2' -frames:v 60 -c:v libx265 -preset ultrafast \ + /// -pix_fmt yuv420p10le -x265-params \ + /// 'bframes=4:b-pyramid=1:b-adapt=0:keyint=30:min-keyint=30:scenecut=0:open-gop=1' \ + /// -tag:v hvc1 -movflags +faststart healthy.mp4 + /// ffmpeg -i healthy.mp4 -map 0:v:0 -c:v copy -bsf:v 'setts=pts=DTS' \ + /// -movflags +faststart missing.mp4 + private static let healthyFixtureBase64 = """ + AAAAHGZ0eXBpc29tAAACAGlzb21pc28ybXA0MQAAD1htb292AAAAbG12aGQAAAAAAAAAAAAAAAAAAAPoAAAEsAABAAABAAAAAAAA + AAAAAAAAAQAAAAAAAAAAAAAAAAAAAAEAAAAAAAAAAAAAAAAAAEAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACAAAOgnRy + YWsAAABcdGtoZAAAAAMAAAAAAAAAAAAAAAEAAAAAAAAEsAAAAAAAAAAAAAAAAAAAAAAAAQAAAAAAAAAAAAAAAAAAAAEAAAAAAAAA + AAAAAAAAAEAAAAAAQAAAAEAAAAAAACRlZHRzAAAAHGVsc3QAAAAAAAAAAQAABLAAAAIAAAEAAAAADfptZGlhAAAAIG1kaGQAAAAA + AAAAAAAAAAAAADIAAAA8AFXEAAAAAAAtaGRscgAAAAAAAAAAdmlkZQAAAAAAAAAAAAAAAFZpZGVvSGFuZGxlcgAAAA2lbWluZgAA + ABR2bWhkAAAAAQAAAAAAAAAAAAAAJGRpbmYAAAAcZHJlZgAAAAAAAAABAAAADHVybCAAAAABAAANZXN0YmwAAAoZc3RzZAAAAAAA + AAABAAAKCWh2YzEAAAAAAAAAAQAAAAAAAAAAAAAAAAAAAAAAQABAAEgAAABIAAAAAAAAAAEVTGF2YzYyLjI4LjEwMSBsaWJ4MjY1 + AAAAAAAAAAAAAAAY//8AAAmFaHZjQwECIAAAAJAAAAAAAB7wAPz9+voAAA8EoAABABhAAQwB//8CIAAAAwCQAAADAAADAB6VmAmh + AAEAKkIBAQIgAAADAJAAAAMAAAMAHqAggQTZZWZKTC8BaAgAAAMACAAAAwGQQKIAAQAGRAHAc8CJJwABCQpOAQX///////////8F + LKLeCbUXR9u7VaT+f8L8TngyNjUgKGJ1aWxkIDIxNikgLSA0LjIrMS1lNDQ0NzQ0OltNYWMgT1MgWF1bY2xhbmcgMjEuMC4wXVs2 + NCBiaXRdIDEwYml0IC0gSC4yNjUvSEVWQyBjb2RlYyAtIENvcHlyaWdodCAyMDEzLTIwMTggKGMpIE11bHRpY29yZXdhcmUsIElu + YyAtIGh0dHA6Ly94MjY1Lm9yZyAtIG9wdGlvbnM6IGNwdWlkPTM0IGZyYW1lLXRocmVhZHM9MSBuby13cHAgbm8tcG1vZGUgbm8t + cG1lIG5vLXBzbnIgbm8tc3NpbSBsb2ctbGV2ZWw9MCBiaXRkZXB0aD0xMCBpbnB1dC1jc3A9MSBmcHM9NTAvMSBpbnB1dC1yZXM9 + NjR4NjQgaW50ZXJsYWNlPTAgdG90YWwtZnJhbWVzPTAgbGV2ZWwtaWRjPTAgaGlnaC10aWVyPTEgdWhkLWJkPTAgcmVmPTEgbm8t + YWxsb3ctbm9uLWNvbmZvcm1hbmNlIG5vLXJlcGVhdC1oZWFkZXJzIGFubmV4YiBuby1hdWQgbm8tZW9iIG5vLWVvcyBuby1ocmQg + aW5mbyBoYXNoPTAgdGVtcG9yYWwtbGF5ZXJzPTAgb3Blbi1nb3AgbWluLWtleWludD0zMCBrZXlpbnQ9MzAgZ29wLWxvb2thaGVh + ZD0wIGJmcmFtZXM9NCBiLWFkYXB0PTAgYi1weXJhbWlkIGJmcmFtZS1iaWFzPTAgcmMtbG9va2FoZWFkPTUgbG9va2FoZWFkLXNs + aWNlcz0wIHNjZW5lY3V0PTAgbm8taGlzdC1zY2VuZWN1dCByYWRsPTAgbm8tc3BsaWNlIG5vLWludHJhLXJlZnJlc2ggY3R1PTMy + IG1pbi1jdS1zaXplPTE2IG5vLXJlY3Qgbm8tYW1wIG1heC10dS1zaXplPTMyIHR1LWludGVyLWRlcHRoPTEgdHUtaW50cmEtZGVw + dGg9MSBsaW1pdC10dT0wIHJkb3EtbGV2ZWw9MCBkeW5hbWljLXJkPTAuMDAgbm8tc3NpbS1yZCBuby1zaWduaGlkZSBuby10c2tp + cCBuci1pbnRyYT0wIG5yLWludGVyPTAgbm8tY29uc3RyYWluZWQtaW50cmEgc3Ryb25nLWludHJhLXNtb290aGluZyBtYXgtbWVy + Z2U9MiBsaW1pdC1yZWZzPTAgbm8tbGltaXQtbW9kZXMgbWU9MCBzdWJtZT0wIG1lcmFuZ2U9NTcgdGVtcG9yYWwtbXZwIG5vLWZy + YW1lLWR1cCBuby1obWUgbm8td2VpZ2h0cCBuby13ZWlnaHRiIG5vLWFuYWx5emUtc3JjLXBpY3MgZGVibG9jaz0wOjAgbm8tc2Fv + IG5vLXNhby1ub24tZGVibG9jayByZD0yIHNlbGVjdGl2ZS1zYW89MCBlYXJseS1za2lwIHJza2lwIGZhc3QtaW50cmEgbm8tdHNr + aXAtZmFzdCBuby1jdS1sb3NzbGVzcyBuby1iLWludHJhIG5vLXNwbGl0cmQtc2tpcCByZHBlbmFsdHk9MCBwc3ktcmQ9Mi4wMCBw + c3ktcmRvcT0wLjAwIG5vLXJkLXJlZmluZSBuby1sb3NzbGVzcyBjYnFwb2Zmcz0wIGNycXBvZmZzPTAgcmM9Y3JmIGNyZj0yOC4w + IHFjb21wPTAuNjAgcXBzdGVwPTQgc3RhdHMtd3JpdGU9MCBzdGF0cy1yZWFkPTAgaXByYXRpbz0xLjQwIHBicmF0aW89MS4zMCBh + cS1tb2RlPTEgYXEtc3RyZW5ndGg9MC4wMCBjdXRyZWUgem9uZS1jb3VudD0wIG5vLXN0cmljdC1jYnIgcWctc2l6ZT0zMiBuby1y + Yy1ncmFpbiBxcG1heD02OSBxcG1pbj0wIG5vLWNvbnN0LXZidiBzYXI9MSBvdmVyc2Nhbj0wIHZpZGVvZm9ybWF0PTUgcmFuZ2U9 + MCBjb2xvcnByaW09MiB0cmFuc2Zlcj0yIGNvbG9ybWF0cml4PTIgY2hyb21hbG9jPTAgZGlzcGxheS13aW5kb3c9MCBjbGw9MCww + IG1pbi1sdW1hPTAgbWF4LWx1bWE9MTAyMyBsb2cyLW1heC1wb2MtbHNiPTggdnVpLXRpbWluZy1pbmZvIHZ1aS1ocmQtaW5mbyBz + bGljZXM9MSBuby1vcHQtcXAtcHBzIG5vLW9wdC1yZWYtbGlzdC1sZW5ndGgtcHBzIG5vLW11bHRpLXBhc3Mtb3B0LXJwcyBzY2Vu + ZWN1dC1iaWFzPTAuMDUgbm8tb3B0LWN1LWRlbHRhLXFwIG5vLWFxLW1vdGlvbiBuby1oZHIxMCBuby1oZHIxMC1vcHQgbm8tZGhk + cjEwLW9wdCBuby1pZHItcmVjb3Zlcnktc2VpIGFuYWx5c2lzLXJldXNlLWxldmVsPTAgYW5hbHlzaXMtc2F2ZS1yZXVzZS1sZXZl + bD0wIGFuYWx5c2lzLWxvYWQtcmV1c2UtbGV2ZWw9MCBzY2FsZS1mYWN0b3I9MCByZWZpbmUtaW50cmE9MCByZWZpbmUtaW50ZXI9 + MCByZWZpbmUtbXY9MSByZWZpbmUtY3R1LWRpc3RvcnRpb249MCBuby1saW1pdC1zYW8gY3R1LWluZm89MCBuby1sb3dwYXNzLWRj + dCByZWZpbmUtYW5hbHlzaXMtdHlwZT0wIGNvcHktcGljPTEgbWF4LWF1c2l6ZS1mYWN0b3I9MS4wIG5vLWR5bmFtaWMtcmVmaW5l + IG5vLXNpbmdsZS1zZWkgbm8taGV2Yy1hcSBuby1zdnQgbm8tZmllbGQgcXAtYWRhcHRhdGlvbi1yYW5nZT0xLjAwIHNjZW5lY3V0 + LWF3YXJlLXFwPTBjb25mb3JtYW5jZS13aW5kb3ctb2Zmc2V0cyByaWdodD0wIGJvdHRvbT0wIGRlY29kZXItbWF4LXJhdGU9MCBu + by12YnYtbGl2ZS1tdWx0aS1wYXNzIG5vLW1jc3RmIG5vLXNicmMgbm8tZnJhbWUtcmOAAAAACmZpZWwBAAAAABBwYXNwAAAAAQAA + AAEAAAAUYnRydAAAAAAAAE6eAAAAAAAAABhzdHRzAAAAAAAAAAEAAAA8AAABAAAAABhzdHNzAAAAAAAAAAIAAAABAAAAGwAAAEhz + ZHRwAAAAACAQEBgYGBAQGBgYEBAYGBgQEBgYGBAQGBgYIBAYGBgQEBgYGBAQGBgYEBAYGBgQEBgYGBAQGBgYEBAYGAAAAZhjdHRz + AAAAAAAAADEAAAABAAACAAAAAAEAAAYAAAAAAQAAAwAAAAACAAAAAAAAAAEAAAEAAAAAAQAABgAAAAABAAADAAAAAAIAAAAAAAAA + AQAAAQAAAAABAAAGAAAAAAEAAAMAAAAAAgAAAAAAAAABAAABAAAAAAEAAAYAAAAAAQAAAwAAAAACAAAAAAAAAAEAAAEAAAAAAQAA + BgAAAAABAAADAAAAAAIAAAAAAAAAAQAAAQAAAAABAAAGAAAAAAEAAAMAAAAAAgAAAAAAAAABAAABAAAAAAEAAAYAAAAAAQAAAwAA + AAACAAAAAAAAAAEAAAEAAAAAAQAABgAAAAABAAADAAAAAAIAAAAAAAAAAQAAAQAAAAABAAAGAAAAAAEAAAMAAAAAAgAAAAAAAAAB + AAABAAAAAAEAAAYAAAAAAQAAAwAAAAACAAAAAAAAAAEAAAEAAAAAAQAABgAAAAABAAADAAAAAAIAAAAAAAAAAQAAAQAAAAABAAAF + AAAAAAEAAAIAAAAAAQAAAAAAAAABAAABAAAAABxzdHNjAAAAAAAAAAEAAAABAAAAPAAAAAEAAAEEc3RzegAAAAAAAAAAAAAAPAAA + BBUAAAAVAAAADgAAAA8AAAAPAAAADgAAACwAAAAPAAAAEAAAABAAAAAPAAAALAAAABAAAAAQAAAAEAAAAA8AAAA6AAAAEAAAABAA + AAAQAAAADwAAAC4AAAAQAAAAEAAAABAAAAAPAAADAwAAABAAAAAQAAAAEAAAAA8AAAAZAAAADgAAAA8AAAAPAAAADgAAAC4AAAAP + AAAAEAAAABAAAAAPAAAAMAAAABAAAAAQAAAAEAAAAA8AAAA1AAAAEAAAABAAAAAQAAAADwAAACwAAAAQAAAAEAAAABAAAAAPAAAA + LwAAABAAAAAPAAAADwAAABRzdGNvAAAAAAAAAAEAAA+EAAAAYnVkdGEAAABabWV0YQAAAAAAAAAhaGRscgAAAAAAAAAAbWRpcmFw + cGwAAAAAAAAAAAAAAAAtaWxzdAAAACWpdG9vAAAAHWRhdGEAAAABAAAAAExhdmY2Mi4xMi4xMDEAAAAIZnJlZQAAC9NtZGF0AAAE + ESgBrCmAqAtj3ixvvtxkKmMD3eX+ImIdIq0/LQeyKklx7fWmEt6THfOXuqnxTy5j68zrVuZCkBZyAW3NTMnVjDWR58jaXrdK4MTV + 4zZD4kgnUXLTiZjW6GVzt8f/1mmD9wKt8fsbkIZldLLRz2XXJreSW52v0Hs30w1FCqCcD0vuQ5liavcsOoD3gsstUZYvgeFGsWtL + bFSkSDeWJ7BcAiN/z9yxYVkNWrpWVYkKE/+JabKF0F11CrKkNRv45k/fLEamFfTqAI5vBqQVIqE5o925E14g4ZQF+nWtk1qaMRWT + qnDREz035U3J2USY03KgMr2qGDDxxTMsFw6tOV+ipcPuS1AtJvyEaNS6cABF+nUKnegDInOgCmVCPWZ5u75iUFCZTzEwVncVIlXZ + eRYVLfgLpMWt2RfKQEFuapdJSWuwwjb8LamSib4jum3wDfrVdM2j0bJAnnZvDO6Iy3yaeP26GhKv///ba5Q48eqAjXtiPE4j2Pwo + A+qlwxf9TbPz479o2DJDzfm7kSBtTxJVt+yoyuVS0iPHwUvyMDZn1szJSSHzD6EJt8zAaLmgCQnlDmB0xPcZdikT8DnLOkwnEltF + QkIXnxUpaJgcASICN1J0EM9R8lVZjK+rTElRT3tsykgs7QSpDt7Mf0S5PPEgNHUwmMMHn6Gs2k1zpXQRD60DtcgMWfoqprALJcjZ + onISd/e6G+pVlY+ByiKB+XTN4QPK64Tqk0ObanoRNqWoz3lZ62JcZpR/NYLaYvTE+mwNemnifXmNoHT9wPkC6zyzE4B7IKmt5Oq5 + DjqcqxdeUrMFvSTc3q2TgkaX6sOQF6estWisjYbGFCbA38FpEURoFpQuc8wdY8uYDhbjCCDNOxMOnijsFNrXpYhUs3ZNTAx13AMC + Phd72hbDRGCZpO80Z4fO937bk4WUMdvK8JZrn+PgG/Kr+mprjHf7AvP88YXxdFGwkEhDTYHg8wmVU4/AfE9AWVKChz5//ynZsaMo + pC23dnt7BCckf9BzoZDmZxCsyyKn5XCjddEl+W4I/UO2rdWQjjIUOQM0vhfoi2+KrC0CUHiAKZ3ag9x+y5cNeHN09L2S9XAALIkr + OchQrgsB2AvE/AO1nQa0DOwLPAaQWYEgJV5yTe08S371798MB1SrF2jU88SIleLNF7cob3v01E+ibmbQGTCd3FsV7CZbspvs9XOq + aAUf2jJ1ksoU7qji2v+b1OON7e2ODMFaXuoAcqZoF/bgiZggZJnvHretRW6ZGfjIHOAECVXpwXHuW/MgEFh1necwZOM0bti9FSvb + 7kru2YB2KOdVMc8DEAjSUO2vdmU4ShS8g5hU+YKnpbWHyI5Gt7hyQ8bo1xrHMBT00EY+Cyn8hUbj8t8QkrEvGDGJvgAAABECAdAp + S4gUQLNaRKOrvmUrlwAAAAoCAeBknWCBZJJ4AAAACwAB4CT1XokDGJDwAAAACwAB4ETXXokDCJDwAAAACgAB4Ia34IGMkPAAAAAo + AgHQUJLV4gUQs1pBXstflTAR3T1JbAVIbopYqDUPYTj4IWE81h/fcAAAAAsCAeECJ1dYIFmSeAAAAAwAAeDG9VXokDGAkPAAAAAM + AAHg5tV16JAwgJDwAAAACwAB4SIt14IGMJDwAAAAKAIB0Hiy1dYgUbNX3UUiza+6i4MPZOqs+PtC41JPtDUUci+9EDsBEIAAAAAM + AgHhoidS1ggWQJJ4AAAADAAB4Wb1VeiQMYCQ8AAAAAwAAeGG1XXokDCAkPAAAAALAAHhwi3XggYwkPAAAAA2AgHQoLLV1iBRs1pF + 8J9CPc4R37vjlJUMsDOIYRTv0YJs83cr9E8qQK09NOxkCdS4zy+1btkwAAAADAIB4kInUtYIFsCSeAAAAAwAAeIG9VXokDCAkPAA + AAAMAAHiJtV16JAxgJDwAAAACwAB4mIt14IGMJDwAAAAKgIB0Miy1dYgU7NaRdlMD+s+HomYswol7sg/lZAGfD5iKIaClD9WuaxS + 2AAAAAwCAeLiJ1LWCBZAkngAAAAMAAHipvVV6JAwgJDwAAAADAAB4sbVdeiQMYCQ8AAAAAsAAeMCLdeCBjCQ8AAAAv8qAax4WUjJ + CWCjD2eNFYBL9VgaG1nDmWYZT44HnoNrRoPGSdWf2Hq+kAVp0AETi8bXaTSzsF+JUlDhD3/F6v4RJdJ0nr//gzctK9Kf0HYjnaTZ + QMpnbQiyS3n232faVg21N8KsG1ZNqscyqiiIt+wc12DDVvXwz86qXsby+oe0oXOGHLAA6ze7alSy/+qrwwd7NVYDo6yNBZR23bKp + M5r65RAYpV46uGId2xk+8hFx+DOKF9516y9z2Sc00Ri24Iq8OdCVWNvHWKPB635A0pBuAawUdfGgyCjIPQh1+ZsBXLhT41eJcSlV + UWuNOqZIyerG7zNStDP2dylZqCW9Ul3XmIVPSEc2f//1fOq+gXwDQffb0/SFIXZLPKzHdWYD8RUI1pDR0l5ExhcvZ/zMdpZ4t8Rb + 3QIVCbN0yjPy1vJ4jjIoHM1NcKMLIDnBVOh+zt3QM3leO+4ZFjjf8r6ocDE3KoN2gNvpPm5L7CIhQ2ifdpvoTgBmjVh6cTm/Wf49 + 1E5/q/5y8gPCFZxtlbEBQZkn2j7DPTns4UYiJ1uD4UzOKIMq4i+rfpo8hTFNA8HtVN5lJ6Tv0c9CPeh5uAdwwKVrbaPtiazzMXZ8 + jOAfZ6hngSyJI4nhYaAOHW/sy5dm1McbDzOwuZ1T0nPhGqyWqtEvQ69uxZZZNkWS55TB8snliUp4sKy1/uLfA8tGJUBuumU7yJbO + xPgOpaR06JWocWRaho4OonanCAUCSTLLif+PF4HqU6crLSctC7oafi+qctoINabIeRpCshIeTFlTDX7bdZIBUWd74n7FdxIpmLzt + JCcvZyMLQNpn1zHi6WV4IBFALtCCOBSJ2P/TcS9/Z0C/BRexcGZFe88yAbVa+15bZQ44+t92JW+PlM3cDHMdP+cGWxf7QdajToUs + mR17uo1Y6BteMHVLrjVz37Mu4OZtaIqp1LwfD/Gl0BAF8ph1b9l6rYoUppu2nj1CrHd/+0kJgooDopxHesfYiVHVpsapV0t6aHAr + jyCdhMCBL/e4CAAAAAwSAeOCJ1LWCBbAkngAAAAMEAHjRvVV6JAxgJDwAAAADBAB42bVdeiQMYCQ8AAAAAsQAeOiLdeCBhCQ8AAA + ABUCAdEZS4gUwKWUARtgIeLbo2Dsm4AAAAAKAgHkJJ1ggWSSeAAAAAsAAePk9V6JAxiQ8AAAAAsAAeQE116JAxiQ8AAAAAoAAeRG + t+CBhJDwAAAAKgIB0UCS1eIFMLNaRZJaHzbJq9zDac6CVeR4Ose49AXhreJ5h3CAkxnAgAAAAAsCAeTCJ1dYIFmSeAAAAAwAAeSG + 9VXokDCAkPAAAAAMAAHkptV16JAxgJDwAAAACwAB5OIt14IGEJDwAAAALAIB0Wiy1dYgU7NX0oxEpyVXkd0PlmoD4UCzwOy1Kn4H + M30P7lZDezVWpePWAAAADAIB5WInUtYIFkCSeAAAAAwAAeUm9VXokDCAkPAAAAAMAAHlRtV16JAxgJDwAAAACwAB5YIt14IGEJDw + AAAAMQIB0ZCy1dYgU7NaQVA7FgvGoiPkdZiJRmkpLH4HCwR/7s/PnTGo0asEbqN5ii4n+OAAAAAMAgHmAidS1ggWwJJ4AAAADAAB + 5cb1VeiQMICQ8AAAAAwAAeXm1XXokDCAkPAAAAALAAHmIi3XggYwkPAAAAAoAgHRuLLV1iBTs1pIU6hYy2wO0STGAgb99r9Labtc + ybfbnmQ6/OsmvgAAAAwCAeaiJ1LWCBbAkngAAAAMAAHmZvVV6JAwgJDwAAAADAAB5obVdeiQMICQ8AAAAAsAAebCLdeCBhCQ8AAA + ACsCAdHYslXWIFOtmAzX0NRed3pQXw+62wWHkLbUGOmJJygg3cAOzHxoBHGAAAAADAIB5yIlUtYIFsCSeAAAAAsAAecG9deiQMaQ + 8AAAAAsAAedCLVeCBjCQ8A== + """ + + private static let missingFixtureBase64 = """ + AAAAHGZ0eXBpc29tAAACAGlzb21pc28ybXA0MQAADcBtb292AAAAbG12aGQAAAAAAAAAAAAAAAAAAAPoAAAEiAABAAABAAAAAAAA + AAAAAAAAAQAAAAAAAAAAAAAAAAAAAAEAAAAAAAAAAAAAAAAAAEAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACAAAM6nRy + YWsAAABcdGtoZAAAAAMAAAAAAAAAAAAAAAEAAAAAAAAEiAAAAAAAAAAAAAAAAAAAAAAAAQAAAAAAAAAAAAAAAAAAAAEAAAAAAAAA + AAAAAAAAAEAAAAAAQAAAAEAAAAAAACRlZHRzAAAAHGVsc3QAAAAAAAAAAQAABIgAAAIAAAEAAAAADGJtZGlhAAAAIG1kaGQAAAAA + AAAAAAAAAAAAADIAAAA8AFXEAAAAAAAtaGRscgAAAAAAAAAAdmlkZQAAAAAAAAAAAAAAAFZpZGVvSGFuZGxlcgAAAAwNbWluZgAA + ABR2bWhkAAAAAQAAAAAAAAAAAAAAJGRpbmYAAAAcZHJlZgAAAAAAAAABAAAADHVybCAAAAABAAALzXN0YmwAAAoZc3RzZAAAAAAA + AAABAAAKCWh2YzEAAAAAAAAAAQAAAAAAAAAAAAAAAAAAAAAAQABAAEgAAABIAAAAAAAAAAEVTGF2YzYyLjI4LjEwMSBsaWJ4MjY1 + AAAAAAAAAAAAAAAY//8AAAmFaHZjQwECIAAAAJAAAAAAAB7wAPz9+voAAA8EoAABABhAAQwB//8CIAAAAwCQAAADAAADAB6VmAmh + AAEAKkIBAQIgAAADAJAAAAMAAAMAHqAggQTZZWZKTC8BaAgAAAMACAAAAwGQQKIAAQAGRAHAc8CJJwABCQpOAQX///////////8F + LKLeCbUXR9u7VaT+f8L8TngyNjUgKGJ1aWxkIDIxNikgLSA0LjIrMS1lNDQ0NzQ0OltNYWMgT1MgWF1bY2xhbmcgMjEuMC4wXVs2 + NCBiaXRdIDEwYml0IC0gSC4yNjUvSEVWQyBjb2RlYyAtIENvcHlyaWdodCAyMDEzLTIwMTggKGMpIE11bHRpY29yZXdhcmUsIElu + YyAtIGh0dHA6Ly94MjY1Lm9yZyAtIG9wdGlvbnM6IGNwdWlkPTM0IGZyYW1lLXRocmVhZHM9MSBuby13cHAgbm8tcG1vZGUgbm8t + cG1lIG5vLXBzbnIgbm8tc3NpbSBsb2ctbGV2ZWw9MCBiaXRkZXB0aD0xMCBpbnB1dC1jc3A9MSBmcHM9NTAvMSBpbnB1dC1yZXM9 + NjR4NjQgaW50ZXJsYWNlPTAgdG90YWwtZnJhbWVzPTAgbGV2ZWwtaWRjPTAgaGlnaC10aWVyPTEgdWhkLWJkPTAgcmVmPTEgbm8t + YWxsb3ctbm9uLWNvbmZvcm1hbmNlIG5vLXJlcGVhdC1oZWFkZXJzIGFubmV4YiBuby1hdWQgbm8tZW9iIG5vLWVvcyBuby1ocmQg + aW5mbyBoYXNoPTAgdGVtcG9yYWwtbGF5ZXJzPTAgb3Blbi1nb3AgbWluLWtleWludD0zMCBrZXlpbnQ9MzAgZ29wLWxvb2thaGVh + ZD0wIGJmcmFtZXM9NCBiLWFkYXB0PTAgYi1weXJhbWlkIGJmcmFtZS1iaWFzPTAgcmMtbG9va2FoZWFkPTUgbG9va2FoZWFkLXNs + aWNlcz0wIHNjZW5lY3V0PTAgbm8taGlzdC1zY2VuZWN1dCByYWRsPTAgbm8tc3BsaWNlIG5vLWludHJhLXJlZnJlc2ggY3R1PTMy + IG1pbi1jdS1zaXplPTE2IG5vLXJlY3Qgbm8tYW1wIG1heC10dS1zaXplPTMyIHR1LWludGVyLWRlcHRoPTEgdHUtaW50cmEtZGVw + dGg9MSBsaW1pdC10dT0wIHJkb3EtbGV2ZWw9MCBkeW5hbWljLXJkPTAuMDAgbm8tc3NpbS1yZCBuby1zaWduaGlkZSBuby10c2tp + cCBuci1pbnRyYT0wIG5yLWludGVyPTAgbm8tY29uc3RyYWluZWQtaW50cmEgc3Ryb25nLWludHJhLXNtb290aGluZyBtYXgtbWVy + Z2U9MiBsaW1pdC1yZWZzPTAgbm8tbGltaXQtbW9kZXMgbWU9MCBzdWJtZT0wIG1lcmFuZ2U9NTcgdGVtcG9yYWwtbXZwIG5vLWZy + YW1lLWR1cCBuby1obWUgbm8td2VpZ2h0cCBuby13ZWlnaHRiIG5vLWFuYWx5emUtc3JjLXBpY3MgZGVibG9jaz0wOjAgbm8tc2Fv + IG5vLXNhby1ub24tZGVibG9jayByZD0yIHNlbGVjdGl2ZS1zYW89MCBlYXJseS1za2lwIHJza2lwIGZhc3QtaW50cmEgbm8tdHNr + aXAtZmFzdCBuby1jdS1sb3NzbGVzcyBuby1iLWludHJhIG5vLXNwbGl0cmQtc2tpcCByZHBlbmFsdHk9MCBwc3ktcmQ9Mi4wMCBw + c3ktcmRvcT0wLjAwIG5vLXJkLXJlZmluZSBuby1sb3NzbGVzcyBjYnFwb2Zmcz0wIGNycXBvZmZzPTAgcmM9Y3JmIGNyZj0yOC4w + IHFjb21wPTAuNjAgcXBzdGVwPTQgc3RhdHMtd3JpdGU9MCBzdGF0cy1yZWFkPTAgaXByYXRpbz0xLjQwIHBicmF0aW89MS4zMCBh + cS1tb2RlPTEgYXEtc3RyZW5ndGg9MC4wMCBjdXRyZWUgem9uZS1jb3VudD0wIG5vLXN0cmljdC1jYnIgcWctc2l6ZT0zMiBuby1y + Yy1ncmFpbiBxcG1heD02OSBxcG1pbj0wIG5vLWNvbnN0LXZidiBzYXI9MSBvdmVyc2Nhbj0wIHZpZGVvZm9ybWF0PTUgcmFuZ2U9 + MCBjb2xvcnByaW09MiB0cmFuc2Zlcj0yIGNvbG9ybWF0cml4PTIgY2hyb21hbG9jPTAgZGlzcGxheS13aW5kb3c9MCBjbGw9MCww + IG1pbi1sdW1hPTAgbWF4LWx1bWE9MTAyMyBsb2cyLW1heC1wb2MtbHNiPTggdnVpLXRpbWluZy1pbmZvIHZ1aS1ocmQtaW5mbyBz + bGljZXM9MSBuby1vcHQtcXAtcHBzIG5vLW9wdC1yZWYtbGlzdC1sZW5ndGgtcHBzIG5vLW11bHRpLXBhc3Mtb3B0LXJwcyBzY2Vu + ZWN1dC1iaWFzPTAuMDUgbm8tb3B0LWN1LWRlbHRhLXFwIG5vLWFxLW1vdGlvbiBuby1oZHIxMCBuby1oZHIxMC1vcHQgbm8tZGhk + cjEwLW9wdCBuby1pZHItcmVjb3Zlcnktc2VpIGFuYWx5c2lzLXJldXNlLWxldmVsPTAgYW5hbHlzaXMtc2F2ZS1yZXVzZS1sZXZl + bD0wIGFuYWx5c2lzLWxvYWQtcmV1c2UtbGV2ZWw9MCBzY2FsZS1mYWN0b3I9MCByZWZpbmUtaW50cmE9MCByZWZpbmUtaW50ZXI9 + MCByZWZpbmUtbXY9MSByZWZpbmUtY3R1LWRpc3RvcnRpb249MCBuby1saW1pdC1zYW8gY3R1LWluZm89MCBuby1sb3dwYXNzLWRj + dCByZWZpbmUtYW5hbHlzaXMtdHlwZT0wIGNvcHktcGljPTEgbWF4LWF1c2l6ZS1mYWN0b3I9MS4wIG5vLWR5bmFtaWMtcmVmaW5l + IG5vLXNpbmdsZS1zZWkgbm8taGV2Yy1hcSBuby1zdnQgbm8tZmllbGQgcXAtYWRhcHRhdGlvbi1yYW5nZT0xLjAwIHNjZW5lY3V0 + LWF3YXJlLXFwPTBjb25mb3JtYW5jZS13aW5kb3ctb2Zmc2V0cyByaWdodD0wIGJvdHRvbT0wIGRlY29kZXItbWF4LXJhdGU9MCBu + by12YnYtbGl2ZS1tdWx0aS1wYXNzIG5vLW1jc3RmIG5vLXNicmMgbm8tZnJhbWUtcmOAAAAACmZpZWwBAAAAABBwYXNwAAAAAQAA + AAEAAAAUYnRydAAAAAAAAE6eAABOngAAABhzdHRzAAAAAAAAAAEAAAA8AAABAAAAABhzdHNzAAAAAAAAAAIAAAABAAAAGwAAAEhz + ZHRwAAAAACAQEBgYGBAQGBgYEBAYGBgQEBgYGBAQGBgYIBAYGBgQEBgYGBAQGBgYEBAYGBgQEBgYGBAQGBgYEBAYGAAAABxzdHNj + AAAAAAAAAAEAAAABAAAAPAAAAAEAAAEEc3RzegAAAAAAAAAAAAAAPAAABBUAAAAVAAAADgAAAA8AAAAPAAAADgAAACwAAAAPAAAA + EAAAABAAAAAPAAAALAAAABAAAAAQAAAAEAAAAA8AAAA6AAAAEAAAABAAAAAQAAAADwAAAC4AAAAQAAAAEAAAABAAAAAPAAADAwAA + ABAAAAAQAAAAEAAAAA8AAAAZAAAADgAAAA8AAAAPAAAADgAAAC4AAAAPAAAAEAAAABAAAAAPAAAAMAAAABAAAAAQAAAAEAAAAA8A + AAA1AAAAEAAAABAAAAAQAAAADwAAACwAAAAQAAAAEAAAABAAAAAPAAAALwAAABAAAAAPAAAADwAAABRzdGNvAAAAAAAAAAEAAA3s + AAAAYnVkdGEAAABabWV0YQAAAAAAAAAhaGRscgAAAAAAAAAAbWRpcmFwcGwAAAAAAAAAAAAAAAAtaWxzdAAAACWpdG9vAAAAHWRh + dGEAAAABAAAAAExhdmY2Mi4xMi4xMDEAAAAIZnJlZQAAC9NtZGF0AAAEESgBrCmAqAtj3ixvvtxkKmMD3eX+ImIdIq0/LQeyKklx + 7fWmEt6THfOXuqnxTy5j68zrVuZCkBZyAW3NTMnVjDWR58jaXrdK4MTV4zZD4kgnUXLTiZjW6GVzt8f/1mmD9wKt8fsbkIZldLLR + z2XXJreSW52v0Hs30w1FCqCcD0vuQ5liavcsOoD3gsstUZYvgeFGsWtLbFSkSDeWJ7BcAiN/z9yxYVkNWrpWVYkKE/+JabKF0F11 + CrKkNRv45k/fLEamFfTqAI5vBqQVIqE5o925E14g4ZQF+nWtk1qaMRWTqnDREz035U3J2USY03KgMr2qGDDxxTMsFw6tOV+ipcPu + S1AtJvyEaNS6cABF+nUKnegDInOgCmVCPWZ5u75iUFCZTzEwVncVIlXZeRYVLfgLpMWt2RfKQEFuapdJSWuwwjb8LamSib4jum3w + DfrVdM2j0bJAnnZvDO6Iy3yaeP26GhKv///ba5Q48eqAjXtiPE4j2PwoA+qlwxf9TbPz479o2DJDzfm7kSBtTxJVt+yoyuVS0iPH + wUvyMDZn1szJSSHzD6EJt8zAaLmgCQnlDmB0xPcZdikT8DnLOkwnEltFQkIXnxUpaJgcASICN1J0EM9R8lVZjK+rTElRT3tsykgs + 7QSpDt7Mf0S5PPEgNHUwmMMHn6Gs2k1zpXQRD60DtcgMWfoqprALJcjZonISd/e6G+pVlY+ByiKB+XTN4QPK64Tqk0ObanoRNqWo + z3lZ62JcZpR/NYLaYvTE+mwNemnifXmNoHT9wPkC6zyzE4B7IKmt5Oq5DjqcqxdeUrMFvSTc3q2TgkaX6sOQF6estWisjYbGFCbA + 38FpEURoFpQuc8wdY8uYDhbjCCDNOxMOnijsFNrXpYhUs3ZNTAx13AMCPhd72hbDRGCZpO80Z4fO937bk4WUMdvK8JZrn+PgG/Kr + +mprjHf7AvP88YXxdFGwkEhDTYHg8wmVU4/AfE9AWVKChz5//ynZsaMopC23dnt7BCckf9BzoZDmZxCsyyKn5XCjddEl+W4I/UO2 + rdWQjjIUOQM0vhfoi2+KrC0CUHiAKZ3ag9x+y5cNeHN09L2S9XAALIkrOchQrgsB2AvE/AO1nQa0DOwLPAaQWYEgJV5yTe08S371 + 798MB1SrF2jU88SIleLNF7cob3v01E+ibmbQGTCd3FsV7CZbspvs9XOqaAUf2jJ1ksoU7qji2v+b1OON7e2ODMFaXuoAcqZoF/bg + iZggZJnvHretRW6ZGfjIHOAECVXpwXHuW/MgEFh1necwZOM0bti9FSvb7kru2YB2KOdVMc8DEAjSUO2vdmU4ShS8g5hU+YKnpbWH + yI5Gt7hyQ8bo1xrHMBT00EY+Cyn8hUbj8t8QkrEvGDGJvgAAABECAdApS4gUQLNaRKOrvmUrlwAAAAoCAeBknWCBZJJ4AAAACwAB + 4CT1XokDGJDwAAAACwAB4ETXXokDCJDwAAAACgAB4Ia34IGMkPAAAAAoAgHQUJLV4gUQs1pBXstflTAR3T1JbAVIbopYqDUPYTj4 + IWE81h/fcAAAAAsCAeECJ1dYIFmSeAAAAAwAAeDG9VXokDGAkPAAAAAMAAHg5tV16JAwgJDwAAAACwAB4SIt14IGMJDwAAAAKAIB + 0Hiy1dYgUbNX3UUiza+6i4MPZOqs+PtC41JPtDUUci+9EDsBEIAAAAAMAgHhoidS1ggWQJJ4AAAADAAB4Wb1VeiQMYCQ8AAAAAwA + AeGG1XXokDCAkPAAAAALAAHhwi3XggYwkPAAAAA2AgHQoLLV1iBRs1pF8J9CPc4R37vjlJUMsDOIYRTv0YJs83cr9E8qQK09NOxk + CdS4zy+1btkwAAAADAIB4kInUtYIFsCSeAAAAAwAAeIG9VXokDCAkPAAAAAMAAHiJtV16JAxgJDwAAAACwAB4mIt14IGMJDwAAAA + KgIB0Miy1dYgU7NaRdlMD+s+HomYswol7sg/lZAGfD5iKIaClD9WuaxS2AAAAAwCAeLiJ1LWCBZAkngAAAAMAAHipvVV6JAwgJDw + AAAADAAB4sbVdeiQMYCQ8AAAAAsAAeMCLdeCBjCQ8AAAAv8qAax4WUjJCWCjD2eNFYBL9VgaG1nDmWYZT44HnoNrRoPGSdWf2Hq+ + kAVp0AETi8bXaTSzsF+JUlDhD3/F6v4RJdJ0nr//gzctK9Kf0HYjnaTZQMpnbQiyS3n232faVg21N8KsG1ZNqscyqiiIt+wc12DD + VvXwz86qXsby+oe0oXOGHLAA6ze7alSy/+qrwwd7NVYDo6yNBZR23bKpM5r65RAYpV46uGId2xk+8hFx+DOKF9516y9z2Sc00Ri2 + 4Iq8OdCVWNvHWKPB635A0pBuAawUdfGgyCjIPQh1+ZsBXLhT41eJcSlVUWuNOqZIyerG7zNStDP2dylZqCW9Ul3XmIVPSEc2f//1 + fOq+gXwDQffb0/SFIXZLPKzHdWYD8RUI1pDR0l5ExhcvZ/zMdpZ4t8Rb3QIVCbN0yjPy1vJ4jjIoHM1NcKMLIDnBVOh+zt3QM3le + O+4ZFjjf8r6ocDE3KoN2gNvpPm5L7CIhQ2ifdpvoTgBmjVh6cTm/Wf491E5/q/5y8gPCFZxtlbEBQZkn2j7DPTns4UYiJ1uD4UzO + KIMq4i+rfpo8hTFNA8HtVN5lJ6Tv0c9CPeh5uAdwwKVrbaPtiazzMXZ8jOAfZ6hngSyJI4nhYaAOHW/sy5dm1McbDzOwuZ1T0nPh + GqyWqtEvQ69uxZZZNkWS55TB8snliUp4sKy1/uLfA8tGJUBuumU7yJbOxPgOpaR06JWocWRaho4OonanCAUCSTLLif+PF4HqU6cr + LSctC7oafi+qctoINabIeRpCshIeTFlTDX7bdZIBUWd74n7FdxIpmLztJCcvZyMLQNpn1zHi6WV4IBFALtCCOBSJ2P/TcS9/Z0C/ + BRexcGZFe88yAbVa+15bZQ44+t92JW+PlM3cDHMdP+cGWxf7QdajToUsmR17uo1Y6BteMHVLrjVz37Mu4OZtaIqp1LwfD/Gl0BAF + 8ph1b9l6rYoUppu2nj1CrHd/+0kJgooDopxHesfYiVHVpsapV0t6aHArjyCdhMCBL/e4CAAAAAwSAeOCJ1LWCBbAkngAAAAMEAHj + RvVV6JAxgJDwAAAADBAB42bVdeiQMYCQ8AAAAAsQAeOiLdeCBhCQ8AAAABUCAdEZS4gUwKWUARtgIeLbo2Dsm4AAAAAKAgHkJJ1g + gWSSeAAAAAsAAePk9V6JAxiQ8AAAAAsAAeQE116JAxiQ8AAAAAoAAeRGt+CBhJDwAAAAKgIB0UCS1eIFMLNaRZJaHzbJq9zDac6C + VeR4Ose49AXhreJ5h3CAkxnAgAAAAAsCAeTCJ1dYIFmSeAAAAAwAAeSG9VXokDCAkPAAAAAMAAHkptV16JAxgJDwAAAACwAB5OIt + 14IGEJDwAAAALAIB0Wiy1dYgU7NX0oxEpyVXkd0PlmoD4UCzwOy1Kn4HM30P7lZDezVWpePWAAAADAIB5WInUtYIFkCSeAAAAAwA + AeUm9VXokDCAkPAAAAAMAAHlRtV16JAxgJDwAAAACwAB5YIt14IGEJDwAAAAMQIB0ZCy1dYgU7NaQVA7FgvGoiPkdZiJRmkpLH4H + CwR/7s/PnTGo0asEbqN5ii4n+OAAAAAMAgHmAidS1ggWwJJ4AAAADAAB5cb1VeiQMICQ8AAAAAwAAeXm1XXokDCAkPAAAAALAAHm + Ii3XggYwkPAAAAAoAgHRuLLV1iBTs1pIU6hYy2wO0STGAgb99r9LabtcybfbnmQ6/OsmvgAAAAwCAeaiJ1LWCBbAkngAAAAMAAHm + ZvVV6JAwgJDwAAAADAAB5obVdeiQMICQ8AAAAAsAAebCLdeCBhCQ8AAAACsCAdHYslXWIFOtmAzX0NRed3pQXw+62wWHkLbUGOmJ + Jygg3cAOzHxoBHGAAAAADAIB5yIlUtYIFsCSeAAAAAsAAecG9deiQMaQ8AAAAAsAAedCLVeCBjCQ8A== + """ +} diff --git a/Tests/AetherEngineTests/DeinterlaceHardwareTests.swift b/Tests/AetherEngineTests/DeinterlaceHardwareTests.swift index 4daa6708c..51c713a56 100644 --- a/Tests/AetherEngineTests/DeinterlaceHardwareTests.swift +++ b/Tests/AetherEngineTests/DeinterlaceHardwareTests.swift @@ -8,11 +8,15 @@ import AetherLibavfilter /// AV_PIX_FMT_VIDEOTOOLBOX frames wrapping CVPixelBuffers, and mode=send_field doubles the /// output cadence on the sink's time_base. /// -/// Both tests self-skip (early return, engine falls back to .software) when the linked build has -/// no yadif_videotoolbox or the machine has no Metal device (CI VMs), the fallback itself is -/// covered by the software tests. +/// The three tests are reported as skipped (not passed) when the linked build has no +/// yadif_videotoolbox or the machine has no Metal device (CI VMs): the engine then falls back to +/// .software, and the fallback itself is covered by the software tests. struct DeinterlaceHardwareTests { + /// Built once: the probe stands up and tears down a whole filter graph. + private static let hardwareAvailable = DeinterlaceFilter.prewarmHardwarePipeline() + private static let needsHardware: Comment = "needs yadif_videotoolbox and a Metal device" + private func makeFrame(pts: Int64) -> UnsafeMutablePointer { let f = av_frame_alloc()! f.pointee.width = 64 @@ -52,7 +56,7 @@ struct DeinterlaceHardwareTests { guard hardware else { var ff: UnsafeMutablePointer? = f av_frame_free(&ff) - return ([], false) // no yadif_videotoolbox / no Metal device: caller skips + return ([], false) } #expect(filter.push(f) >= 0) @@ -72,7 +76,8 @@ struct DeinterlaceHardwareTests { return (seconds, hardware) } - @Test("HW engine emits VideoToolbox frames wrapping CVPixelBuffers") + @Test("HW engine emits VideoToolbox frames wrapping CVPixelBuffers", + .enabled(if: DeinterlaceHardwareTests.hardwareAvailable, DeinterlaceHardwareTests.needsHardware)) func hardwareEmitsVideoToolboxFrames() { var sawVTFrame = false let (seconds, hardware) = run(fieldRate: .field, count: 8) { out in @@ -91,18 +96,18 @@ struct DeinterlaceHardwareTests { Issue.record("AV_PIX_FMT_VIDEOTOOLBOX frame with nil data[3]") } } - guard hardware else { return } // linked build predates yadif_videotoolbox, or no Metal + #expect(hardware) #expect(sawVTFrame, "hw graph engaged but produced no frames") #expect(!seconds.isEmpty) } - @Test("send_field doubles output cadence; send_frame keeps it") + @Test("send_field doubles output cadence; send_frame keeps it", + .enabled(if: DeinterlaceHardwareTests.hardwareAvailable, DeinterlaceHardwareTests.needsHardware)) func fieldRateControlsCadence() { let streamTBSeconds = 1.0 / 30.0 let (fieldSeconds, hwField) = run(fieldRate: .field, count: 8) - guard hwField else { return } // self-skip, see type doc let (frameSeconds, hwFrame) = run(fieldRate: .frame, count: 8) - guard hwFrame else { return } + #expect(hwField && hwFrame) // Field rate must produce ~2x the frames of frame rate (lookahead trims the tails). #expect(fieldSeconds.count > frameSeconds.count + 2, @@ -129,11 +134,10 @@ struct DeinterlaceHardwareTests { } } - @Test("warm-up releases its temporary graph and a fresh hardware graph remains usable") + @Test("warm-up releases its temporary graph and a fresh hardware graph remains usable", + .enabled(if: DeinterlaceHardwareTests.hardwareAvailable, DeinterlaceHardwareTests.needsHardware)) func warmupLeavesFreshGraphUsable() { - guard DeinterlaceFilter.prewarmHardwarePipeline() else { - return - } + #expect(DeinterlaceFilter.prewarmHardwarePipeline()) let (seconds, hardware) = run(fieldRate: .field, count: 8) #expect(hardware) diff --git a/Tests/AetherEngineTests/DemuxedTimestampBoundsTests.swift b/Tests/AetherEngineTests/DemuxedTimestampBoundsTests.swift new file mode 100644 index 000000000..45dfac6ff --- /dev/null +++ b/Tests/AetherEngineTests/DemuxedTimestampBoundsTests.swift @@ -0,0 +1,256 @@ +import CoreMedia +import Foundation +import Testing +import AetherLibavcodec +import AetherLibavformat +import AetherLibavutil +@testable import AetherEngine + +/// A minimal Matroska file with one S_TEXT/UTF8 track, one Cluster per block, built by hand because +/// the pinned FFmpeg carries no Matroska muxer. Every field is written verbatim, which is the point: +/// matroskadec stores a uint64 Cluster Timestamp into an int64 pts unchecked, clamps BlockDuration to +/// INT64_MAX, and derives the stream time base from TimestampScale, so a crafted file reaches any +/// magnitude and a time base whose numerator is above one. +enum HostileTimestampMatroska { + struct Block { + var cluster: UInt64 + var duration: UInt64? + } + + private static func vintSize(_ n: Int) -> [UInt8] { + var bytes: [UInt8] = [0x01] + for shift in stride(from: 48, through: 0, by: -8) { + bytes.append(UInt8((n >> shift) & 0xFF)) + } + return bytes + } + + private static func element(_ id: [UInt8], _ payload: [UInt8]) -> [UInt8] { + id + vintSize(payload.count) + payload + } + + private static func uint(_ value: UInt64) -> [UInt8] { + var bytes: [UInt8] = [] + var v = value + repeat { + bytes.insert(UInt8(v & 0xFF), at: 0) + v >>= 8 + } while v > 0 + return bytes + } + + static func make(timestampScale: UInt64 = 1_000_000, blocks: [Block]) -> Data { + let header = element([0x1A, 0x45, 0xDF, 0xA3], + element([0x42, 0x86], uint(1)) + + element([0x42, 0xF7], uint(1)) + + element([0x42, 0xF2], uint(4)) + + element([0x42, 0xF3], uint(8)) + + element([0x42, 0x82], Array("matroska".utf8)) + + element([0x42, 0x87], uint(4)) + + element([0x42, 0x85], uint(2))) + let info = element([0x15, 0x49, 0xA9, 0x66], + element([0x2A, 0xD7, 0xB1], uint(timestampScale)) + + element([0x4D, 0x80], Array("aether-audit".utf8)) + + element([0x57, 0x41], Array("aether-audit".utf8))) + let track = element([0xD7], uint(1)) + + element([0x73, 0xC5], uint(1)) + + element([0x83], uint(0x11)) + + element([0x9C], uint(0)) + + element([0x86], Array("S_TEXT/UTF8".utf8)) + let tracks = element([0x16, 0x54, 0xAE, 0x6B], element([0xAE], track)) + var clusters: [UInt8] = [] + for (index, block) in blocks.enumerated() { + let payload: [UInt8] = [0x81, 0x00, 0x00, 0x00] + Array("line \(index)".utf8) + var group = element([0xA1], payload) + if let duration = block.duration { group += element([0x9B], uint(duration)) } + clusters += element([0x1F, 0x43, 0xB6, 0x75], + element([0xE7], uint(block.cluster)) + element([0xA0], group)) + } + return Data(header + element([0x18, 0x53, 0x80, 0x67], info + tracks + clusters)) + } + + /// Every packet the engine's demuxer hands out, as (pts, dts, duration), with the stream's time base. + static func readBack(_ data: Data) throws -> (timeBase: AVRational, packets: [(pts: Int64, dts: Int64, duration: Int64)]) { + let demuxer = Demuxer() + defer { demuxer.close() } + try demuxer.open(reader: DataIOReader(data: data), formatHint: "matroska") + let stream = try #require(demuxer.stream(at: 0)) + var packets: [(pts: Int64, dts: Int64, duration: Int64)] = [] + while let packet = try demuxer.readPacket() { + var owned: UnsafeMutablePointer? = packet + defer { trackedPacketFree(&owned) } + packets.append((packet.pointee.pts, packet.pointee.dts, packet.pointee.duration)) + } + return (stream.pointee.time_base, packets) + } +} + +@Suite("Demuxed timestamps are bounded before any consumer does arithmetic on them") +struct DemuxedTimestampBoundsTests { + + private static let unset = Int64.min + + /// matroskadec derives 3/1e9 from TimestampScale 3, so the seconds rule and the tick rule both + /// have something to catch: 2^61 ticks are 6.9e9 s, 2^62 ticks fail on their own. + @Test("out-of-range timestamps leave the demuxer unset, and an INT64_MAX duration leaves it 0") + func outOfRangeTimestampsLeaveTheDemuxerUnset() throws { + let read = try HostileTimestampMatroska.readBack(HostileTimestampMatroska.make( + timestampScale: 3, + blocks: [.init(cluster: 1000, duration: 500), + .init(cluster: 1 << 61, duration: 500), + .init(cluster: 1 << 62, duration: 500), + .init(cluster: 2000, duration: UInt64.max), + .init(cluster: 3000, duration: 1)])) + #expect(read.timeBase.num == 3 && read.timeBase.den == 1_000_000_000) + try #require(read.packets.count == 5) + #expect(read.packets[0] == (1000, 1000, 500)) + #expect(read.packets[1] == (Self.unset, Self.unset, 500)) + #expect(read.packets[2] == (Self.unset, Self.unset, 500)) + #expect(read.packets[3] == (2000, 2000, 0)) + #expect(read.packets[4] == (3000, 3000, 1)) + } + + /// The bound must not cost a real source its clock: epoch-anchored wall-clock timestamps at + /// 1 GHz and 10 MHz are what live fMP4 and some Matroska writers emit. + @Test("epoch-anchored nanosecond and 100 ns timestamps pass untouched") + func epochAnchoredTimestampsSurvive() throws { + let nanos: UInt64 = 1_790_000_000_000_000_000 + let atNanos = try HostileTimestampMatroska.readBack(HostileTimestampMatroska.make( + timestampScale: 1, blocks: [.init(cluster: nanos, duration: 40_000_000)])) + #expect(atNanos.timeBase.num == 1 && atNanos.timeBase.den == 1_000_000_000) + #expect(atNanos.packets.map(\.pts) == [Int64(nanos)]) + #expect(atNanos.packets.map(\.duration) == [40_000_000]) + + let hundreds: UInt64 = 17_900_000_000_000_000 + let atHundreds = try HostileTimestampMatroska.readBack(HostileTimestampMatroska.make( + timestampScale: 100, blocks: [.init(cluster: hundreds, duration: 400_000)])) + #expect(atHundreds.timeBase.num == 1 && atHundreds.timeBase.den == 10_000_000) + #expect(atHundreds.packets.map(\.pts) == [Int64(hundreds)]) + } + + /// Audit FEA-101 / SUB-105: the sidecar decoder runs its own `av_read_frame` loop, a second raw + /// channel next to the Demuxer. + @Test("the sidecar subtitle reader bounds its own read loop") + func sidecarReaderBoundsItsReadLoop() async throws { + let data = HostileTimestampMatroska.make(blocks: [ + .init(cluster: 1000, duration: 500), + .init(cluster: 2000, duration: UInt64.max), + .init(cluster: 5_000_000_000_000, duration: 500), + ]) + let url = FileManager.default.temporaryDirectory + .appendingPathComponent("hostile-timestamps-\(UUID().uuidString).mkv") + defer { try? FileManager.default.removeItem(at: url) } + try data.write(to: url) + let cues = try await SubtitleDecoder.decodeFile(url: url).cues + #expect(cues.count == 3) + for cue in cues { + #expect(cue.startTime.isFinite && cue.endTime.isFinite) + #expect(abs(cue.startTime) < SourceTimestampBounds.maxPlausibleSeconds) + #expect(abs(cue.endTime) < SourceTimestampBounds.maxPlausibleSeconds) + } + #expect(cues.contains { $0.startTime == 1 && $0.endTime == 1.5 }) + } + + /// Audit FEA-101, the software-route vector: the SW host's subtitle tap harvests what the demuxer + /// hands it, and the drain rebuilt the packet with `Int64(durationSeconds * 1000)`. matroskadec + /// clamps an all-ones BlockDuration to INT64_MAX, which at 1 ms is exactly 2^63 after the round + /// trip through seconds. + @Test("a subtitle block whose duration matroskadec clamps to INT64_MAX drains without trapping") + func clampedBlockDurationDrains() throws { + let data = HostileTimestampMatroska.make(blocks: [.init(cluster: 1000, duration: UInt64.max)]) + let demuxer = Demuxer() + defer { demuxer.close() } + try demuxer.open(reader: DataIOReader(data: data), formatHint: "matroska") + let stream = try #require(demuxer.stream(at: 0)) + let decoder = try #require(EmbeddedSubtitleDecoder(stream: stream, sourceVideoWidth: 16, sourceVideoHeight: 16)) + let store = SubtitlePacketStore() + while let packet = try demuxer.readPacket() { + var owned: UnsafeMutablePointer? = packet + defer { trackedPacketFree(&owned) } + store.harvest(streamIndex: 0, packet: packet, timeBase: stream.pointee.time_base, writer: .pump) + } + let entries = store.entries(streamIndex: 0, from: 0, through: 10) + try #require(entries.count == 1) + #expect(entries[0].durationSeconds == 0) + let event = AetherEngine.decodeStoredSubtitlePacket(entries[0], with: decoder) + #expect(event?.cues.first?.startTime == 1) + } + + /// Audit DEC-101: the decoders turn a frame timestamp into a CMTime as `pts * num`. A value the + /// demuxer bound would already have unset still reaches them through the fold and through + /// decoder-computed frame timestamps, so the conversion itself has to be total. + @Test("the software decoder survives a frame timestamp whose product with the numerator overflows") + func softwareDecoderSurvivesNumeratorOverflow() throws { + let data = try #require(Data(base64Encoded: Issue220SoftwareDecoderDrainTests.fixtureBase64, + options: .ignoreUnknownCharacters)) + let demuxer = Demuxer() + try demuxer.open(reader: DataIOReader(data: data), formatHint: "mp4") + defer { demuxer.close() } + let videoIndex = demuxer.videoStreamIndex + let stream = try #require(demuxer.stream(at: videoIndex)) + stream.pointee.time_base = AVRational(num: 3, den: 1_000_000_000) + let frames = FrameTimes() + let decoder = SoftwareVideoDecoder() + try decoder.open(stream: stream) { _, pts, _ in frames.append(pts) } + defer { decoder.close() } + + let base = (Int64(1) << 62) - 1_000_000 + var index: Int64 = 0 + while let packet = try? demuxer.readPacket() { + if packet.pointee.stream_index == videoIndex { + packet.pointee.pts = base + index * 1000 + packet.pointee.dts = packet.pointee.pts + index += 1 + decoder.decode(packet: packet) + } + var owned: UnsafeMutablePointer? = packet + trackedPacketFree(&owned) + } + let delivered = frames.values + #expect(!delivered.isEmpty) + #expect(delivered.allSatisfy { !$0.isValid }) + } + + @Test("the audio decoder survives a frame timestamp whose product with the numerator overflows") + func audioDecoderSurvivesNumeratorOverflow() throws { + let demuxer = Demuxer() + defer { demuxer.close() } + try demuxer.open(reader: DataIOReader(data: ProbeTestFixtures.eac3()), formatHint: "mp4") + let audioIndex = demuxer.audioStreamIndex + let stream = try #require(demuxer.stream(at: audioIndex)) + stream.pointee.time_base = AVRational(num: 3, den: 1_000_000_000) + let decoder = AudioDecoder() + try decoder.open(stream: stream) + defer { decoder.close() } + + let base = (Int64(1) << 62) - 1_000_000 + var index: Int64 = 0 + var buffers = 0 + while let packet = try demuxer.readPacket() { + var owned: UnsafeMutablePointer? = packet + defer { trackedPacketFree(&owned) } + guard packet.pointee.stream_index == audioIndex else { continue } + packet.pointee.pts = base + index * 1000 + packet.pointee.dts = packet.pointee.pts + index += 1 + buffers += decoder.decode(packet: packet).count + } + #expect(index > 0) + #expect(buffers > 0) + } + + private final class FrameTimes: @unchecked Sendable { + private let lock = NSLock() + private var storage: [CMTime] = [] + func append(_ time: CMTime) { + lock.lock() + storage.append(time) + lock.unlock() + } + var values: [CMTime] { + lock.lock() + defer { lock.unlock() } + return storage + } + } +} diff --git a/Tests/AetherEngineTests/DemuxerMarkClosedReachTests.swift b/Tests/AetherEngineTests/DemuxerMarkClosedReachTests.swift new file mode 100644 index 000000000..7c1b52a64 --- /dev/null +++ b/Tests/AetherEngineTests/DemuxerMarkClosedReachTests.swift @@ -0,0 +1,115 @@ +import Testing +import Foundation +import AetherLibavcodec +@testable import AetherEngine + +/// Audit DMX-102 / DMX-109 / DMX-112: `Demuxer.markClosed()` is the one lock-free abort a teardown +/// has, and it only reached the stages that already had a provider to mark. The provider was +/// published after `provider.open()` returned, so a close during the connect (up to 15 s on a live +/// source) found nothing; a local-path source had no provider at all, so the interrupt callback +/// installed on it never saw the close; and a pump parked for an origin slot ignored it. +/// +/// The bounds below are the assertions: every abort is measured against a wait the defect would +/// have sat out in full (10 s of withheld first byte, 10 s of slot wait). +@Suite("markClosed reaches every blocking stage of a demuxer", .serialized) +struct DemuxerMarkClosedReachTests { + + /// Cross-thread carrier for the result of an open running on a thread of its own. + private final class OpenOutcome: @unchecked Sendable { + private let lock = NSLock() + private var _finished = false + private var _threw = false + var finished: Bool { lock.withLock { _finished } } + var threw: Bool { lock.withLock { _threw } } + func finish(threw: Bool) { lock.withLock { _finished = true; _threw = threw } } + } + + private func openOnAThread(_ demuxer: Demuxer, url: URL, isLive: Bool) -> OpenOutcome { + let outcome = OpenOutcome() + Thread.detachNewThread { + do { + try demuxer.open(url: url, isLive: isLive) + outcome.finish(threw: false) + } catch { + outcome.finish(threw: true) + } + } + return outcome + } + + @Test("a close that lands while the source is connecting aborts the open", + .timeLimit(.minutes(2)), arguments: [true, false]) + func closeDuringConnectAbortsTheOpen(isLive: Bool) async throws { + let origin = ThrottledOriginServer( + totalSize: 64 << 20, patternedBody: true, firstByteDelayUs: { _ in 10_000_000 }) + let server = try #require(origin) + defer { server.stop() } + let url = URL(string: "http://127.0.0.1:\(server.port)/movie.ts")! + + let demuxer = Demuxer() + let outcome = openOnAThread(demuxer, url: url, isLive: isLive) + try await waitFor { server.requestedRanges.count >= 1 } + demuxer.markClosed() + + let returned = try await waitFor(upTo: .seconds(4)) { outcome.finished } + #expect(returned, "the open sat out the origin's withheld first byte after markClosed()") + #expect(outcome.threw, "an aborted open must throw, not hand back a demuxer") + demuxer.close() + } + + @Test("a pump parked for an origin slot leaves when the demuxer is closed", + .timeLimit(.minutes(2))) + func closeReachesAParkedPump() async throws { + let server = try #require(ThrottledOriginServer(totalSize: 64 << 20, patternedBody: true)) + defer { server.stop() } + let url = URL(string: "http://127.0.0.1:\(server.port)/movie.ts")! + OriginRequestBudget.shared.setHostLimit(1, for: url) + let holder = OriginRequestBudget.shared.acquire(for: url, label: "holder", timeout: 1) + defer { OriginRequestBudget.shared.release(holder) } + + let demuxer = Demuxer() + let outcome = openOnAThread(demuxer, url: url, isLive: false) + try await waitFor { (OriginRequestBudget.shared.snapshot(for: url)?.waiting ?? 0) >= 1 } + demuxer.markClosed() + + let returned = try await waitFor(upTo: .seconds(4)) { outcome.finished } + #expect(returned, "the open stayed parked in the origin's slot queue after markClosed()") + #expect(outcome.threw) + try await waitFor { (OriginRequestBudget.shared.snapshot(for: url)?.waiting ?? 0) == 0 } + #expect(OriginRequestBudget.shared.snapshot(for: url)?.inflight == 1, + "the closed pump took a slot that belongs to the next session") + demuxer.close() + } + + @Test("a close on a local-path source stops the reads libavformat makes itself") + func closeReachesALocalSource() throws { + let tiny = TinyTransportStreamFixture.data + var repeated = Data() + for _ in 0..<400 { repeated.append(tiny) } + let file = FileManager.default.temporaryDirectory + .appendingPathComponent("markclosed-\(UUID().uuidString).ts") + try repeated.write(to: file) + defer { try? FileManager.default.removeItem(at: file) } + + let demuxer = Demuxer() + defer { demuxer.close() } + try demuxer.open(url: file) + demuxer.markClosed() + + // What libavformat already buffered during the probe is still delivered; the next read that + // needs the file has to fail instead of finding it. + var delivered = 0 + var failure: Error? + while failure == nil, delivered < 100_000 { + do { + guard let read = try demuxer.readPacket() else { break } + var packet: UnsafeMutablePointer? = read + trackedPacketFree(&packet) + delivered += 1 + } catch { + failure = error + } + } + #expect(failure != nil, "a closed local source read all \(delivered) packets to its end") + } +} diff --git a/Tests/AetherEngineTests/DemuxerStreamLifetimeTests.swift b/Tests/AetherEngineTests/DemuxerStreamLifetimeTests.swift new file mode 100644 index 000000000..12659d5ae --- /dev/null +++ b/Tests/AetherEngineTests/DemuxerStreamLifetimeTests.swift @@ -0,0 +1,142 @@ +import Testing +import Foundation +import AetherLibavcodec +@testable import AetherEngine + +/// Audit DMX-108: the NAT-7 fix made `stream(at:)` safe against MPEG-TS reallocating `streams` +/// inside `av_read_frame`, and left two holes. The sibling track accessors still walked the live +/// array without the lock, and `close()` freed the streams before it emptied the table, so a caller +/// on another thread could hold a raw `AVStream*` across a live reopen's `close()`. +@Suite("Demuxer stream lifetime and track accessors", .serialized) +struct DemuxerStreamLifetimeTests { + + private final class Flag: @unchecked Sendable { + private let lock = NSLock() + private var _value = false + var value: Bool { lock.withLock { _value } } + func set() { lock.withLock { _value = true } } + } + + private final class Seen: @unchecked Sendable { + private let lock = NSLock() + private var _index: Int32? + var index: Int32? { lock.withLock { _index } } + func set(_ index: Int32) { lock.withLock { _index = index } } + } + + private final class Tracks: @unchecked Sendable { + private let lock = NSLock() + private var _video: Int32? + private var _subtitles: [Int32]? + private var _indices: Set? + var video: Int32? { lock.withLock { _video } } + var subtitles: [Int32]? { lock.withLock { _subtitles } } + var indices: Set? { lock.withLock { _indices } } + func set(video: Int32, subtitles: [Int32], indices: Set) { + lock.withLock { _video = video; _subtitles = subtitles; _indices = indices } + } + } + + @Test("close waits for a caller inside withStream and hands out no stream once it has started", + .timeLimit(.minutes(1))) + func closeWaitsForStreamUsers() async throws { + let url = try MultiSubtitleContainerFixture.write() + defer { try? FileManager.default.removeItem(at: url) } + let demuxer = Demuxer() + try demuxer.open(url: url) + + let entered = Flag() + let closed = Flag() + let seenInsideTheBody = Seen() + let release = DispatchSemaphore(value: 0) + // A wait the time limit cancels skips the signal below; this one lets the caller and the + // close behind it finish anyway. An extra signal is harmless: nothing waits twice. + defer { release.signal() } + Thread.detachNewThread { + _ = demuxer.withStream(at: MultiSubtitleContainerFixture.videoStreamIndex) { stream in + entered.set() + release.wait() + seenInsideTheBody.set(stream.pointee.index) + } + } + try await waitFor { entered.value } + + Thread.detachNewThread { + demuxer.close() + closed.set() + } + // The table is emptied before the streams are freed: nothing new gets a stream from here. + try await waitFor { demuxer.stream(at: MultiSubtitleContainerFixture.videoStreamIndex) == nil } + #expect(demuxer.withStream(at: MultiSubtitleContainerFixture.videoStreamIndex) { _ in true } == nil, + "a stream was handed out after close() had started") + + let closedUnderTheCaller = try await waitFor(upTo: .milliseconds(300)) { closed.value } + #expect(!closedUnderTheCaller, "close() freed the streams while a caller was still inside withStream") + + release.signal() + try await waitFor { closed.value } + #expect(seenInsideTheBody.index == MultiSubtitleContainerFixture.videoStreamIndex) + } + + @Test("withStream hands out nothing for an index the demuxer does not have") + func withStreamRejectsUnknownIndices() throws { + let url = try MultiSubtitleContainerFixture.write() + defer { try? FileManager.default.removeItem(at: url) } + let demuxer = Demuxer() + defer { demuxer.close() } + try demuxer.open(url: url) + + #expect(demuxer.withStream(at: -1) { _ in true } == nil) + #expect(demuxer.withStream(at: 99) { _ in true } == nil) + #expect(demuxer.withStream(at: MultiSubtitleContainerFixture.spanishStreamIndex) { + $0.pointee.codecpar.pointee.codec_type == AVMEDIA_TYPE_SUBTITLE + } == true) + } + + @Test("the track accessors answer from their snapshot while a read holds the demuxer", + .timeLimit(.minutes(1))) + func accessorsDoNotWaitOutARead() async throws { + let url = try MultiSubtitleContainerFixture.write() + defer { try? FileManager.default.removeItem(at: url) } + let demuxer = Demuxer() + defer { demuxer.close() } + try demuxer.open(url: url) + + // `isCurrent` runs inside `readPacket` with the access lock held, which is the state a read + // parked on a slow origin leaves the lock in. + let holding = Flag() + let readReleased = DispatchSemaphore(value: 0) + // Runs before the close: a wait the time limit cancels must not leave the read holding the + // access lock that close() waits on. + defer { readReleased.signal() } + let readFinished = Flag() + Thread.detachNewThread { + _ = try? demuxer.readPacket(isCurrent: { + holding.set() + readReleased.wait() + return true + }) + readFinished.set() + } + try await waitFor { holding.value } + + let answered = Tracks() + let done = Flag() + Thread.detachNewThread { + answered.set(video: demuxer.videoStreamIndex, + subtitles: demuxer.subtitleTrackInfos().map { Int32($0.id) }, + indices: demuxer.subtitleStreamIndices()) + done.set() + } + let returned = try await waitFor(upTo: .seconds(5)) { done.value } + readReleased.signal() + try await waitFor { readFinished.value } + + #expect(returned, "a track accessor waited out a read that holds the access lock") + #expect(answered.video == MultiSubtitleContainerFixture.videoStreamIndex) + #expect(answered.subtitles == [MultiSubtitleContainerFixture.englishStreamIndex, + MultiSubtitleContainerFixture.spanishStreamIndex]) + #expect(answered.indices == [MultiSubtitleContainerFixture.englishStreamIndex, + MultiSubtitleContainerFixture.spanishStreamIndex]) + } +} diff --git a/Tests/AetherEngineTests/DemuxerStreamLookupTests.swift b/Tests/AetherEngineTests/DemuxerStreamLookupTests.swift new file mode 100644 index 000000000..b949949d0 --- /dev/null +++ b/Tests/AetherEngineTests/DemuxerStreamLookupTests.swift @@ -0,0 +1,108 @@ +import Testing +import Foundation +import AetherLibavcodec +@testable import AetherEngine + +/// `Demuxer.stream(at:)` is called from the main actor while the demux thread sits inside +/// `av_read_frame`, which on MPEG-TS can reallocate the `streams` array (audit NAT-7). It answers +/// from a table copied under the demuxer's lock, so it neither indexes the live array nor waits out a +/// read that holds that lock for a whole network stall. +@Suite("Demuxer stream lookup") +struct DemuxerStreamLookupTests { + + /// A WAV source whose reads can be made to block, the way a network read parks. + private final class GatedReader: IOReader, @unchecked Sendable { + private let inner: DataIOReader + private let lock = NSLock() + private var armed = false + private var _blocked = false + private let gate = DispatchSemaphore(value: 0) + + init(data: Data) { inner = DataIOReader(data: data) } + + var isBlocked: Bool { lock.lock(); defer { lock.unlock() }; return _blocked } + + func arm() { lock.lock(); armed = true; lock.unlock() } + + func release() { + lock.lock() + armed = false + lock.unlock() + gate.signal() + } + + func read(_ buffer: UnsafeMutablePointer?, size: Int32) -> Int32 { + lock.lock() + let wait = armed + if wait { _blocked = true } + lock.unlock() + if wait { gate.wait() } + return inner.read(buffer, size: size) + } + + func seek(offset: Int64, whence: Int32) -> Int64 { inner.seek(offset: offset, whence: whence) } + func close() { inner.close() } + func cancel() {} + func makeIndependentReader() -> IOReader? { nil } + var discImageProbeEnabled: Bool { false } + } + + private func makeWAV(seconds: Double) -> Data { + let sampleRate = 48_000, channels = 2 + let pcm = Data(count: Int(Double(sampleRate) * seconds) * channels * 2) + var d = Data() + func str(_ s: String) { d.append(s.data(using: .ascii)!) } + func u32(_ v: UInt32) { withUnsafeBytes(of: v.littleEndian) { d.append(contentsOf: $0) } } + func u16(_ v: UInt16) { withUnsafeBytes(of: v.littleEndian) { d.append(contentsOf: $0) } } + str("RIFF"); u32(UInt32(36 + pcm.count)); str("WAVE") + str("fmt "); u32(16); u16(1); u16(UInt16(channels)); u32(UInt32(sampleRate)) + u32(UInt32(sampleRate * channels * 2)); u16(UInt16(channels * 2)); u16(16) + str("data"); u32(UInt32(pcm.count)); d.append(pcm) + return d + } + + @Test("every opened stream is found, an index past them is not, and nothing is after close") + func lookupFollowsTheOpenStreams() throws { + let demuxer = Demuxer() + try demuxer.open(reader: DataIOReader(data: makeWAV(seconds: 1))) + let audio = demuxer.audioStreamIndex + try #require(audio >= 0) + #expect(demuxer.stream(at: audio)?.pointee.index == audio) + #expect(demuxer.stream(at: audio + 1) == nil) + #expect(demuxer.stream(at: -1) == nil) + demuxer.close() + #expect(demuxer.stream(at: audio) == nil) + } + + @Test("a lookup does not wait for a read that holds the demuxer", .timeLimit(.minutes(1))) + func lookupDoesNotWaitOutARead() async throws { + let reader = GatedReader(data: makeWAV(seconds: 20)) + let demuxer = Demuxer() + try demuxer.open(reader: reader) + let audio = demuxer.audioStreamIndex + try #require(audio >= 0) + reader.arm() + + let done = DispatchSemaphore(value: 0) + Thread.detachNewThread { + while let packet = try? demuxer.readPacket() { + var p: UnsafeMutablePointer? = packet + trackedPacketFree(&p) + } + done.signal() + } + try await waitFor { reader.isBlocked } + + #expect(demuxer.stream(at: audio)?.pointee.index == audio) + + demuxer.markClosed() + reader.release() + await withCheckedContinuation { (continuation: CheckedContinuation) in + Thread.detachNewThread { + done.wait() + continuation.resume() + } + } + demuxer.close() + } +} diff --git a/Tests/AetherEngineTests/DetourBlockCacheTests.swift b/Tests/AetherEngineTests/DetourBlockCacheTests.swift index 9985e6287..b76f84db3 100644 --- a/Tests/AetherEngineTests/DetourBlockCacheTests.swift +++ b/Tests/AetherEngineTests/DetourBlockCacheTests.swift @@ -116,6 +116,60 @@ struct DetourBlockCacheTests { #expect(Array(dst[0..<4]) == [100, 101, 102, 103]) // newest value served } + /// A clock a test moves by hand. + private final class ManualClock: @unchecked Sendable { + private let lock = NSLock() + private var seconds: TimeInterval = 1000 + var now: TimeInterval { lock.lock(); defer { lock.unlock() }; return seconds } + func advance(_ by: TimeInterval) { lock.lock(); seconds += by; lock.unlock() } + } + + // audit PERF-108: only close() emptied the cache, so the blocks of a backward scrub stayed + // resident (up to 32 MB per reader) for the rest of the session. + @Test("a block idle past the threshold is evicted and a recently hit one survives") + func idleEviction() { + let clock = ManualClock() + let cache = DetourBlockCache(blockSize: 16, maxBlocks: 8, now: { clock.now }) + cache.insert(0, patternBlock(base: 0)) + cache.insert(1, patternBlock(base: 16)) + cache.insert(2, patternBlock(base: 32)) + + clock.advance(20) + _ = cache.block(1) + var dst = [UInt8](repeating: 0, count: 16) + #expect(dst.withUnsafeMutableBufferPointer { cache.serveCached(into: $0.baseAddress!, maxLen: 4, at: 32) } == 4) + clock.advance(20) + + // Block 0 has been idle 40 s; blocks 1 and 2 were hit 20 s ago. + #expect(cache.evictIdle(olderThan: 30) == 1) + #expect(cache.residentCount == 2) + #expect(cache.block(0) == nil) + #expect(cache.block(1) != nil) + #expect(cache.block(2) != nil) + + clock.advance(31) + #expect(cache.evictIdle(olderThan: 30) == 2) + #expect(cache.residentCount == 0) + #expect(cache.evictIdle(olderThan: 30) == 0) + } + + @Test("a block that was evicted for age leaves the LRU order, so a later insert does not trip on it") + func evictedBlockLeavesTheOrder() { + let clock = ManualClock() + let cache = DetourBlockCache(blockSize: 16, maxBlocks: 2, now: { clock.now }) + cache.insert(0, patternBlock(base: 0)) + clock.advance(60) + cache.insert(1, patternBlock(base: 16)) + #expect(cache.evictIdle(olderThan: 30) == 1) + + cache.insert(2, patternBlock(base: 32)) + cache.insert(3, patternBlock(base: 48)) + #expect(cache.residentCount == 2) + #expect(cache.block(1) == nil, "the oldest surviving block is the one the size cap evicts") + #expect(cache.block(2) != nil) + #expect(cache.block(3) != nil) + } + @Test("clear empties the cache") func clearEmpties() { let cache = DetourBlockCache(blockSize: 16, maxBlocks: 3) diff --git a/Tests/AetherEngineTests/DetourBlockIdleReleaseTests.swift b/Tests/AetherEngineTests/DetourBlockIdleReleaseTests.swift new file mode 100644 index 000000000..283abed55 --- /dev/null +++ b/Tests/AetherEngineTests/DetourBlockIdleReleaseTests.swift @@ -0,0 +1,46 @@ +import Testing +import Foundation +@testable import AetherEngine + +/// Audit PERF-108: the detour cache was only emptied at close, so the blocks a backward scrub filled +/// (up to 32 MB per reader) stayed resident for the rest of the session. The read path now sweeps +/// blocks nothing has read for a while. +@Suite("Detour block idle release") +struct DetourBlockIdleReleaseTests { + + @Test("a detour block nothing reads any more is released by the read path", .timeLimit(.minutes(2))) + func idleBlockIsReleased() async throws { + let maybe = ThrottledOriginServer(totalSize: 64 * 1024 * 1024, throttleUs: 1000) + let server = try #require(maybe) + defer { server.stop() } + let url = URL(string: "http://127.0.0.1:\(server.port)/movie.bin")! + let reader = AVIOReader(url: url, detourIdleSeconds: 0.2) + defer { reader.markClosed(); reader.close() } + try reader.open() + + let chunk = 256 * 1024 + let buf = UnsafeMutablePointer.allocate(capacity: chunk) + defer { buf.deallocate() } + var read = 0 + while read < 13 * 1024 * 1024 { + let n = Int(reader.read(into: buf, size: Int32(chunk))) + #expect(n > 0, "forward read failed at \(read)") + if n <= 0 { return } + read += n + } + let frontier = Int64(read) + + // Backward past the retained head and the window: this fills the 4 to 8 MB block. + #expect(reader.seek(offset: 5 * 1024 * 1024, whence: Int32(SEEK_SET)) == 5 * 1024 * 1024) + #expect(reader.read(into: buf, size: Int32(chunk)) > 0) + #expect(reader.detourResidentBlocksForTesting == 1, "no detour block was filled, so this measured nothing") + + // Back to the window. Nothing reads the block from here on, and each read is a chance for + // the sweep, which runs at most once a second. + #expect(reader.seek(offset: frontier, whence: Int32(SEEK_SET)) == frontier) + try await waitFor { + _ = reader.read(into: buf, size: Int32(chunk)) + return reader.detourResidentBlocksForTesting == 0 + } + } +} diff --git a/Tests/AetherEngineTests/DetourOriginTicketTests.swift b/Tests/AetherEngineTests/DetourOriginTicketTests.swift new file mode 100644 index 000000000..e219b99c2 --- /dev/null +++ b/Tests/AetherEngineTests/DetourOriginTicketTests.swift @@ -0,0 +1,62 @@ +import Testing +import Foundation +@testable import AetherEngine + +/// Audit DMX-105: a detour block took an origin ticket in `detourFetchBlock` and a second one inside +/// `syncRequest`. On an origin capped at two requests, with the pump holding one, the outer ticket +/// took the last slot and the inner acquire waited its whole four second budget for it: +4 s per +/// block, and a third request on the books against a limit of two. A reader that already holds a +/// ticket must never block on another, so the detour holds one. +@Suite("Detour origin ticket", .serialized) +struct DetourOriginTicketTests { + + @Test("a detour block behind a pump holding one of two slots takes one slot and no wait", + .timeLimit(.minutes(2))) + func detourTakesOneTicket() async throws { + let total: Int64 = 64 * 1024 * 1024 + // About 6 MB/s: the pump still has seconds of its 32 MB range to go when the detour is asked for. + let maybe = ThrottledOriginServer(totalSize: total, throttleUs: 40_000) + let server = try #require(maybe) + defer { server.stop() } + let url = URL(string: "http://127.0.0.1:\(server.port)/movie.bin")! + OriginRequestBudget.shared.setHostLimit(2, for: url) + defer { OriginRequestBudget.shared.setHostLimit(nil, for: url) } + + let reader = AVIOReader(url: url) + defer { reader.markClosed(); reader.close() } + try reader.open() + + let chunk = 256 * 1024 + let buf = UnsafeMutablePointer.allocate(capacity: chunk) + defer { buf.deallocate() } + var read = 0 + while read < 13 * 1024 * 1024 { + let n = Int(reader.read(into: buf, size: Int32(chunk))) + #expect(n > 0, "forward read failed at \(read)") + if n <= 0 { return } + read += n + } + #expect(reader.hasLiveConnectionForTesting, "the pump was not on the link, so the detour had no rival") + + // Backward past the retained head and the window (which trimmed to about 8 MB): the read + // that opens the detour block holding 4 to 8 MB. + let target = Int64(5 * 1024 * 1024) + #expect(reader.seek(offset: target, whence: Int32(SEEK_SET)) == target) + let served = reader.read(into: buf, size: Int32(chunk)) + + #expect(served > 0) + // The witness is the pump still on the link, not a clock: with the old double acquire the + // second ticket waited out its 4 s budget, by which time the pump's range had ended (checked + // by reverting 0fd16c57). A wall-clock bound on this read measured 2.0 to 2.3 s on a loaded + // CI runner for a correct detour. + #expect(reader.hasLiveConnectionForTesting, + "the detour only got its slot once the pump's range had finished") + let books = try #require(OriginRequestBudget.shared.snapshot(for: url)) + #expect(books.limit == 2) + #expect(books.peakInflight == 2, "peak \(books.peakInflight) on the books against a limit of 2") + // The detour block is the 4 MB aligned block holding the target, and nothing else asks for + // exactly that range: the pump's refills are 32 MB and the tail prefetch is a suffix. + #expect(server.requestedRanges.contains { $0.start == 4 * 1024 * 1024 && $0.end == 8 * 1024 * 1024 - 1 }, + "no detour block went out, so this measured nothing: \(server.requestedRanges)") + } +} diff --git a/Tests/AetherEngineTests/DiscReaderRangeAnswerTests.swift b/Tests/AetherEngineTests/DiscReaderRangeAnswerTests.swift new file mode 100644 index 000000000..877d2e800 --- /dev/null +++ b/Tests/AetherEngineTests/DiscReaderRangeAnswerTests.swift @@ -0,0 +1,111 @@ +import Testing +import Foundation +@testable import AetherEngine + +/// Audit NET-102: the remote disc reader checked a range answer only after URLSession had buffered +/// the whole body, so an origin that ignored Range (or answered `bytes N-EOF`) put a 40 GB image +/// into memory before the check could refuse it. The check now runs at the response head, and what +/// is kept is capped at the range that was asked for. +@Suite("Disc reader range answers") +struct DiscReaderRangeAnswerTests { + + private let total = Int64(512 * 1024 * 1024) + + private final class FirstClaim: @unchecked Sendable { + private let lock = NSLock() + private var claimed = false + func take() -> Bool { + lock.lock() + defer { lock.unlock() } + if claimed { return false } + claimed = true + return true + } + } + + private func url(_ server: ThrottledOriginServer, _ name: String = "movie.iso") -> URL { + URL(string: "http://127.0.0.1:\(server.port)/\(name)")! + } + + private func read(_ reader: HTTPDiscIOReader, at offset: Int64, count: Int) -> [UInt8]? { + guard reader.seek(offset: offset, whence: SEEK_SET) == offset else { return nil } + var out = [UInt8](repeating: 0, count: count) + let n = out.withUnsafeMutableBufferPointer { reader.read($0.baseAddress, size: Int32(count)) } + return n == Int32(count) ? out : nil + } + + @Test("an origin that answers the range probe with a 200 is hung up on at the head", + .timeLimit(.minutes(1))) + func plain200IsHungUpOn() throws { + let maybe = ThrottledOriginServer(totalSize: total, respond: { _, _, _ in .serve200 }) + let server = try #require(maybe) + defer { server.stop() } + + #expect(HTTPDiscIOReader(url: url(server)) == nil) + #expect(server.bytesWritten < 16 * 1024 * 1024, + "the origin wrote \(server.bytesWritten / 1024 / 1024) MB of a \(total / 1024 / 1024) MB body") + } + + @Test("open names the origin that cannot address bytes", .timeLimit(.minutes(1))) + func openThrowsForAnOriginWithoutRanges() throws { + let maybe = ThrottledOriginServer(totalSize: total, respond: { _, _, _ in .serve200 }) + let server = try #require(maybe) + defer { server.stop() } + + #expect(throws: AVIOReaderError.originIgnoresRange) { + try HTTPDiscIOReader.open(url: url(server)) + } + } + + @Test("a remote disc image on such an origin fails the demuxer open with that cause", + .timeLimit(.minutes(1))) + func demuxerOpenFailsTyped() throws { + let maybe = ThrottledOriginServer(totalSize: total, respond: { _, _, _ in .serve200 }) + let server = try #require(maybe) + defer { server.stop() } + + let demuxer = Demuxer() + defer { demuxer.close() } + #expect(throws: AVIOReaderError.originIgnoresRange) { + try demuxer.open(url: url(server)) + } + #expect(server.bytesWritten < 16 * 1024 * 1024) + } + + @Test("a 206 that runs past the range is cut where the range ends", .timeLimit(.minutes(1))) + func overWide206IsCutAtTheRange() throws { + let maybe = ThrottledOriginServer(totalSize: total, ignoreRangeEnd: true) + let server = try #require(maybe) + defer { server.stop() } + + let reader = try #require(HTTPDiscIOReader(url: url(server), baseChunkSize: 64 * 1024, + maxChunkSize: 64 * 1024)) + defer { reader.close() } + #expect(read(reader, at: 100_000, count: 4096) == [UInt8](repeating: 0x55, count: 4096)) + #expect(read(reader, at: 9_000_000, count: 4096) == [UInt8](repeating: 0x55, count: 4096)) + #expect(server.bytesWritten < 32 * 1024 * 1024, + "the origin wrote \(server.bytesWritten / 1024 / 1024) MB for two 64 KB ranges") + } + + @Test("a 206 that starts elsewhere is hung up on at the head and the range asked again", + .timeLimit(.minutes(1))) + func misplaced206IsHungUpOn() throws { + let target: Int64 = 64 * 1024 * 1024 + let once = FirstClaim() + let respond: @Sendable (Int, Int64, String) -> ThrottledOriginServer.Directive = { _, offset, _ in + offset == target && once.take() ? .serve206From(start: target - 4096) : .serve206 + } + let maybe = ThrottledOriginServer(totalSize: total, ignoreRangeEnd: true, patternedBody: true, + respond: respond) + let server = try #require(maybe) + defer { server.stop() } + + let reader = try #require(HTTPDiscIOReader(url: url(server), baseChunkSize: 64 * 1024, + maxChunkSize: 64 * 1024)) + defer { reader.close() } + let expected = (0..<4096).map { ThrottledOriginServer.patternByte(at: target + Int64($0)) } + #expect(read(reader, at: target, count: 4096) == expected) + #expect(server.bytesWritten < 32 * 1024 * 1024, + "the origin wrote \(server.bytesWritten / 1024 / 1024) MB, so the misplaced body was taken") + } +} diff --git a/Tests/AetherEngineTests/DiscRecognitionBoundsTests.swift b/Tests/AetherEngineTests/DiscRecognitionBoundsTests.swift new file mode 100644 index 000000000..f488ee3b5 --- /dev/null +++ b/Tests/AetherEngineTests/DiscRecognitionBoundsTests.swift @@ -0,0 +1,225 @@ +import Foundation +import Testing +@testable import AetherEngine + +/// Audit NET-103: disc recognition trusted the image's own counts. A crafted Blu-ray made the +/// title-assembly loop quadratic (1000 PlayItems x 2000 extents took 0.6 s, 4000 took 9.2 s), kept every +/// parsed `.mpls` alive (a UDF directory under its 8 MB cap names about 160k files that can all point at +/// one ICB, so memory passed 2 GB after about 1000 entries), and grouped a DVD by any VTS number. +struct DiscRecognitionBoundsTests { + + private typealias Extents = [(offset: Int64, length: Int64)] + + private func be16(_ v: Int) -> [UInt8] { [UInt8((v >> 8) & 0xff), UInt8(v & 0xff)] } + private func be32(_ v: Int) -> [UInt8] { + [UInt8((v >> 24) & 0xff), UInt8((v >> 16) & 0xff), UInt8((v >> 8) & 0xff), UInt8(v & 0xff)] + } + + private func mpls(playItems declared: Int, present: Int? = nil) -> [UInt8] { + func item(_ index: Int) -> [UInt8] { + var body: [UInt8] = [] + body += Array(String(format: "%05d", index % 100_000).utf8) + body += Array("M2TS".utf8) + body += be16(0) + body.append(0) + body += be32(0) + body += be32(45_000) + body += [UInt8](repeating: 0, count: 8) + return be16(body.count) + body + } + var playlist: [UInt8] = [] + playlist += be32(0) + playlist += be16(0) + playlist += be16(declared) + playlist += be16(0) + for index in 0..<(present ?? declared) { playlist += item(index) } + var out: [UInt8] = [] + out += Array("MPLS".utf8) + out += Array("0200".utf8) + out += be32(40) + out += be32(0) + out += [UInt8](repeating: 0, count: 40 - out.count) + out += playlist + return out + } + + private func entries(_ count: Int) -> [UDFEntry] { + (0.. [MPLSPlaylist] { + DiscReader.scanPlaylists(list, + extents: { _ in self.extentCalls += 1; return [(offset: 0, length: self.declaredLength)] }, + read: { _ in self.readCalls += 1; return self.bytes }) + } + } + + // MARK: - MPLSParser + + @Test("a playlist may hold at most 999 PlayItems, the BD-ROM limit") + func playItemCountIsBounded() { + #expect(MPLSParser.parse(mpls(playItems: 999))?.clipIDs.count == 999) + #expect(MPLSParser.parse(mpls(playItems: 1000)) == nil) + #expect(MPLSParser.parse(mpls(playItems: 65_535, present: 10)) == nil) + } + + // MARK: - PLAYLIST scan + + @Test("a PLAYLIST directory of 160k aliases is read at most 4000 times") + func playlistEntriesAreCapped() { + let source = PlaylistSource(bytes: mpls(playItems: 1)) + let parsed = source.scan(entries(160_000)) + #expect(parsed.count == DiscReader.maxPlaylistFiles) + #expect(source.extentCalls == DiscReader.maxPlaylistFiles) + #expect(source.readCalls == DiscReader.maxPlaylistFiles) + } + + @Test("entries that are not .mpls files cost nothing against the cap") + func otherEntriesAreNotCounted() { + let source = PlaylistSource(bytes: mpls(playItems: 1)) + var list = (0..<10_000).map { UDFEntry(name: "\($0).clpi", isDir: false, icbBlock: 7, icbPartRef: 1) } + list += entries(3) + #expect(source.scan(list).count == 3) + #expect(source.extentCalls == 3) + } + + @Test("playlist bytes are read up to a total budget") + func playlistBytesAreBudgeted() { + let oneMiB = 1 << 20 + var padded = mpls(playItems: 1) + padded += [UInt8](repeating: 0, count: oneMiB - padded.count) + let source = PlaylistSource(bytes: padded) + let parsed = source.scan(entries(200)) + let expected = Int(DiscReader.maxPlaylistBytes) / oneMiB + #expect(parsed.count == expected) + #expect(source.readCalls == expected) + } + + @Test("an oversized playlist extent is skipped without being read or charged") + func oversizedPlaylistIsNotRead() { + var reads = 0 + let huge: Extents = [(offset: 0, length: 64 * 1024 * 1024)] + let parsed = DiscReader.scanPlaylists(entries(10), extents: { _ in huge }, read: { _ in reads += 1; return [] }) + #expect(parsed.isEmpty) + #expect(reads == 0) + } + + @Test("parsed PlayItems are kept up to a total budget") + func playItemsAreBudgeted() { + let source = PlaylistSource(bytes: mpls(playItems: 999)) + let parsed = source.scan(entries(600)) + #expect(parsed.count == DiscReader.maxPlaylistItems / 999) + #expect(parsed.reduce(0) { $0 + $1.clipIDs.count } <= DiscReader.maxPlaylistItems) + } + + @Test("a source that stops delivering ends the scan after a short run of failures") + func unreadableSourceEndsTheScan() { + let source = PlaylistSource(bytes: [], declaredLength: 100) + #expect(source.scan(entries(1000)).isEmpty) + #expect(source.readCalls < 100) + } + + @Test("an unparseable playlist does not end the scan") + func garbagePlaylistsDoNotEndTheScan() { + let garbage = [UInt8](repeating: 0xAB, count: 64) + let source = PlaylistSource(bytes: garbage) + #expect(source.scan(entries(100)).isEmpty) + #expect(source.readCalls == 100) + } + + // MARK: - Title assembly + + private func extents(_ count: Int, length: Int64 = 1 << 20, base: Int64 = 0) -> Extents { + (0.. String { guard let text = try? String(contentsOf: Self.repoRoot.appendingPathComponent(relativePath), diff --git a/Tests/AetherEngineTests/DolbyVisionRecordAuditInputBudgetTests.swift b/Tests/AetherEngineTests/DolbyVisionRecordAuditInputBudgetTests.swift new file mode 100644 index 000000000..cc73d3c97 --- /dev/null +++ b/Tests/AetherEngineTests/DolbyVisionRecordAuditInputBudgetTests.swift @@ -0,0 +1,84 @@ +import Foundation +import Testing +@testable import AetherEngine + +/// Audit BIT-103: the RPU walk bounded packets and the bytes of the packets it was handed, but not the +/// bytes libavformat consumes inside one `av_read_frame`. Matroska resyncs byte by byte through data no +/// element starts in, so a video stream followed by junk read the whole source before the walk got to +/// count anything. The fixture is that shape: two HEVC frames, then zeros. +@Suite("DV record audit: the RPU walk holds an input byte budget below the demuxer") +struct DolbyVisionRecordAuditInputBudgetTests { + + private static let junkBytes = 24 * 1024 * 1024 + private static let budget: Int64 = 1 << 20 + + private final class CountingReader: IOReader, @unchecked Sendable { + private let base: FileIOReader + private let lock = NSLock() + private var total: Int64 = 0 + + init(_ base: FileIOReader) { self.base = base } + + var bytesRead: Int64 { lock.lock(); defer { lock.unlock() }; return total } + + func read(_ buffer: UnsafeMutablePointer?, size: Int32) -> Int32 { + let n = base.read(buffer, size: size) + if n > 0 { lock.lock(); total += Int64(n); lock.unlock() } + return n + } + func seek(offset: Int64, whence: Int32) -> Int64 { base.seek(offset: offset, whence: whence) } + func close() { base.close() } + var discImageProbeEnabled: Bool { false } + } + + private func open(_ demuxer: Demuxer, _ url: URL, reader: CountingReader?) throws { + let profile = DemuxerOpenProfile.dolbyVisionRecordAuditDemuxer(callerProbesize: nil, callerMaxAnalyzeDuration: nil) + if let reader { + try demuxer.open(reader: reader, profile: profile) + } else { + try demuxer.open(url: url, profile: profile) + } + } + + @Test("a junk tail stops the walk at its byte budget on a custom reader") + func junkTailStopsOnTheBudgetOnACustomReader() throws { + let data = try TailHeavyMatroska.make(foreignBytes: Self.junkBytes, junkTail: true) + try ProbeTestFixtures.withFile(data) { url in + let reader = CountingReader(try #require(FileIOReader(url: url))) + let demuxer = Demuxer() + defer { demuxer.close() } + try open(demuxer, url, reader: reader) + let atOpen = reader.bytesRead + #expect(DolbyVisionRecordAudit.rpuProfile(walking: demuxer, byteBudget: Self.budget) == nil) + #expect(demuxer.inputByteBudgetExhausted) + #expect(reader.bytesRead - atOpen <= Self.budget) + } + } + + @Test("a junk tail stops the walk at its byte budget on a local file") + func junkTailStopsOnTheBudgetOnALocalFile() throws { + let data = try TailHeavyMatroska.make(foreignBytes: Self.junkBytes, junkTail: true) + try ProbeTestFixtures.withFile(data) { url in + let demuxer = Demuxer() + defer { demuxer.close() } + try open(demuxer, url, reader: nil) + #expect(DolbyVisionRecordAudit.rpuProfile(walking: demuxer, byteBudget: Self.budget) == nil) + #expect(demuxer.inputByteBudgetExhausted) + } + } + + @Test("a budget that covers the file reaches its end, so the cap is the budget and not the fixture") + func coveringBudgetReadsToTheEnd() throws { + let data = try TailHeavyMatroska.make(foreignBytes: Self.junkBytes, junkTail: true) + try ProbeTestFixtures.withFile(data) { url in + let reader = CountingReader(try #require(FileIOReader(url: url))) + let demuxer = Demuxer() + defer { demuxer.close() } + try open(demuxer, url, reader: reader) + let atOpen = reader.bytesRead + #expect(DolbyVisionRecordAudit.rpuProfile(walking: demuxer, byteBudget: Int64(data.count) * 2) == nil) + #expect(!demuxer.inputByteBudgetExhausted) + #expect(reader.bytesRead - atOpen > Self.budget) + } + } +} diff --git a/Tests/AetherEngineTests/DolbyVisionRecordAuditTests.swift b/Tests/AetherEngineTests/DolbyVisionRecordAuditTests.swift index d6e329c56..9d9d778bf 100644 --- a/Tests/AetherEngineTests/DolbyVisionRecordAuditTests.swift +++ b/Tests/AetherEngineTests/DolbyVisionRecordAuditTests.swift @@ -286,6 +286,15 @@ struct DolbyVisionRecordAuditTests { #expect(r.convertP7ToProfile81 == false) } + @Test("the RPU walk ends on foreign packets and bytes, not only on video packets") + func walkIsBoundedByEveryPacket() { + #expect(!DolbyVisionRecordAudit.walkExhausted(packetsRead: 63, bytesRead: 0, packetBudget: 4)) + #expect(DolbyVisionRecordAudit.walkExhausted(packetsRead: 64, bytesRead: 0, packetBudget: 4)) + #expect(DolbyVisionRecordAudit.walkExhausted( + packetsRead: 1, bytesRead: DolbyVisionRecordAudit.walkByteBudget, packetBudget: 4)) + #expect(!DolbyVisionRecordAudit.walkExhausted(packetsRead: 1, bytesRead: 0, packetBudget: Int.max)) + } + // MARK: - Against real media @Test("the relabelled fixture's RPU reports the profile its record hides", diff --git a/Tests/AetherEngineTests/DoviConvertProbeTests.swift b/Tests/AetherEngineTests/DoviConvertProbeTests.swift new file mode 100644 index 000000000..063aab2ae --- /dev/null +++ b/Tests/AetherEngineTests/DoviConvertProbeTests.swift @@ -0,0 +1,70 @@ +import Foundation +import Testing +@testable import AetherEngine + +private func fixtureURL(_ name: String) -> URL { + URL(fileURLWithPath: #filePath) + .deletingLastPathComponent() + .deletingLastPathComponent() + .deletingLastPathComponent() + .appendingPathComponent("Fixtures") + .appendingPathComponent(name) +} + +private func fixtureExists(_ name: String) -> Bool { + FileManager.default.fileExists(atPath: fixtureURL(name).path) +} + +/// `aetherctl dovitest` writes a source's video stream as Annex B for `dovi_tool`. It resolved the +/// framing of neither the converter calls nor its writer, so an Annex-B source (a Blu-ray remux) came +/// out as one bogus NAL per packet, with every packet counted as converted (audit BIT-106). +@Suite("dovitest probe") +struct DoviConvertProbeTests { + + private func nalTypes(of file: URL) throws -> [UInt8] { + let bytes = try Data(contentsOf: file) + var types: [UInt8] = [] + bytes.withUnsafeBytes { raw in + guard let base = raw.bindMemory(to: UInt8.self).baseAddress else { return } + A53SEIParser.forEachNAL(base, bytes.count, .annexB) { nal, _ in types.append((nal[0] >> 1) & 0x3F) } + } + return types + } + + @Test("A raw Annex-B stream probes to the same NALs as the hvcC file it was written from", + .enabled(if: fixtureExists("hdr10-hevc.mp4"), "needs Fixtures/hdr10-hevc.mp4 (Scripts/fetch-fixtures.sh)"), + .timeLimit(.minutes(2))) + func annexBSourceKeepsItsNALs() throws { + let dir = FileManager.default.temporaryDirectory + .appendingPathComponent("dovi-probe-\(UUID().uuidString)", isDirectory: true) + try FileManager.default.createDirectory(at: dir, withIntermediateDirectories: true) + defer { try? FileManager.default.removeItem(at: dir) } + + let first = dir.appendingPathComponent("first.hevc") + let second = dir.appendingPathComponent("second.hevc") + let fromMP4 = try AetherEngine.doviConvertProbe(url: fixtureURL("hdr10-hevc.mp4"), outputPath: first.path) + let fromAnnexB = try AetherEngine.doviConvertProbe(url: first, outputPath: second.path) + + #expect(fromMP4.videoStreamFound && fromMP4.packetsProcessed > 0) + #expect(fromAnnexB.videoStreamFound && fromAnnexB.packetsProcessed > 0) + #expect(fromAnnexB.failures == 0) + + // The raw demuxer also hands the stream's parameter sets over as extradata, so they head the + // second output once more on top of the in-band copy. Repeated parameter sets are legal Annex B. + let notParameterSet: (UInt8) -> Bool = { !(32...34).contains($0) } + let expected = try nalTypes(of: first).filter(notParameterSet) + #expect(expected.count >= fromMP4.packetsProcessed, "at least one NAL per packet") + let actual = try nalTypes(of: second).filter(notParameterSet) + #expect(actual == expected, "\(actual.count) NALs (\(actual.prefix(12))) against \(expected.count) (\(expected.prefix(12)))") + } + + @Test("An output path that cannot be opened throws instead of reporting an empty run", + .enabled(if: fixtureExists("hdr10-hevc.mp4"), "needs Fixtures/hdr10-hevc.mp4 (Scripts/fetch-fixtures.sh)")) + func unwritableOutputThrows() { + let path = FileManager.default.temporaryDirectory + .appendingPathComponent("dovi-probe-missing-\(UUID().uuidString)/out.hevc").path + #expect(throws: (any Error).self) { + _ = try AetherEngine.doviConvertProbe(url: fixtureURL("hdr10-hevc.mp4"), outputPath: path) + } + } +} diff --git a/Tests/AetherEngineTests/DoviRpuConverterTests.swift b/Tests/AetherEngineTests/DoviRpuConverterTests.swift index eee659ea2..2942a60d4 100644 --- a/Tests/AetherEngineTests/DoviRpuConverterTests.swift +++ b/Tests/AetherEngineTests/DoviRpuConverterTests.swift @@ -1,5 +1,6 @@ import Testing import AetherLibavcodec +import Dovi @testable import AetherEngine /// Deterministic NAL-walk checks for the DV P7 -> P8.1 converter (#132/#135). @@ -184,4 +185,200 @@ struct DoviRpuConverterTests { #expect(DoviRpuConverter.enhancementLayerType(pkt, framing: .annexB) == nil) #expect(annexBNALTypes(pkt) == [1, 62]) } + + // MARK: - aetherctl dovitest output (audit BIT-106) + + /// The probe's writer used to walk every packet as 4-byte-length NALs whatever the source was, + /// so an Annex-B packet holding four NALs came out as one 1-byte NAL. + @Test("The probe writes every NAL of a packet behind a start code, in either framing") + func probeEmitsEveryNALInEitherFraming() { + let nals = [hevcNAL(type: 35, payload: [0x50]), + hevcNAL(type: 19, payload: [UInt8](repeating: 0xAA, count: 40)), + hevcNAL(type: 62, payload: [UInt8](repeating: 0x11, count: 9)), + hevcNAL(type: 63, payload: [UInt8](repeating: 0x22, count: 30))] + for (framing, build) in [(VideoNALFraming.annexB, annexBPacket), + (VideoNALFraming.lengthPrefixed(size: 4), avccPacket)] { + let pkt = build(nals) + defer { var p: UnsafeMutablePointer? = pkt; av_packet_free(&p) } + let out = AetherEngine.doviProbeAnnexB(UnsafePointer(pkt), framing: framing) + var emitted: [[UInt8]] = [] + out.withUnsafeBytes { raw in + let base = raw.bindMemory(to: UInt8.self).baseAddress! + A53SEIParser.forEachNAL(base, out.count, .annexB) { nal, len in + emitted.append([UInt8](UnsafeBufferPointer(start: nal, count: len))) + } + } + #expect(emitted == nals, "framing \(framing)") + #expect(Array(out.prefix(4)) == [0, 0, 0, 1]) + } + } + + @Test("Annex-B extradata is written as it is, hvcC parameter sets are start-coded") + func probeParameterSetsFollowTheExtradataFraming() { + let annexB: [UInt8] = [0, 0, 0, 1, 0x40, 0x01, 0xAA, 0, 0, 0, 1, 0x42, 0x01, 0xBB] + let annexBOut = annexB.withUnsafeBufferPointer { + AetherEngine.doviProbeParameterSets(extradata: $0.baseAddress, size: $0.count, framing: .annexB) + } + #expect([UInt8](annexBOut) == annexB) + + var hvcC = [UInt8](repeating: 0, count: 22) + hvcC[0] = 1 + hvcC[21] = 0x03 + hvcC.append(1) // numOfArrays + hvcC += [0x20, 0x00, 0x01, 0x00, 0x03, 0x40, 0x01, 0xAA] // VPS array, one NAL of 3 bytes + let hvcCOut = hvcC.withUnsafeBufferPointer { + AetherEngine.doviProbeParameterSets(extradata: $0.baseAddress, size: $0.count, framing: .lengthPrefixed(size: 4)) + } + #expect([UInt8](hvcCOut) == [0, 0, 0, 1, 0x40, 0x01, 0xAA]) + #expect(AetherEngine.doviProbeParameterSets(extradata: nil, size: 0, framing: .annexB).isEmpty) + } + + // MARK: - Audit PERF-111: one rebuild, byte-identical to the two-copy rebuild it replaced + + /// A real unspec62 RPU NAL (139 bytes, from Dolby's Profile 8.1 test signal), so the rewrite-and- + /// keep branch is exercised and not only the degrade branch. + private static let realRPU: [UInt8] = { + let hex = "7c0119080908406136506f003ff801ffc00fffd0000008000006800000400000340000030200000301a2566000035ea2566f9fceb1c256644ca00000100000030080000003008000000301c36224301860a5e308e0514000001a63e5affff000000300000300000300060200f80e1530100a0000030000030000030024180fa000040fa00640a3c503f380" + var out: [UInt8] = [] + var i = hex.startIndex + while i < hex.endIndex { + let j = hex.index(i, offsetBy: 2) + out.append(UInt8(hex[i.. (bytes: [UInt8], ok: Bool)? { + var outputNALs: [[UInt8]] = [] + var converted = false, droppedEL = false, degraded = false + input.withUnsafeBufferPointer { buf in + A53SEIParser.forEachNAL(buf.baseAddress!, buf.count, framing) { nal, len in + switch (nal[0] >> 1) & 0x3F { + case 62: + guard let rpu = dovi_parse_unspec62_nalu(nal, len) else { degraded = true; return } + if dovi_convert_rpu_with_mode(rpu, 2) != 0 { dovi_rpu_free(rpu); degraded = true; return } + guard let out = dovi_write_unspec62_nalu(rpu) else { dovi_rpu_free(rpu); degraded = true; return } + guard let data = out.pointee.data, out.pointee.len > 0 else { + dovi_data_free(out); dovi_rpu_free(rpu); degraded = true; return + } + outputNALs.append([UInt8](UnsafeBufferPointer(start: data, count: out.pointee.len))) + dovi_data_free(out) + dovi_rpu_free(rpu) + converted = true + case 63: + droppedEL = true + default: + outputNALs.append([UInt8](UnsafeBufferPointer(start: nal, count: len))) + } + } + } + if !converted && !droppedEL && !degraded { return nil } + var bytes: [UInt8] = [] + for nal in outputNALs { + if framing == .annexB { + bytes += [0, 0, 0, 1] + } else { + let n = nal.count + bytes += [UInt8((n >> 24) & 0xFF), UInt8((n >> 16) & 0xFF), UInt8((n >> 8) & 0xFF), UInt8(n & 0xFF)] + } + bytes += nal + } + return bytes.isEmpty ? nil : (bytes, !degraded) + } + + private func packetBytes(_ pkt: UnsafeMutablePointer) -> [UInt8] { + [UInt8](UnsafeBufferPointer(start: pkt.pointee.data, count: Int(pkt.pointee.size))) + } + + private func framed(_ nals: [[UInt8]], _ framing: VideoNALFraming) -> [UInt8] { + var bytes: [UInt8] = [] + for nal in nals { + if framing == .annexB { + bytes += [0, 0, 0, 1] + } else { + let n = nal.count + bytes += [UInt8((n >> 24) & 0xFF), UInt8((n >> 16) & 0xFF), UInt8((n >> 8) & 0xFF), UInt8(n & 0xFF)] + } + bytes += nal + } + return bytes + } + + private func packet(from bytes: [UInt8]) -> UnsafeMutablePointer { + let pkt = av_packet_alloc()! + _ = av_new_packet(pkt, Int32(bytes.count)) + bytes.withUnsafeBytes { _ = memcpy(pkt.pointee.data, $0.baseAddress, bytes.count) } + return pkt + } + + private func slice(_ count: Int, seed: UInt8) -> [UInt8] { + let header = hevcNAL(type: 1, payload: []) + var state = UInt32(seed) &+ 1 + return header + (0..> 24) + } + } + + private func expectMatchesLegacy(_ nals: [[UInt8]], framing: VideoNALFraming, + expectSuccess: Bool, sourceLocation: SourceLocation = #_sourceLocation) { + let input = framed(nals, framing) + let reference = legacyRebuild(input, framing: framing) + let pkt = packet(from: input) + defer { free(pkt) } + + let ok = DoviRpuConverter.convertPacketToProfile81(pkt, framing: framing) + #expect(ok == expectSuccess, sourceLocation: sourceLocation) + if let reference { + #expect(reference.ok == ok, sourceLocation: sourceLocation) + #expect(packetBytes(pkt) == reference.bytes, "the rebuilt packet differs from the two-copy rebuild", + sourceLocation: sourceLocation) + let pad = Int(AV_INPUT_BUFFER_PADDING_SIZE) + let tail = [UInt8](UnsafeBufferPointer(start: pkt.pointee.data + Int(pkt.pointee.size), count: pad)) + #expect(tail == [UInt8](repeating: 0, count: pad), "decoders read past size", sourceLocation: sourceLocation) + } else { + #expect(packetBytes(pkt) == input, "a packet with nothing to rewrite is left alone", + sourceLocation: sourceLocation) + } + } + + @Test("A real RPU is rewritten and every other NAL survives byte for byte, in order (length-prefixed)") + func realRPUMatchesTheTwoCopyRebuild() { + let sei = hevcNAL(type: 39, payload: [0x01, 0x02, 0x03, 0x00, 0x00, 0x03, 0x01]) + expectMatchesLegacy([hevcNAL(type: 32, payload: [0x0C]), sei, slice(40_000, seed: 1), + Self.realRPU, hevcNAL(type: 63, payload: [0xCC, 0xDD])], + framing: .lengthPrefixed(size: 4), expectSuccess: true) + } + + @Test("A real RPU is rewritten and every other NAL survives byte for byte, in order (Annex B)") + func realRPUMatchesTheTwoCopyRebuildAnnexB() { + expectMatchesLegacy([slice(3_000, seed: 2), Self.realRPU, slice(9_000, seed: 3)], + framing: .annexB, expectSuccess: true) + } + + @Test("A malformed RPU degrades exactly as before: dropped, the rest untouched, failure reported") + func degradedPacketMatchesTheTwoCopyRebuild() { + expectMatchesLegacy([slice(5_000, seed: 4), hevcNAL(type: 62, payload: [0x00]), slice(700, seed: 5), + hevcNAL(type: 63, payload: [0x01])], + framing: .lengthPrefixed(size: 4), expectSuccess: false) + } + + @Test("Several RPUs in one packet are each rewritten in place of their own position") + func severalRPUsKeepTheirPositions() { + expectMatchesLegacy([slice(100, seed: 6), Self.realRPU, slice(200, seed: 7), Self.realRPU, slice(300, seed: 8)], + framing: .lengthPrefixed(size: 4), expectSuccess: true) + } + + @Test("A packet that is nothing but RPU and EL is left untouched") + func onlyRPUAndELIsLeftAlone() { + let input = framed([hevcNAL(type: 63, payload: [0x01]), hevcNAL(type: 63, payload: [0x02])], .lengthPrefixed(size: 4)) + let pkt = packet(from: input) + defer { free(pkt) } + // The EL is dropped, which would leave nothing: the degenerate guard keeps the packet. + #expect(DoviRpuConverter.convertPacketToProfile81(pkt) == true) + #expect(packetBytes(pkt) == input) + } } diff --git a/Tests/AetherEngineTests/EngineLogCaptureTests.swift b/Tests/AetherEngineTests/EngineLogCaptureTests.swift new file mode 100644 index 000000000..ded7ac650 --- /dev/null +++ b/Tests/AetherEngineTests/EngineLogCaptureTests.swift @@ -0,0 +1,24 @@ +import Foundation +import Testing +@testable import AetherEngine + +/// The property the capture exists for: two tests listening at once do not disconnect each other. +/// Swapping the global handler failed it, since whichever test restored last put back a stale one. +@Suite("Engine log capture") +struct EngineLogCaptureTests { + + @Test("ending one capture leaves an overlapping one connected") + func endingOneLeavesTheOtherConnected() { + let marker = "capture-\(UUID().uuidString)" + let first = EngineLogCapture() + let second = EngineLogCapture() + defer { second.end() } + + EngineLog.emit("\(marker) before") + first.end() + EngineLog.emit("\(marker) after") + + #expect(first.matching(marker).count == 1) + #expect(second.matching(marker).count == 2) + } +} diff --git a/Tests/AetherEngineTests/EngineSessionRedirectCredentialTests.swift b/Tests/AetherEngineTests/EngineSessionRedirectCredentialTests.swift new file mode 100644 index 000000000..76f809a98 --- /dev/null +++ b/Tests/AetherEngineTests/EngineSessionRedirectCredentialTests.swift @@ -0,0 +1,128 @@ +import Foundation +import Testing +@testable import AetherEngine + +/// Audit NET-108: URLSession's own redirect carries every custom header except `Authorization` to the +/// new host (measured: `X-Emby-Token`, `Cookie`, `Range` and `Referer` all arrived at a cross-host +/// target). The engine's owned sessions had no redirect handler, so a 302 on a playlist, segment, key +/// or disc range took the host's token wherever it pointed. The session-level delegate now applies the +/// same rule a per-task redirect handler does. +@Suite("Engine sessions keep credentials off a redirect target", .serialized, .timeLimit(.minutes(2))) +struct EngineSessionRedirectCredentialTests { + + private let credentialFields = ["X-Emby-Token": "SOURCE-ONLY", "Cookie": "sid=SOURCE-ONLY", + "Authorization": "Bearer SOURCE-ONLY"] + + private func request(to url: URL) -> URLRequest { + var request = URLRequest(url: url) + for (field, value) in credentialFields { request.setValue(value, forHTTPHeaderField: field) } + request.setValue("bytes=0-1023", forHTTPHeaderField: "Range") + request.setValue("https://portal.example/", forHTTPHeaderField: "Referer") + return request + } + + private func redirectingPair() throws -> (source: ThrottledOriginServer, target: ThrottledOriginServer) { + let target = try #require(ThrottledOriginServer(totalSize: 1024)) + let targetPort = target.port + let sourceMaybe = ThrottledOriginServer( + totalSize: 1024, + respond: { _, _, _ in .redirect(to: "http://127.0.0.1:\(targetPort)/cdn/seg.ts") }) + let source = try #require(sourceMaybe) + return (source, target) + } + + private func expectScrubbed(_ seen: [String: String], sourceLocation: SourceLocation = #_sourceLocation) { + for field in credentialFields.keys { + #expect(seen[field.lowercased()] == nil, "\(field) reached the redirect target", + sourceLocation: sourceLocation) + } + #expect(seen["range"] == "bytes=0-1023", sourceLocation: sourceLocation) + #expect(seen["referer"] == "https://portal.example/", sourceLocation: sourceLocation) + } + + @Test("a data(for:) fetch on an engine session arrives at a cross-origin target without credentials") + func sessionLevelFetch() async throws { + let (source, target) = try redirectingPair() + defer { source.stop(); target.stop() } + let session = URLSession( + configuration: .ephemeral, delegate: EngineTLS.sessionDelegate, delegateQueue: nil) + defer { session.invalidateAndCancel() } + + let url = try #require(URL(string: "http://127.0.0.1:\(source.port)/Videos/seg.ts")) + _ = try await session.data(for: request(to: url)) + + let seen = try #require(target.requestHeaders.last) + expectScrubbed(seen) + } + + /// The relay's `UpstreamPump` shape: a per-task delegate that answers the body callbacks and has + /// no redirect method of its own, so the session-level one has to answer for it. + @Test("a task delegate without a redirect method falls back to the session's rule") + func taskDelegateWithoutRedirect() async throws { + let (source, target) = try redirectingPair() + defer { source.stop(); target.stop() } + let session = URLSession( + configuration: .ephemeral, delegate: EngineTLS.sessionDelegate, delegateQueue: nil) + defer { session.invalidateAndCancel() } + + let url = try #require(URL(string: "http://127.0.0.1:\(source.port)/Videos/seg.ts")) + _ = try await session.data(for: request(to: url), delegate: BodyOnlyTaskDelegate()) + + let seen = try #require(target.requestHeaders.last) + expectScrubbed(seen) + } + + @Test("a same-origin hop and an http to https upgrade keep the credentials, a downgrade loses them", + arguments: [ + ("https://media.example/a.m3u8", "https://media.example/b.m3u8", true), + ("http://media.example:8096/a.m3u8", "https://media.example:8920/a.m3u8", true), + ("https://media.example/a.m3u8", "http://media.example/a.m3u8", false), + ("https://media.example/a.m3u8", "https://cdn.other/a.m3u8", false), + ("https://media.example/a.m3u8", "https://media.example:8443/a.m3u8", false), + ]) + func redirectDecision(from: String, to: String, keeps: Bool) async throws { + let fromURL = try #require(URL(string: from)) + let toURL = try #require(URL(string: to)) + let session = URLSession(configuration: .ephemeral) + defer { session.invalidateAndCancel() } + let task = session.dataTask(with: request(to: fromURL)) + // What URLSession hands the delegate: the new URL with the old request's headers copied over. + var carried = request(to: toURL) + carried.setValue(nil, forHTTPHeaderField: "Authorization") + let response = try #require(HTTPURLResponse( + url: fromURL, statusCode: 302, httpVersion: "HTTP/1.1", headerFields: ["Location": to])) + + let box = RequestBox() + EngineTLS.sessionDelegate.urlSession( + session, task: task, willPerformHTTPRedirection: response, newRequest: carried, + completionHandler: { box.set($0) }) + let next = try #require(box.get()) + + for field in ["X-Emby-Token", "Cookie"] { + #expect((next.value(forHTTPHeaderField: field) != nil) == keeps, "\(field) \(from) -> \(to)") + } + #expect(next.value(forHTTPHeaderField: "Range") == "bytes=0-1023") + #expect(next.value(forHTTPHeaderField: "Referer") == "https://portal.example/") + } + + /// The system-trust probe must keep running without the host's evaluator, which is what it + /// measures, while still scrubbing a redirect like every other engine session. + @Test("the redirect-only delegate answers no trust challenge") + func redirectOnlyDelegateHasNoChallengeMethod() { + let delegate: NSObject = EngineTLS.redirectDelegate + let challenge = #selector(URLSessionDelegate.urlSession(_:didReceive:completionHandler:)) + #expect(!delegate.responds(to: challenge)) + #expect(EngineTLS.sessionDelegate.responds(to: challenge)) + } +} + +private final class BodyOnlyTaskDelegate: NSObject, URLSessionTaskDelegate, @unchecked Sendable { + func urlSession(_ session: URLSession, task: URLSessionTask, didFinishCollecting metrics: URLSessionTaskMetrics) {} +} + +private final class RequestBox: @unchecked Sendable { + private let lock = NSLock() + private var value: URLRequest? + func set(_ request: URLRequest?) { lock.withLock { value = request } } + func get() -> URLRequest? { lock.withLock { value } } +} diff --git a/Tests/AetherEngineTests/ForwardOnlyHeadIsKeptTests.swift b/Tests/AetherEngineTests/ForwardOnlyHeadIsKeptTests.swift new file mode 100644 index 000000000..78d896149 --- /dev/null +++ b/Tests/AetherEngineTests/ForwardOnlyHeadIsKeptTests.swift @@ -0,0 +1,245 @@ +import Testing +import Foundation +import AetherLibavcodec +@testable import AetherEngine + +/// Audit HLS-103 (70dc485d was incomplete): a forward-only source cannot rewind, so whatever the +/// start-up probes read off its head is gone from the archive. `probeVideoNALFraming` (Annex-B +/// extradata) and the in-band hvcC rebuild both did `seek(to: 0)` and then consumed packets; FFmpeg +/// flushes the packet queue before it even tries the seek, and 70dc485d's step-6 rewind is skipped +/// on these sources, so the opening GOP never reached the producer. +@Suite("A forward-only source keeps its opening packets", .serialized) +struct ForwardOnlyHeadIsKeptTests { + + /// The tiny H.264 TS as a source that cannot be repositioned: every seek is refused, which is how + /// the demuxer learns the source is forward-only. + private final class ForwardOnlyReader: IOReader, @unchecked Sendable { + private let bytes = TinyTransportStreamFixture.data + private let lock = NSLock() + private var position = 0 + + func read(_ buffer: UnsafeMutablePointer?, size: Int32) -> Int32 { + guard let buffer, size > 0 else { return -1 } + return lock.withLock { + let n = min(Int(size), bytes.count - position) + guard n > 0 else { return 0 } + bytes.withUnsafeBytes { raw in _ = memcpy(buffer, raw.baseAddress! + position, n) } + position += n + return Int32(n) + } + } + + func seek(offset: Int64, whence: Int32) -> Int64 { -1 } + func close() {} + var discImageProbeEnabled: Bool { false } + } + + struct Signature: Equatable { + let streamIndex: Int32 + let pts: Int64 + let dts: Int64 + let size: Int32 + } + + private static func signature(_ packet: UnsafeMutablePointer) -> Signature { + Signature(streamIndex: packet.pointee.stream_index, pts: packet.pointee.pts, + dts: packet.pointee.dts, size: packet.pointee.size) + } + + private func openForwardOnly() throws -> Demuxer { + let demuxer = Demuxer() + try demuxer.open(reader: ForwardOnlyReader(), formatHint: "mpegts") + return demuxer + } + + /// The packets a plain read delivers, in order: what the producer would see with no probe at all. + private func firstPackets(_ count: Int) throws -> [Signature] { + let demuxer = try openForwardOnly() + defer { demuxer.close() } + var out: [Signature] = [] + while out.count < count, let read = try demuxer.readPacket() { + out.append(Self.signature(read)) + var packet: UnsafeMutablePointer? = read + trackedPacketFree(&packet) + } + return out + } + + private func readSome(_ demuxer: Demuxer, _ count: Int) throws -> [Signature] { + var out: [Signature] = [] + while out.count < count, let read = try demuxer.readPacket() { + out.append(Self.signature(read)) + var packet: UnsafeMutablePointer? = read + trackedPacketFree(&packet) + } + return out + } + + @Test("packets peeked on a forward-only source are read again, in order, from the first") + func peekedPacketsAreReadAgain() throws { + let expected = try firstPackets(12) + + let demuxer = try openForwardOnly() + defer { demuxer.close() } + #expect(!demuxer.isSourceSeekable) + var peeked: [Signature] = [] + try demuxer.peekPackets(maxPackets: 6) { packet in + peeked.append(Self.signature(packet)) + return false + } + #expect(peeked == Array(expected.prefix(6))) + + // A second look starts from the head again and extends the held run. + var second: [Signature] = [] + try demuxer.peekPackets(maxPackets: 9) { packet in + second.append(Self.signature(packet)) + return false + } + #expect(second == Array(expected.prefix(9))) + + #expect(try readSome(demuxer, 12) == expected, + "a peek changed what the producer reads") + } + + @Test("a peek that has seen enough stops and holds only what it read") + func peekStopsWhenSatisfied() throws { + let expected = try firstPackets(4) + let demuxer = try openForwardOnly() + defer { demuxer.close() } + var inspected = 0 + try demuxer.peekPackets(maxPackets: 100) { _ in + inspected += 1 + return inspected == 3 + } + #expect(inspected == 3) + #expect(try readSome(demuxer, 4) == expected) + } + + @Test("a refused seek on a forward-only source leaves its packets alone") + func refusedSeeksKeepTheHead() throws { + let expected = try firstPackets(12) + let demuxer = try openForwardOnly() + defer { demuxer.close() } + try demuxer.peekPackets(maxPackets: 4) { _ in false } + + #expect(!demuxer.seek(to: 0), "a forward-only source was asked to rewind") + #expect(!demuxer.seek(to: 0, streamIndex: demuxer.videoStreamIndex)) + #expect(!demuxer.seekBounded(to: 0, timeout: 1)) + #expect(!demuxer.seekByteEstimate(to: 1, knownDuration: 8, timeout: 1)) + + #expect(try readSome(demuxer, 12) == expected, "a refused seek flushed the packets behind it") + } + + @Test("a refused seek on a forward-only source keeps the packets the open read ahead") + func refusedSeekKeepsTheOpenQueue() throws { + let expected = try firstPackets(12) + let demuxer = try openForwardOnly() + defer { demuxer.close() } + + #expect(!demuxer.seek(to: 0), "a forward-only source was asked to rewind") + #expect(try readSome(demuxer, 12) == expected, + "the refused seek flushed what find_stream_info had buffered") + } + + @Test("a seek on a source that can rewind drops what was peeked") + func seekableSourcesDropThePeek() throws { + let demuxer = Demuxer() + try demuxer.open(reader: TinyTransportStreamFixture.LiveReader(), formatHint: "mpegts") + defer { demuxer.close() } + #expect(demuxer.isSourceSeekable) + var seen: [Signature] = [] + try demuxer.peekPackets(maxPackets: 8) { seen.append(Self.signature($0)); return false } + #expect(seen.count == 8) + + // Three of the held packets go out, five stay held; a rewind must not hand those five out. + _ = try readSome(demuxer, 3) + #expect(demuxer.seek(to: 0)) + let after = try #require(try readSome(demuxer, 1).first) + #expect(after == seen[0], "a held packet came back after the rewind instead of the head") + } + + @Test("the NAL framing probe leaves the packets it looked at for the producer") + func framingProbeDoesNotConsume() throws { + let expected = try firstPackets(12) + #expect(expected.count == 12) + + let demuxer = try openForwardOnly() + defer { demuxer.close() } + #expect(!demuxer.isSourceSeekable, "the fixture reader must read as forward-only") + let engine = HLSVideoEngine(url: URL(fileURLWithPath: "/nonexistent/forward-only.ts"), + dvModeAvailable: false) + let framing = engine.probeVideoNALFraming(demuxer: demuxer, videoStreamIndex: demuxer.videoStreamIndex) + #expect(framing == .annexB) + + #expect(try readSome(demuxer, 12) == expected, + "the probe consumed packets the producer will never see") + } + + /// Video samples (track 1) across every fragment of a segment, from the `trun` sample counts. + private static func videoSampleCount(_ segment: Data) -> Int { + func u32(_ off: Int) -> UInt32 { + segment.withUnsafeBytes { UInt32(bigEndian: $0.loadUnaligned(fromByteOffset: off, as: UInt32.self)) } + } + func boxes(_ range: Range) -> [(String, Range)] { + var out: [(String, Range)] = [] + var off = range.lowerBound + while off + 8 <= range.upperBound { + let size = Int(u32(off)) + guard size >= 8, off + size <= range.upperBound else { break } + out.append((String(decoding: segment[off + 4.. Bool { + VideoRoutingPolicy.promotesForwardOnlySourceToSequential( + isSourceSeekable: seekable, isLive: live, declaredSequential: declared, + isCustomSource: custom, routedSoftware: software, preferred: preferred, + containerDurationSeconds: duration) + } + + @Test("a forward-only VOD URL with a container duration is promoted") + func promoted() { + #expect(promotes()) + } + + @Test("everything the promotion must leave alone") + func leftAlone() { + #expect(!promotes(seekable: true), "a seekable source keeps the ordinary native path") + #expect(!promotes(live: true), "live is exempt from the forward-only rule already") + #expect(!promotes(declared: true), "a host declaration needs no promotion") + #expect(!promotes(custom: true), "a custom reader is the host's to describe") + #expect(!promotes(software: true), "the promotion never moves a session between hosts") + #expect(!promotes(preferred: .software), "a host that asked for software gets software") + } + + @Test("no usable duration keeps the software path") + func needsDuration() { + for duration in [0, -1, .nan, .infinity] as [Double] { + #expect(!promotes(duration: duration), "promoted with duration \(duration)") + } + } + + @Test("only the software host leaves the picture behind on a wireless receiver") + func pictureStaysLocal() { + #expect(AetherEngine.airPlayPictureStaysLocal(backend: .software, wirelessAirPlayRoute: true)) + for backend in [PlaybackBackend.native, .audio, .none] { + #expect(!AetherEngine.airPlayPictureStaysLocal(backend: backend, wirelessAirPlayRoute: true), + "\(backend) reported a local picture") + } + #expect(!AetherEngine.airPlayPictureStaysLocal(backend: .software, wirelessAirPlayRoute: false)) + } +} diff --git a/Tests/AetherEngineTests/FrameDecodeContextHardwareTests.swift b/Tests/AetherEngineTests/FrameDecodeContextHardwareTests.swift index 7d5751f0e..85f254e9f 100644 --- a/Tests/AetherEngineTests/FrameDecodeContextHardwareTests.swift +++ b/Tests/AetherEngineTests/FrameDecodeContextHardwareTests.swift @@ -30,6 +30,38 @@ struct FrameDecodeContextHardwareTests { #expect(context.hardwareDecoderName == "none") } + /// Audit BIT-102: a cache-backed still went through VideoToolbox for every title, including Dolby + /// Vision Profile 5 / 10.0 whose planes are IPT-PQ-C2. VideoToolbox's P010 output skips the DV + /// converter, which only takes `yuv420p10le`, so those thumbnails carried the #103 cast again. + @Test("a no-base-layer Dolby Vision record never takes the hardware path") + func noBaseLayerStaysOnSoftware() { + #expect(FrameDecodeContext.stillUsesHardware(allows: true, disabled: false, dvNoBaseLayer: false)) + #expect(!FrameDecodeContext.stillUsesHardware(allows: true, disabled: false, dvNoBaseLayer: true)) + #expect(!FrameDecodeContext.stillUsesHardware(allows: true, disabled: true, dvNoBaseLayer: false)) + #expect(!FrameDecodeContext.stillUsesHardware(allows: false, disabled: false, dvNoBaseLayer: false)) + } + + private static let profile5FixtureURL = URL(fileURLWithPath: #filePath) + .deletingLastPathComponent() + .deletingLastPathComponent() + .deletingLastPathComponent() + .appendingPathComponent( + "Fixtures/user/Patterns_Of_Nature_DoVi_24_P5_UHD_HEVC-10mbps_DD+JOC-768kbps_iOS.mp4") + + /// Dolby's own Profile 5 signal (see `DolbyVisionRecordAuditTests`), which cannot be committed. + @Test("a real Profile 5 title opened from a cache reader decodes in software", + .enabled(if: FileManager.default.fileExists(atPath: FrameDecodeContextHardwareTests.profile5FixtureURL.path))) + func profile5FromCacheReaderIsSoftware() throws { + let data = try Data(contentsOf: Self.profile5FixtureURL, options: .alwaysMapped) + let context = FrameDecodeContext(reader: DataIOReader(data: data), formatHint: "mp4") + defer { context.close() } + + try context.ensureOpen() + + #expect(context.isDolbyVisionNoBaseLayer) + #expect(context.hardwareDecoderName == "none") + } + /// One 64x64 MPEG-4 Part 2 frame. VideoToolbox does not offer that legacy codec, while /// FFmpeg's software decoder does; this is the deterministic decline/fallback witness. /// Regenerate with: diff --git a/Tests/AetherEngineTests/H264CompositionOffsetRepairTests.swift b/Tests/AetherEngineTests/H264CompositionOffsetRepairTests.swift index 322edeccb..d01688edd 100644 --- a/Tests/AetherEngineTests/H264CompositionOffsetRepairTests.swift +++ b/Tests/AetherEngineTests/H264CompositionOffsetRepairTests.swift @@ -177,6 +177,29 @@ struct H264CompositionOffsetRepairTests { #expect(landingDTS == 498498) } + /// Audit BIT-101: in fragmented MP4 the step comes from `tfdt` (64-bit) and a landing picture + /// order can sit near -2^31 through `delta_pic_order_cnt_bottom`, so the re-anchor product + /// leaves Int64. The picture stays unrepaired and the next keyframe gets another chance. + @Test("a re-anchor whose arithmetic leaves Int64 is refused, not trapped on") + func reanchorOverflowIsRefused() { + var rewriter = H264CompositionOffsetRepair.Rewriter( + plan: H264CompositionOffsetRepair.Plan(step: 1 << 34, decodeLead: 1 << 35, shift: 0, pocStep: 2)) + rewriter.noteSeek() + #expect(rewriter.rewrite(dts: 1 << 40, pictureOrderCount: -2_147_482_646, isKeyframe: true) == nil) + #expect(rewriter.unrepairedPictures == 1) + #expect(rewriter.sequenceAnchorDTS == nil) + let landing = rewriter.rewrite(dts: 1 << 40, pictureOrderCount: 0, isKeyframe: true) + #expect(landing?.pts == 1 << 40) + } + + @Test("a presentation shift between opposite ends of Int64 falls back to no shift") + func presentationShiftOverflow() { + #expect(H264CompositionOffsetRepair.presentationShift( + streamStartTime: Int64.max - 1, ladderStart: -(Int64.max - 1), decodeLead: 2002) == 0) + #expect(H264CompositionOffsetRepair.presentationShift( + streamStartTime: -(Int64.max - 1), ladderStart: Int64.max - 1, decodeLead: 2002) == 0) + } + @Test("without an anchor a picture is emitted untouched rather than guessed at") func passesThroughWithoutAnchor() { var rewriter = H264CompositionOffsetRepair.Rewriter(plan: plan) diff --git a/Tests/AetherEngineTests/H264SPSTests.swift b/Tests/AetherEngineTests/H264SPSTests.swift index 92a90e3a8..e0bc14182 100644 --- a/Tests/AetherEngineTests/H264SPSTests.swift +++ b/Tests/AetherEngineTests/H264SPSTests.swift @@ -47,24 +47,29 @@ final class H264SPSTests: XCTestCase { bytes.withUnsafeBufferPointer { body($0) } } - // #133: a mid-stream join must start on a true IDR (NAL type 5), not an open-GOP recovery - // point (non-IDR slice, type 1), or the panel renders references it never received (green frames). - func testContainsIDRDetectsType5Slice() { - let au = annexB([sps720, pps, [0x65, 0x88, 0x84, 0x00]]) // 0x65 = IDR slice (type 5) - XCTAssertTrue(withBuf(au) { H264SPS.containsIDR(fromAnnexB: $0) }) + // #133 / AE#627: a mid-stream join opens on an entry point: an IDR (NAL type 5), or an intra + // picture behind a recovery point SEI with recovery_frame_cnt 0. A bare non-IDR slice is neither. + private var iSlice: [UInt8] { [0x41, 0x88, 0x84, 0x00] } // non-IDR, first_mb 0, slice_type 7 (I) + private var pSlice: [UInt8] { [0x41, 0x9a, 0x00, 0x00] } // non-IDR, first_mb 0, slice_type 5 (P) + private var recoveryNow: [UInt8] { [0x06, 0x06, 0x01, 0xc4, 0x80] } // recovery point, cnt 0 (x264's bytes) + private var recoveryLater: [UInt8] { [0x06, 0x06, 0x01, 0x71, 0x80] } // recovery point, cnt 2 (gradual refresh) + + func testIDRIsAnEntry() { + let au = annexB([sps720, pps, [0x65, 0x88, 0x84, 0x00]]) + XCTAssertTrue(withBuf(au) { H264SPS.isRandomAccessEntry(fromAnnexB: $0) }) } - func testContainsIDRRejectsNonIDRAccessUnit() { - let au = annexB([sps720, pps, [0x41, 0x9a, 0x00, 0x00]]) // 0x41 = non-IDR slice (type 1) - XCTAssertFalse(withBuf(au) { H264SPS.containsIDR(fromAnnexB: $0) }) + func testBareNonIDRSliceIsNotAnEntry() { + XCTAssertFalse(withBuf(annexB([sps720, pps, pSlice])) { H264SPS.isRandomAccessEntry(fromAnnexB: $0) }) + XCTAssertFalse(withBuf(annexB([sps720, pps, iSlice])) { H264SPS.isRandomAccessEntry(fromAnnexB: $0) }) } - func testContainsIDRRejectsBareParameterSets() { - let au = annexB([sps720, pps]) // params only, no coded slice - XCTAssertFalse(withBuf(au) { H264SPS.containsIDR(fromAnnexB: $0) }) + func testBareParameterSetsAreNotAnEntry() { + XCTAssertFalse(withBuf(annexB([sps720, pps])) { H264SPS.isRandomAccessEntry(fromAnnexB: $0) }) + XCTAssertFalse(withBuf(annexB([sps720, pps, recoveryNow])) { H264SPS.isRandomAccessEntry(fromAnnexB: $0) }) } - func testContainsIDRHandlesThreeByteStartCodes() { + func testEntryHandlesThreeByteStartCodes() { let sc3: [UInt8] = [0, 0, 1] var au: [UInt8] = sc3 au += sps720 @@ -72,17 +77,47 @@ final class H264SPSTests: XCTestCase { au += pps au += sc3 au += [0x65, 0x88] - XCTAssertTrue(withBuf(au) { H264SPS.containsIDR(fromAnnexB: $0) }) + XCTAssertTrue(withBuf(au) { H264SPS.isRandomAccessEntry(fromAnnexB: $0) }) + } + + // AE#627: the reporter's feed never sends an IDR. Its entry points are AUD, SPS, PPS, SEI + // [recovery point cnt 0, pic_timing], then an I-picture. + func testImmediateIntraRecoveryPointIsAnEntry() { + let aud: [UInt8] = [0x09, 0xf0] + let au = annexB([aud, sps720, pps, recoveryNow, iSlice]) + XCTAssertTrue(withBuf(au) { H264SPS.isRandomAccessEntry(fromAnnexB: $0) }) + } + + func testRecoveryPointBehindAnotherSEIMessageIsFound() { + let sei: [UInt8] = [0x06, 0x05, 0x02, 0xaa, 0xbb, 0x06, 0x01, 0xc4, 0x80] + XCTAssertTrue(withBuf(annexB([sps720, pps, sei, iSlice])) { H264SPS.isRandomAccessEntry(fromAnnexB: $0) }) + } + + func testGradualRefreshRecoveryPointIsNotAnEntry() { + XCTAssertFalse(withBuf(annexB([sps720, pps, recoveryLater, iSlice])) { + H264SPS.isRandomAccessEntry(fromAnnexB: $0) + }) + } + + func testRecoveryPointOnAPictureWithAPSliceIsNotAnEntry() { + XCTAssertFalse(withBuf(annexB([sps720, pps, recoveryNow, pSlice])) { H264SPS.isRandomAccessEntry(fromAnnexB: $0) }) + XCTAssertFalse(withBuf(annexB([sps720, pps, recoveryNow, iSlice, pSlice])) { + H264SPS.isRandomAccessEntry(fromAnnexB: $0) + }) + } + + func testRecoveryFrameCountParsesTheMessage() { + XCTAssertEqual(H264SPS.recoveryFrameCount(seiRBSP: [0x06, 0x01, 0xc4, 0x80]), 0) + XCTAssertEqual(H264SPS.recoveryFrameCount(seiRBSP: [0x06, 0x01, 0x71, 0x80]), 2) + XCTAssertNil(H264SPS.recoveryFrameCount(seiRBSP: [0x05, 0x02, 0xaa, 0xbb, 0x80])) + XCTAssertNil(H264SPS.recoveryFrameCount(seiRBSP: [0x06, 0x09, 0xc4])) // size past the end } - // #133 join gate: the decodable-access-unit predicate needs SPS + PPS + IDR all present. - // A recovery-point AU (params present, but slice is non-IDR) must NOT satisfy it. func testExtractSPSandPPSStillSucceedsOnRecoveryPointAU() { - let au = annexB([sps720, pps, [0x41, 0x9a]]) + let au = annexB([sps720, pps, recoveryNow, iSlice]) let got = withBuf(au) { H264SPS.extractSPSandPPS(fromAnnexB: $0) } - XCTAssertNotNil(got) // params are there + XCTAssertNotNil(got) XCTAssertEqual(H264SPS.dimensions(fromNAL: got!.sps)?.width, 1280) - XCTAssertFalse(withBuf(au) { H264SPS.containsIDR(fromAnnexB: $0) }) // but no IDR -> gate stays closed } // MARK: - #150 frame_mbs_only_flag fallback diff --git a/Tests/AetherEngineTests/HDR10PlusMetadataScanTests.swift b/Tests/AetherEngineTests/HDR10PlusMetadataScanTests.swift index 2e4bad2bb..3be54aa0c 100644 --- a/Tests/AetherEngineTests/HDR10PlusMetadataScanTests.swift +++ b/Tests/AetherEngineTests/HDR10PlusMetadataScanTests.swift @@ -163,6 +163,29 @@ struct HDR10PlusMetadataScanTests { #expect(scan(Self.annexB([0x4E, 0x01] + escaped))) } + /// Audit BIT-4: the prefilter must never hide a payload the full walk would confirm, in any + /// framing, and must turn away a packet that has no T.35 header at all. + @Test("The T.35 header prefilter keeps every carriage and rejects packets without the header") + func t35PrefilterIsExact() { + let carriers: [[UInt8]] = [ + Self.annexB(Self.sei()), + Self.annexB(Self.sei(hevc: false), fourBytes: true), + Self.lengthPrefixed(Self.sei(), width: 4), + Self.annexB([0x4E, 0x01] + Self.escaped( + Self.message(Self.t35(windows: 3, grids: true, toneMapping: true)) + [0x80])), + ] + for bytes in carriers { + #expect(bytes.withUnsafeBufferPointer { + HDR10PlusMetadataScan.mayContainT35Header($0.baseAddress!, size: $0.count) + }) + } + let slice = Self.annexB([0x26, 0x01] + [UInt8](repeating: 0x5A, count: 4096)) + #expect(!slice.withUnsafeBufferPointer { + HDR10PlusMetadataScan.mayContainT35Header($0.baseAddress!, size: $0.count) + }) + #expect(!scan(slice)) + } + @Test("Raw markers, slices, parameter sets, and unregistered SEI never confirm") func nonMetadataMarkers() { #expect(!scan(Self.header)) diff --git a/Tests/AetherEngineTests/HDR10PlusScanInputBudgetTests.swift b/Tests/AetherEngineTests/HDR10PlusScanInputBudgetTests.swift new file mode 100644 index 000000000..903e7e2d2 --- /dev/null +++ b/Tests/AetherEngineTests/HDR10PlusScanInputBudgetTests.swift @@ -0,0 +1,152 @@ +import Testing +import Foundation +@testable import AetherEngine + +/// The HDR10+ scan drops every stream but video (AVDISCARD_ALL), and the demuxer reads and throws away +/// those streams' blocks inside one `av_read_frame`, where the scan's packet and packet-byte caps never see +/// them. The fixture is the adversarial shape: two tiny video packets, then ~24 MiB of another stream. The +/// scan must stop on its input budget instead of reading the file to its end. +@Suite("HDR10+ scan: the input byte budget holds below the demuxer") +struct HDR10PlusScanInputBudgetTests { + + @Test("A tail of dropped foreign blocks stops the scan on its byte cap, on local and custom inputs", + arguments: [false, true]) + func foreignTailHitsTheInputBudget(customReader: Bool) throws { + let data = try TailHeavyMatroska.make(foreignBytes: 24 * 1024 * 1024) + try ProbeTestFixtures.withFile(data) { url in + let outcome = try Self.scan(url, customReader: customReader, maxBytes: 1024) + #expect(outcome.stopReason == .byteCap) + #expect(outcome.packetsRead == 2) + } + } + + @Test("A budget that covers the file still reaches EOF, so the cap is the budget and not the fixture") + func coveringBudgetReachesEOF() throws { + let data = try TailHeavyMatroska.make(foreignBytes: 24 * 1024 * 1024) + try ProbeTestFixtures.withFile(data) { url in + let outcome = try Self.scan(url, customReader: false, maxBytes: 16 * 1024 * 1024) + #expect(outcome.stopReason == .demuxEOF) + } + } + + @Test("The input budget is four times maxBytes with a 4 MiB floor, and saturates") + func inputBudgetArithmetic() { + #expect(AetherEngine.hdr10PlusInputByteBudget(maxBytes: 16 * 1024 * 1024) == 64 * 1024 * 1024) + #expect(AetherEngine.hdr10PlusInputByteBudget(maxBytes: 0) == 4 * 1024 * 1024) + #expect(AetherEngine.hdr10PlusInputByteBudget(maxBytes: -5) == 4 * 1024 * 1024) + #expect(AetherEngine.hdr10PlusInputByteBudget(maxBytes: .max) == .max) + } + + private static func scan(_ url: URL, customReader: Bool, maxBytes: Int64) throws -> HDR10PlusDetectionOutcome { + let demuxer = Demuxer() + defer { demuxer.close() } + if customReader { + try demuxer.open(reader: try #require(FileIOReader(url: url))) + } else { + try demuxer.open(url: url) + } + return AetherEngine.detectHDR10Plus( + demuxer: demuxer, videoIndex: demuxer.videoStreamIndex, + options: HDR10PlusDetectionOptions(maxPackets: 32, maxBytes: maxBytes, timeBudget: 60)) + } +} + +/// Matroska with the plain HDR10 fixture's two HEVC frames at the head and a long PCM tail after them. +/// `junkTail` swaps the PCM clusters for zeros that no EBML element starts in, which libavformat +/// resyncs through byte by byte inside a single `av_read_frame`. +enum TailHeavyMatroska { + + static func make(foreignBytes: Int, junkTail: Bool = false) throws -> Data { + let mp4 = try ProbeTestFixtures.decode(HDR10PlusProbeIntegrationTests.hdr10Base64) + let (hvcC, frames) = try hevcTrack(mp4) + + var out = Data() + out += element([0x1A, 0x45, 0xDF, 0xA3], + element([0x42, 0x82], Array("matroska".utf8)) + + element([0x42, 0x87], uint(4)) + + element([0x42, 0x85], uint(2))) + + let info = element([0x15, 0x49, 0xA9, 0x66], element([0x2A, 0xD7, 0xB1], uint(1_000_000))) + let video = element([0xAE], + element([0xD7], uint(1)) + element([0x73, 0xC5], uint(1)) + + element([0x83], uint(1)) + element([0x86], Array("V_MPEGH/ISO/HEVC".utf8)) + + element([0x63, 0xA2], hvcC) + + // DefaultDuration: without a frame rate, stream analysis at open reads + // (and queues) seconds of the tail looking for one. + element([0x23, 0xE3, 0x83], uint(40_000_000)) + + element([0xE0], element([0xB0], uint(64)) + element([0xBA], uint(64)))) + let audio = element([0xAE], + element([0xD7], uint(2)) + element([0x73, 0xC5], uint(2)) + + element([0x83], uint(2)) + element([0x86], Array("A_PCM/INT/LIT".utf8)) + + element([0xE1], element([0xB5], float64(48_000)) + + element([0x9F], uint(2)) + element([0x62, 0x64], uint(16)))) + let tracks = element([0x16, 0x54, 0xAE, 0x6B], video + audio) + + // 32 KiB of 16-bit stereo at 48 kHz is 170 ms. One cluster per block keeps every relative + // timestamp at zero. + let blockBytes = 32 * 1024 + var clusters: [UInt8] = [] + clusters += element([0x1F, 0x43, 0xB6, 0x75], + element([0xE7], uint(0)) + + simpleBlock(track: 1, relative: 0, payload: frames[0]) + + simpleBlock(track: 1, relative: 40, payload: frames[1])) + let silence = [UInt8](repeating: 0, count: blockBytes) + if junkTail { clusters += [UInt8](repeating: 0, count: foreignBytes) } + for i in 0..<(junkTail ? 0 : max(1, foreignBytes / blockBytes)) { + clusters += element([0x1F, 0x43, 0xB6, 0x75], + element([0xE7], uint(100 + i * 170)) + + simpleBlock(track: 2, relative: 0, payload: silence)) + } + out += element([0x18, 0x53, 0x80, 0x67], info + tracks + clusters) + return out + } + + /// The hvcC payload and the sample bytes of the fixture's single video track (one chunk). + private static func hevcTrack(_ mp4: Data) throws -> (hvcC: [UInt8], frames: [[UInt8]]) { + let bytes = [UInt8](mp4) + func box(_ type: String) throws -> Int { + let tag = Array(type.utf8) + let at = try #require((4...(bytes.count - 4)).first { Array(bytes[$0..<$0 + 4]) == tag }) + return at - 4 + } + func u32(_ i: Int) -> Int { bytes[i.. [UInt8] { + element([0xA3], [0x80 | track, UInt8((relative >> 8) & 0xFF), UInt8(relative & 0xFF), 0x80] + payload) + } + + private static func element(_ id: [UInt8], _ payload: [UInt8]) -> [UInt8] { + var size: [UInt8] = [0x01] + for shift in stride(from: 48, through: 0, by: -8) { size.append(UInt8((payload.count >> shift) & 0xFF)) } + return id + size + payload + } + + private static func uint(_ v: Int) -> [UInt8] { + var bytes: [UInt8] = [] + var v = v + repeat { + bytes.insert(UInt8(v & 0xFF), at: 0) + v >>= 8 + } while v > 0 + return bytes + } + + private static func float64(_ v: Double) -> [UInt8] { + let bits = v.bitPattern + return (0..<8).map { UInt8((bits >> (56 - 8 * UInt64($0))) & 0xFF) } + } +} diff --git a/Tests/AetherEngineTests/HDRVividProbeTests.swift b/Tests/AetherEngineTests/HDRVividProbeTests.swift new file mode 100644 index 000000000..5eba7b856 --- /dev/null +++ b/Tests/AetherEngineTests/HDRVividProbeTests.swift @@ -0,0 +1,128 @@ +import Foundation +import Testing +@testable import AetherEngine + +/// #699: HDR Vivid (CUVA T/UWA 005.1) detection in the opt-in probe pass. The fixture is a two-frame +/// 64x64 HEVC/HLG MP4 with a CUVA T.35 SEI that ffprobe reports as "HDR Dynamic Metadata CUVA 005.1 +/// 2021 (Vivid)", built by `Scripts/make-hdr10plus-fixture.py --vivid`. +@Suite("HDR Vivid probe detection (#699)") +struct HDRVividProbeTests { + + // MARK: - the body validator + + /// The fixture's CUVA body: one tone-mapping parameter set with base curve and one 3-spline, two + /// saturation gains, 207 bits and one zero padding bit. + private static let body: [UInt8] = [ + 0x01, 0x00, 0x0B, 0xE6, 0x5F, 0xFF, 0xFF, 0xAB, 0x4A, 0xB3, 0x33, 0x0F, 0xF0, + 0x00, 0x52, 0x88, 0x01, 0x1C, 0x00, 0x00, 0xFF, 0xBF, 0xEF, 0xF4, 0x4C, 0x32, + ] + + private func complete(_ bytes: [UInt8]) -> Bool { + bytes.withUnsafeBufferPointer { HDRVividMetadataScan.bodyIsComplete($0) } + } + + @Test("a complete CUVA body validates, with or without trailing zero bytes") + func completeBody() { + #expect(complete(Self.body)) + #expect(complete(Self.body + [0, 0])) + } + + @Test("a body cut short of its last field is not evidence") + func truncatedBody() { + #expect(!complete(Array(Self.body.dropLast()))) + } + + @Test("a set bit after the last field is not evidence") + func nonZeroPadding() { + var bytes = Self.body + bytes[bytes.count - 1] |= 1 + #expect(!complete(bytes)) + #expect(!complete(Self.body + [0x80])) + } + + @Test("only system_start_code 1 to 7 has a defined syntax") + func startCodeRange() { + for code: UInt8 in [0, 8, 0xFF] { + var bytes = Self.body + bytes[0] = code + #expect(!complete(bytes), "start code \(code)") + } + var seven = Self.body + seven[0] = 7 + #expect(complete(seven)) + } + + @Test("a cap keeps what the pass already confirmed, and the default reads as the HDR10+ target") + func outcomeKeepsPartialFindings() { + let partial = HDR10PlusDetectionOutcome( + stopReason: .packetCap, packetsRead: 32, bytesRead: 1000, found: .hdr10Plus) + #expect(partial.carriesHDR10Plus) + #expect(!partial.carriesHDRVivid) + #expect(HDR10PlusDetectionOutcome(stopReason: .found, packetsRead: 1, bytesRead: 1).carriesHDR10Plus) + #expect(!HDR10PlusDetectionOutcome(stopReason: .found, packetsRead: 1, bytesRead: 1).carriesHDRVivid) + } + + // MARK: - end to end + + private static func writeFixture(_ base64: String, name: String) throws -> URL { + let data = try #require(Data(base64Encoded: base64, options: .ignoreUnknownCharacters)) + let url = FileManager.default.temporaryDirectory + .appendingPathComponent("aether-hdr-vivid-\(name)-\(UUID().uuidString).mp4") + try data.write(to: url) + return url + } + + @Test("the base probe does not read packets, so it does not report HDR Vivid") + func baseProbeIsOptIn() throws { + let url = try Self.writeFixture(Self.vividBase64, name: "base") + defer { try? FileManager.default.removeItem(at: url) } + let probe = try AetherEngine.probe(url: url) + #expect(probe.videoFormat == .hlg) + #expect(!probe.carriesHDRVividMetadata) + } + + @Test("asking for .hdrVivid finds the CUVA payload and leaves the HLG label alone") + func detectingFindsVivid() throws { + let url = try Self.writeFixture(Self.vividBase64, name: "vivid") + defer { try? FileManager.default.removeItem(at: url) } + let probe = try AetherEngine.probe(url: url, detecting: .hdrVivid) + #expect(probe.carriesHDRVividMetadata) + #expect(!probe.carriesHDR10PlusMetadata) + #expect(probe.videoFormat == .hlg) + } + + @Test("asking for both reports each format only on the source that carries it") + func bothTargetsStaySeparate() throws { + let vivid = try Self.writeFixture(Self.vividBase64, name: "both-vivid") + let plus = try Self.writeFixture(HDR10PlusProbeIntegrationTests.hdr10PlusBase64, name: "both-plus") + defer { + try? FileManager.default.removeItem(at: vivid) + try? FileManager.default.removeItem(at: plus) + } + let vividProbe = try AetherEngine.probe(url: vivid, detecting: [.hdr10Plus, .hdrVivid]) + #expect(vividProbe.carriesHDRVividMetadata) + #expect(!vividProbe.carriesHDR10PlusMetadata) + let plusProbe = try AetherEngine.probe(url: plus, detecting: [.hdr10Plus, .hdrVivid]) + #expect(plusProbe.carriesHDR10PlusMetadata) + #expect(plusProbe.videoFormat == .hdr10Plus) + #expect(!plusProbe.carriesHDRVividMetadata) + } + + static let vividBase64 = """ + AAAAHGZ0eXBpc29tAAACAGlzb21pc28ybXA0MQAAAAhmcmVlAAAAfm1kYXQAAAAkTgEEHyYABAAFAQAL5l///6tKszMP8ABSiAEc + AAD/v+/0TDKAAAAADigBr3jrrvv//FtlXy08AAAAJE4BBB8mAAQABQEAC+Zf//+rSrMzD/AAUogBHAAA/7/v9EwygAAAABAoAa8J + 4CQEyH//J2Eew0j8AAADw21vb3YAAABsbXZoZAAAAAAAAAAAAAAAAAAAA+gAAADIAAEAAAEAAAAAAAAAAAAAAAABAAAAAAAAAAAA + AAAAAAAAAQAAAAAAAAAAAAAAAAAAQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAIAAALtdHJhawAAAFx0a2hkAAAAAwAA + AAAAAAAAAAAAAQAAAAAAAADIAAAAAAAAAAAAAAAAAAAAAAABAAAAAAAAAAAAAAAAAAAAAQAAAAAAAAAAAAAAAAAAQAAAAABAAAAA + QAAAAAAAJGVkdHMAAAAcZWxzdAAAAAAAAAABAAAAyAAAAAAAAQAAAAACZW1kaWEAAAAgbWRoZAAAAAAAAAAAAAAAAAAST4AAA6mA + VcQAAAAAAC1oZGxyAAAAAAAAAAB2aWRlAAAAAAAAAAAAAAAAVmlkZW9IYW5kbGVyAAAAAhBtaW5mAAAAFHZtaGQAAAABAAAAAAAA + AAAAAAAkZGluZgAAABxkcmVmAAAAAAAAAAEAAAAMdXJsIAAAAAEAAAHQc3RibAAAAWRzdHNkAAAAAAAAAAEAAAFUaHZjMQAAAAAA + AAABAAAAAAAAAAAAAAAAAAAAAABAAEAASAAAAEgAAAAAAAAAAQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABj//wAA + AMdodmNDAQQIAAAAnagAAAAAHvAA/P36+gAADwOgAAIAF0ABDAH//wQIAAADAJ2oAAADAAAeugJAABdAAQwB//8ECAAAAwCdqAAA + AwAAHroCQKEAAgArQgEBBAgAAAMAnagAAAMAAB6gIIEE2W6kkyvAWoSJBIIAAAMAAgAAAwAUEAArQgEBBAgAAAMAnagAAAMAAB6g + IIEE2W6kkyvAWoSJBIIAAAMAAgAAAwAUEKIAAgAHRAHBcrAiQAAIRAHBcrAiQAAAAAATY29scm5jbHgACQASAAkAAAAAEHBhc3AA + AAABAAAAAQAAABRidHJ0AAAAAAAAEnAAABJwAAAAGHN0dHMAAAAAAAAAAQAAAAIAAdTAAAAAHHN0c2MAAAAAAAAAAQAAAAEAAAAC + AAAAAQAAABxzdHN6AAAAAAAAAAAAAAACAAAAOgAAADwAAAAUc3RjbwAAAAAAAAABAAAALAAAAGJ1ZHRhAAAAWm1ldGEAAAAAAAAA + IWhkbHIAAAAAAAAAAG1kaXJhcHBsAAAAAAAAAAAAAAAALWlsc3QAAAAlqXRvbwAAAB1kYXRhAAAAAQAAAABMYXZmNjIuMTIuMTAx + """ +} diff --git a/Tests/AetherEngineTests/HEVCCodecStringTests.swift b/Tests/AetherEngineTests/HEVCCodecStringTests.swift index b24c70def..f106db63a 100644 --- a/Tests/AetherEngineTests/HEVCCodecStringTests.swift +++ b/Tests/AetherEngineTests/HEVCCodecStringTests.swift @@ -1,4 +1,7 @@ import Testing +import Foundation +import AetherLibavcodec +import AetherLibavutil @testable import AetherEngine /// AE#187 part 2: the plain-HEVC CODECS string. The `.none` / `.profile82` branch hardcoded @@ -113,4 +116,56 @@ struct HEVCCodecStringTests { func tooShortReturnsNil() { #expect(HLSVideoEngine.hevcCodecsString(fromConfigRecord: [1, 1, 0x60, 0, 0, 0]) == nil) } + + /// Audit HLS-3: 8-bit Main L3.1 as MPEG-TS carries it, a VPS and an SPS behind start codes with + /// the emulation-prevention bytes a real encoder writes into the zero runs of the PTL. + static let annexBMain8bit: [UInt8] = [ + 0x00, 0x00, 0x00, 0x01, 0x40, 0x01, 0x0c, 0x01, 0xff, 0xff, 0x01, 0x60, 0x00, 0x00, 0x03, 0x00, + 0x90, 0x00, 0x00, 0x03, 0x00, 0x00, 0x03, 0x00, 0x5d, 0x95, 0x94, 0x09, + 0x00, 0x00, 0x00, 0x01, 0x42, 0x01, 0x01, 0x01, 0x60, 0x00, 0x00, 0x03, 0x00, 0x90, 0x00, 0x00, + 0x03, 0x00, 0x00, 0x03, 0x00, 0x5d, 0xa0, 0x02, 0x80, 0x80, 0x2d, 0x16, 0x59, 0x59, 0xa4, 0x93, + 0x2b, 0x80, + ] + + @Test("Annex-B 8-bit Main SPS -> hvc1.1.6.L93.90, not the Main10 fallback") + func annexBMain8bitFromSPS() { + #expect(HLSVideoEngine.hevcCodecsString(fromAnnexBExtradata: Self.annexBMain8bit) + == "hvc1.1.6.L93.90") + } + + @Test("Annex-B Main10 SPS keeps its own profile and level") + func annexBMain10FromSPS() { + let annexB: [UInt8] = [ + 0x00, 0x00, 0x01, 0x42, 0x01, 0x01, 0x02, 0x20, 0x00, 0x00, 0x03, 0x00, 0x90, 0x00, 0x00, + 0x03, 0x00, 0x00, 0x03, 0x00, 0x78, 0xa0, 0x03, 0xc0, 0x80, 0x11, 0x07, 0xca, + ] + #expect(HLSVideoEngine.hevcCodecsString(fromAnnexBExtradata: annexB) == "hvc1.2.4.L120.90") + } + + @Test("Annex-B extradata without an SPS returns nil") + func annexBWithoutSPS() { + #expect(HLSVideoEngine.hevcCodecsString( + fromAnnexBExtradata: [0x00, 0x00, 0x01, 0x40, 0x01, 0x0c, 0x01, 0xff]) == nil) + } + + @Test("the plain HEVC route declares the Annex-B source's own profile in the master") + func plainRouteUsesAnnexBSPS() throws { + var codecpar: UnsafeMutablePointer? = try #require(avcodec_parameters_alloc()) + defer { avcodec_parameters_free(&codecpar) } + let cp = codecpar! + cp.pointee.codec_type = AVMEDIA_TYPE_VIDEO + cp.pointee.codec_id = AV_CODEC_ID_HEVC + cp.pointee.level = 93 + let bytes = Self.annexBMain8bit + let ed = try #require(av_mallocz(bytes.count + Int(AV_INPUT_BUFFER_PADDING_SIZE))) + .assumingMemoryBound(to: UInt8.self) + bytes.withUnsafeBufferPointer { ed.update(from: $0.baseAddress!, count: bytes.count) } + cp.pointee.extradata = ed + cp.pointee.extradata_size = Int32(bytes.count) + + let engine = HLSVideoEngine(url: URL(fileURLWithPath: "/nonexistent/main8.ts"), + dvModeAvailable: false) + let route = try engine.resolveCodecRoute(codecpar: UnsafePointer(cp)) + #expect(route.primaryCodecs == "hvc1.1.6.L93.90") + } } diff --git a/Tests/AetherEngineTests/HEVCCompositionOffsetRepairTests.swift b/Tests/AetherEngineTests/HEVCCompositionOffsetRepairTests.swift new file mode 100644 index 000000000..4f85989ee --- /dev/null +++ b/Tests/AetherEngineTests/HEVCCompositionOffsetRepairTests.swift @@ -0,0 +1,236 @@ +import Foundation +import AetherLibavcodec +import Testing +@testable import AetherEngine + +/// #699: the #409 defect on HEVC. The reporting asset (a CUVA HDR Vivid conformance stream, 4K50 Main +/// 10) carries B pictures and no `ctts`, so every sample reports `PTS == DTS` and the picture was +/// presented in decode order. Its mini-GOP is five pictures at a reorder delay of 2, which is also what +/// the twin below encodes: the next anchor is decoded four pictures ahead of the last one displayed. +@Suite("HEVC missing composition offsets (#699)") +struct HEVCCompositionOffsetRepairTests { + + /// The reporting asset's head, read with libavcodec's HEVC parser: picture order 0, 5, 3, 1, 2, 4, + /// then the same mini-GOP again, and the third anchor at 15 while 11 to 14 are still to come. + private func reportedSamples() -> [H264CompositionOffsetRepair.Sample] { + let pocs: [Int64] = [0, 5, 3, 1, 2, 4, 10, 8, 6, 7, 9, 15] + return pocs.enumerated().map { index, poc in + H264CompositionOffsetRepair.Sample( + dts: Int64(index - 2) * 24000, + pts: Int64(index - 2) * 24000, + pictureOrderCount: poc, + isKeyframe: index == 0 + ) + } + } + + @Test("a mini-GOP longer than the reorder delay is repaired, not refused at its ragged edge") + func longMiniGOPIsRepaired() { + let verdict = H264CompositionOffsetRepair.classify( + samples: reportedSamples(), videoDelay: 2, streamStartTime: 0, ladderStart: -48000) + #expect(verdict == .repair(.init(step: 24000, decodeLead: 48000, shift: 48000, pocStep: 1))) + } + + @Test("the contiguous prefix has to be at least the minimum sample long") + func contiguousPrefixNeedsTheMinimum() { + #expect(H264CompositionOffsetRepair.hasContiguousPrefix([0, 5, 3, 1, 2, 4, 10, 8, 6, 7, 9, 15])) + // Closes at 6 pictures only, and the next mini-GOP never closes inside the window. + #expect(!H264CompositionOffsetRepair.hasContiguousPrefix([0, 5, 3, 1, 2, 4, 12, 9, 7, 6, 8, 11])) + // A stray rank spread over twice the ladder never fills its range. + #expect(!H264CompositionOffsetRepair.hasContiguousPrefix([0, 10, 6, 2, 4, 8, 20, 16, 12, 14, 18, 30])) + } + + // MARK: - end to end, against the twin + + private struct Timestamps: Equatable, CustomStringConvertible { + var pts: Int64 + var dts: Int64 + var description: String { "pts=\(pts) dts=\(dts)" } + } + + private static func videoTimestamps(base64: String) throws -> [Timestamps] { + let data = try #require(Data(base64Encoded: base64, options: .ignoreUnknownCharacters)) + let demuxer = Demuxer() + try demuxer.open(reader: DataIOReader(data: data), formatHint: "mp4") + defer { demuxer.close() } + let videoIndex = demuxer.videoStreamIndex + var result: [Timestamps] = [] + while let packet = try? demuxer.readPacket() { + if packet.pointee.stream_index == videoIndex { + result.append(Timestamps(pts: packet.pointee.pts, dts: packet.pointee.dts)) + } + var owned: UnsafeMutablePointer? = packet + trackedPacketFree(&owned) + } + return result + } + + @Test("the repaired HEVC twin carries the healthy twin's timestamps, packet for packet") + func repairedTwinMatchesHealthyTwin() throws { + let healthy = try Self.videoTimestamps(base64: Self.healthyFixtureBase64) + let repaired = try Self.videoTimestamps(base64: Self.missingFixtureBase64) + #expect(healthy.count == 30) + #expect(healthy.first == Timestamps(pts: 0, dts: -512)) + #expect(repaired == healthy) + } + + @Test("the healthy HEVC twin is delivered exactly as the container wrote it") + func healthyTwinIsUntouched() throws { + let healthy = try Self.videoTimestamps(base64: Self.healthyFixtureBase64) + #expect(healthy.contains { $0.pts != $0.dts }) + #expect(zip(healthy, healthy.dropFirst()).allSatisfy { $1.dts - $0.dts == 256 }) + } + + /// 64x64 HEVC Main 10, 50 fps, 30 frames, four B pictures per hierarchical mini-GOP. + /// + /// ffmpeg -f lavfi -i 'testsrc=s=64x64:r=50:d=0.6' -frames:v 30 -c:v libx265 -preset ultrafast \ + /// -pix_fmt yuv420p10le -x265-params \ + /// 'bframes=4:b-pyramid=1:b-adapt=0:keyint=30:min-keyint=30:scenecut=0' \ + /// -tag:v hvc1 -movflags +faststart healthy.mp4 + /// ffmpeg -i healthy.mp4 -map 0:v:0 -c:v copy -bsf:v 'setts=pts=DTS' \ + /// -movflags +faststart missing.mp4 + private static let healthyFixtureBase64 = """ + AAAAHGZ0eXBpc29tAAACAGlzb21pc28ybXA0MQAADf5tb292AAAAbG12aGQAAAAAAAAAAAAAAAAAAAPoAAACWAABAAABAAAAAAAA + AAAAAAAAAQAAAAAAAAAAAAAAAAAAAAEAAAAAAAAAAAAAAAAAAEAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACAAANKHRy + YWsAAABcdGtoZAAAAAMAAAAAAAAAAAAAAAEAAAAAAAACWAAAAAAAAAAAAAAAAAAAAAAAAQAAAAAAAAAAAAAAAAAAAAEAAAAAAAAA + AAAAAAAAAEAAAAAAQAAAAEAAAAAAACRlZHRzAAAAHGVsc3QAAAAAAAAAAQAAAlgAAAIAAAEAAAAADKBtZGlhAAAAIG1kaGQAAAAA + AAAAAAAAAAAAADIAAAAeAFXEAAAAAAAtaGRscgAAAAAAAAAAdmlkZQAAAAAAAAAAAAAAAFZpZGVvSGFuZGxlcgAAAAxLbWluZgAA + ABR2bWhkAAAAAQAAAAAAAAAAAAAAJGRpbmYAAAAcZHJlZgAAAAAAAAABAAAADHVybCAAAAABAAAMC3N0YmwAAAoZc3RzZAAAAAAA + AAABAAAKCWh2YzEAAAAAAAAAAQAAAAAAAAAAAAAAAAAAAAAAQABAAEgAAABIAAAAAAAAAAEVTGF2YzYyLjI4LjEwMSBsaWJ4MjY1 + AAAAAAAAAAAAAAAY//8AAAmFaHZjQwECIAAAAJAAAAAAAB7wAPz9+voAAA8EoAABABhAAQwB//8CIAAAAwCQAAADAAADAB6VmAmh + AAEAKkIBAQIgAAADAJAAAAMAAAMAHqAggQTZZWZKTC8BaAgAAAMACAAAAwGQQKIAAQAGRAHAc8CJJwABCQpOAQX///////////8F + LKLeCbUXR9u7VaT+f8L8TngyNjUgKGJ1aWxkIDIxNikgLSA0LjIrMS1lNDQ0NzQ0OltNYWMgT1MgWF1bY2xhbmcgMjEuMC4wXVs2 + NCBiaXRdIDEwYml0IC0gSC4yNjUvSEVWQyBjb2RlYyAtIENvcHlyaWdodCAyMDEzLTIwMTggKGMpIE11bHRpY29yZXdhcmUsIElu + YyAtIGh0dHA6Ly94MjY1Lm9yZyAtIG9wdGlvbnM6IGNwdWlkPTM0IGZyYW1lLXRocmVhZHM9MSBuby13cHAgbm8tcG1vZGUgbm8t + cG1lIG5vLXBzbnIgbm8tc3NpbSBsb2ctbGV2ZWw9MCBiaXRkZXB0aD0xMCBpbnB1dC1jc3A9MSBmcHM9NTAvMSBpbnB1dC1yZXM9 + NjR4NjQgaW50ZXJsYWNlPTAgdG90YWwtZnJhbWVzPTAgbGV2ZWwtaWRjPTAgaGlnaC10aWVyPTEgdWhkLWJkPTAgcmVmPTEgbm8t + YWxsb3ctbm9uLWNvbmZvcm1hbmNlIG5vLXJlcGVhdC1oZWFkZXJzIGFubmV4YiBuby1hdWQgbm8tZW9iIG5vLWVvcyBuby1ocmQg + aW5mbyBoYXNoPTAgdGVtcG9yYWwtbGF5ZXJzPTAgb3Blbi1nb3AgbWluLWtleWludD0zMCBrZXlpbnQ9MzAgZ29wLWxvb2thaGVh + ZD0wIGJmcmFtZXM9NCBiLWFkYXB0PTAgYi1weXJhbWlkIGJmcmFtZS1iaWFzPTAgcmMtbG9va2FoZWFkPTUgbG9va2FoZWFkLXNs + aWNlcz0wIHNjZW5lY3V0PTAgbm8taGlzdC1zY2VuZWN1dCByYWRsPTAgbm8tc3BsaWNlIG5vLWludHJhLXJlZnJlc2ggY3R1PTMy + IG1pbi1jdS1zaXplPTE2IG5vLXJlY3Qgbm8tYW1wIG1heC10dS1zaXplPTMyIHR1LWludGVyLWRlcHRoPTEgdHUtaW50cmEtZGVw + dGg9MSBsaW1pdC10dT0wIHJkb3EtbGV2ZWw9MCBkeW5hbWljLXJkPTAuMDAgbm8tc3NpbS1yZCBuby1zaWduaGlkZSBuby10c2tp + cCBuci1pbnRyYT0wIG5yLWludGVyPTAgbm8tY29uc3RyYWluZWQtaW50cmEgc3Ryb25nLWludHJhLXNtb290aGluZyBtYXgtbWVy + Z2U9MiBsaW1pdC1yZWZzPTAgbm8tbGltaXQtbW9kZXMgbWU9MCBzdWJtZT0wIG1lcmFuZ2U9NTcgdGVtcG9yYWwtbXZwIG5vLWZy + YW1lLWR1cCBuby1obWUgbm8td2VpZ2h0cCBuby13ZWlnaHRiIG5vLWFuYWx5emUtc3JjLXBpY3MgZGVibG9jaz0wOjAgbm8tc2Fv + IG5vLXNhby1ub24tZGVibG9jayByZD0yIHNlbGVjdGl2ZS1zYW89MCBlYXJseS1za2lwIHJza2lwIGZhc3QtaW50cmEgbm8tdHNr + aXAtZmFzdCBuby1jdS1sb3NzbGVzcyBuby1iLWludHJhIG5vLXNwbGl0cmQtc2tpcCByZHBlbmFsdHk9MCBwc3ktcmQ9Mi4wMCBw + c3ktcmRvcT0wLjAwIG5vLXJkLXJlZmluZSBuby1sb3NzbGVzcyBjYnFwb2Zmcz0wIGNycXBvZmZzPTAgcmM9Y3JmIGNyZj0yOC4w + IHFjb21wPTAuNjAgcXBzdGVwPTQgc3RhdHMtd3JpdGU9MCBzdGF0cy1yZWFkPTAgaXByYXRpbz0xLjQwIHBicmF0aW89MS4zMCBh + cS1tb2RlPTEgYXEtc3RyZW5ndGg9MC4wMCBjdXRyZWUgem9uZS1jb3VudD0wIG5vLXN0cmljdC1jYnIgcWctc2l6ZT0zMiBuby1y + Yy1ncmFpbiBxcG1heD02OSBxcG1pbj0wIG5vLWNvbnN0LXZidiBzYXI9MSBvdmVyc2Nhbj0wIHZpZGVvZm9ybWF0PTUgcmFuZ2U9 + MCBjb2xvcnByaW09MiB0cmFuc2Zlcj0yIGNvbG9ybWF0cml4PTIgY2hyb21hbG9jPTAgZGlzcGxheS13aW5kb3c9MCBjbGw9MCww + IG1pbi1sdW1hPTAgbWF4LWx1bWE9MTAyMyBsb2cyLW1heC1wb2MtbHNiPTggdnVpLXRpbWluZy1pbmZvIHZ1aS1ocmQtaW5mbyBz + bGljZXM9MSBuby1vcHQtcXAtcHBzIG5vLW9wdC1yZWYtbGlzdC1sZW5ndGgtcHBzIG5vLW11bHRpLXBhc3Mtb3B0LXJwcyBzY2Vu + ZWN1dC1iaWFzPTAuMDUgbm8tb3B0LWN1LWRlbHRhLXFwIG5vLWFxLW1vdGlvbiBuby1oZHIxMCBuby1oZHIxMC1vcHQgbm8tZGhk + cjEwLW9wdCBuby1pZHItcmVjb3Zlcnktc2VpIGFuYWx5c2lzLXJldXNlLWxldmVsPTAgYW5hbHlzaXMtc2F2ZS1yZXVzZS1sZXZl + bD0wIGFuYWx5c2lzLWxvYWQtcmV1c2UtbGV2ZWw9MCBzY2FsZS1mYWN0b3I9MCByZWZpbmUtaW50cmE9MCByZWZpbmUtaW50ZXI9 + MCByZWZpbmUtbXY9MSByZWZpbmUtY3R1LWRpc3RvcnRpb249MCBuby1saW1pdC1zYW8gY3R1LWluZm89MCBuby1sb3dwYXNzLWRj + dCByZWZpbmUtYW5hbHlzaXMtdHlwZT0wIGNvcHktcGljPTEgbWF4LWF1c2l6ZS1mYWN0b3I9MS4wIG5vLWR5bmFtaWMtcmVmaW5l + IG5vLXNpbmdsZS1zZWkgbm8taGV2Yy1hcSBuby1zdnQgbm8tZmllbGQgcXAtYWRhcHRhdGlvbi1yYW5nZT0xLjAwIHNjZW5lY3V0 + LWF3YXJlLXFwPTBjb25mb3JtYW5jZS13aW5kb3ctb2Zmc2V0cyByaWdodD0wIGJvdHRvbT0wIGRlY29kZXItbWF4LXJhdGU9MCBu + by12YnYtbGl2ZS1tdWx0aS1wYXNzIG5vLW1jc3RmIG5vLXNicmMgbm8tZnJhbWUtcmOAAAAACmZpZWwBAAAAABBwYXNwAAAAAQAA + AAEAAAAUYnRydAAAAAAAAFZoAAAAAAAAABhzdHRzAAAAAAAAAAEAAAAeAAABAAAAABRzdHNzAAAAAAAAAAEAAAABAAAAKnNkdHAA + AAAAIBAQGBgYEBAYGBgQEBgYGBAQGBgYEBAYGBgQEBgYAAAA2GN0dHMAAAAAAAAAGQAAAAEAAAIAAAAAAQAABgAAAAABAAADAAAA + AAIAAAAAAAAAAQAAAQAAAAABAAAGAAAAAAEAAAMAAAAAAgAAAAAAAAABAAABAAAAAAEAAAYAAAAAAQAAAwAAAAACAAAAAAAAAAEA + AAEAAAAAAQAABgAAAAABAAADAAAAAAIAAAAAAAAAAQAAAQAAAAABAAAGAAAAAAEAAAMAAAAAAgAAAAAAAAABAAABAAAAAAEAAAUA + AAAAAQAAAgAAAAABAAAAAAAAAAEAAAEAAAAAHHN0c2MAAAAAAAAAAQAAAAEAAAAeAAAAAQAAAIxzdHN6AAAAAAAAAAAAAAAeAAAE + FQAAABUAAAAOAAAADwAAAA8AAAAOAAAALAAAAA8AAAAQAAAAEAAAAA8AAAAsAAAAEAAAABAAAAAQAAAADwAAADoAAAAQAAAAEAAA + ABAAAAAPAAAALgAAABAAAAAQAAAAEAAAAA8AAAAuAAAAEAAAAA8AAAAPAAAAFHN0Y28AAAAAAAAAAQAADioAAABidWR0YQAAAFpt + ZXRhAAAAAAAAACFoZGxyAAAAAAAAAABtZGlyYXBwbAAAAAAAAAAAAAAAAC1pbHN0AAAAJal0b28AAAAdZGF0YQAAAAEAAAAATGF2 + ZjYyLjEyLjEwMQAAAAhmcmVlAAAGg21kYXQAAAQRKAGsKYCoC2PeLG++3GQqYwPd5f4iYh0irT8tB7IqSXHt9aYS3pMd85e6qfFP + LmPrzOtW5kKQFnIBbc1MydWMNZHnyNpet0rgxNXjNkPiSCdRctOJmNboZXO3x//WaYP3Aq3x+xuQhmV0stHPZdcmt5Jbna/QezfT + DUUKoJwPS+5DmWJq9yw6gPeCyy1Rli+B4Uaxa0tsVKRIN5YnsFwCI3/P3LFhWQ1aulZViQoT/4lpsoXQXXUKsqQ1G/jmT98sRqYV + 9OoAjm8GpBUioTmj3bkTXiDhlAX6da2TWpoxFZOqcNETPTflTcnZRJjTcqAyvaoYMPHFMywXDq05X6Klw+5LUC0m/IRo1LpwAEX6 + dQqd6AMic6AKZUI9Znm7vmJQUJlPMTBWdxUiVdl5FhUt+Aukxa3ZF8pAQW5ql0lJa7DCNvwtqZKJviO6bfAN+tV0zaPRskCedm8M + 7ojLfJp4/boaEq///9trlDjx6oCNe2I8TiPY/CgD6qXDF/1Ns/Pjv2jYMkPN+buRIG1PElW37KjK5VLSI8fBS/IwNmfWzMlJIfMP + oQm3zMBouaAJCeUOYHTE9xl2KRPwOcs6TCcSW0VCQhefFSlomBwBIgI3UnQQz1HyVVmMr6tMSVFPe2zKSCztBKkO3sx/RLk88SA0 + dTCYwwefoazaTXOldBEPrQO1yAxZ+iqmsAslyNmichJ397ob6lWVj4HKIoH5dM3hA8rrhOqTQ5tqehE2pajPeVnrYlxmlH81gtpi + 9MT6bA16aeJ9eY2gdP3A+QLrPLMTgHsgqa3k6rkOOpyrF15SswW9JNzerZOCRpfqw5AXp6y1aKyNhsYUJsDfwWkRRGgWlC5zzB1j + y5gOFuMIIM07Ew6eKOwU2teliFSzdk1MDHXcAwI+F3vaFsNEYJmk7zRnh873ftuThZQx28rwlmuf4+Ab8qv6amuMd/sC8/zxhfF0 + UbCQSENNgeDzCZVTj8B8T0BZUoKHPn//KdmxoyikLbd2e3sEJyR/0HOhkOZnEKzLIqflcKN10SX5bgj9Q7at1ZCOMhQ5AzS+F+iL + b4qsLQJQeIApndqD3H7Llw14c3T0vZL1cAAsiSs5yFCuCwHYC8T8A7WdBrQM7As8BpBZgSAlXnJN7TxLfvXv3wwHVKsXaNTzxIiV + 4s0Xtyhve/TUT6JuZtAZMJ3cWxXsJluym+z1c6poBR/aMnWSyhTuqOLa/5vU443t7Y4MwVpe6gBypmgX9uCJmCBkme8et61FbpkZ + +Mgc4AQJVenBce5b8yAQWHWd5zBk4zRu2L0VK9vuSu7ZgHYo51UxzwMQCNJQ7a92ZThKFLyDmFT5gqeltYfIjka3uHJDxujXGscw + FPTQRj4LKfyFRuPy3xCSsS8YMYm+AAAAEQIB0ClLiBRAs1pEo6u+ZSuXAAAACgIB4GSdYIFkkngAAAALAAHgJPVeiQMYkPAAAAAL + AAHgRNdeiQMIkPAAAAAKAAHghrfggYyQ8AAAACgCAdBQktXiBRCzWkFey1+VMBHdPUlsBUhuilioNQ9hOPghYTzWH99wAAAACwIB + 4QInV1ggWZJ4AAAADAAB4Mb1VeiQMYCQ8AAAAAwAAeDm1XXokDCAkPAAAAALAAHhIi3XggYwkPAAAAAoAgHQeLLV1iBRs1fdRSLN + r7qLgw9k6qz4+0LjUk+0NRRyL70QOwEQgAAAAAwCAeGiJ1LWCBZAkngAAAAMAAHhZvVV6JAxgJDwAAAADAAB4YbVdeiQMICQ8AAA + AAsAAeHCLdeCBjCQ8AAAADYCAdCgstXWIFGzWkXwn0I9zhHfu+OUlQywM4hhFO/Rgmzzdyv0TypArT007GQJ1LjPL7Vu2TAAAAAM + AgHiQidS1ggWwJJ4AAAADAAB4gb1VeiQMICQ8AAAAAwAAeIm1XXokDGAkPAAAAALAAHiYi3XggYwkPAAAAAqAgHQyLLV1iBTs1pF + 2UwP6z4eiZizCiXuyD+VkAZ8PmIohoKUP1a5rFLYAAAADAIB4uInUtYIFkCSeAAAAAwAAeKm9VXokDCAkPAAAAAMAAHixtV16JAx + gJDwAAAACwAB4wIt14IGMJDwAAAAKgIB0OiyVdYgUbNX33n2/kixaxBTSBJOYLe+rSsDXZK4kET9EzPm7DExEgAAAAwCAeNiJVLW + CBbAkngAAAALAAHjRvXXokDGkPAAAAALAAHjgi1XggYwkPA= + """ + + private static let missingFixtureBase64 = """ + AAAAHGZ0eXBpc29tAAACAGlzb21pc28ybXA0MQAADSZtb292AAAAbG12aGQAAAAAAAAAAAAAAAAAAAPoAAACMAABAAABAAAAAAAA + AAAAAAAAAQAAAAAAAAAAAAAAAAAAAAEAAAAAAAAAAAAAAAAAAEAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACAAAMUHRy + YWsAAABcdGtoZAAAAAMAAAAAAAAAAAAAAAEAAAAAAAACMAAAAAAAAAAAAAAAAAAAAAAAAQAAAAAAAAAAAAAAAAAAAAEAAAAAAAAA + AAAAAAAAAEAAAAAAQAAAAEAAAAAAACRlZHRzAAAAHGVsc3QAAAAAAAAAAQAAAjAAAAIAAAEAAAAAC8htZGlhAAAAIG1kaGQAAAAA + AAAAAAAAAAAAADIAAAAeAFXEAAAAAAAtaGRscgAAAAAAAAAAdmlkZQAAAAAAAAAAAAAAAFZpZGVvSGFuZGxlcgAAAAtzbWluZgAA + ABR2bWhkAAAAAQAAAAAAAAAAAAAAJGRpbmYAAAAcZHJlZgAAAAAAAAABAAAADHVybCAAAAABAAALM3N0YmwAAAoZc3RzZAAAAAAA + AAABAAAKCWh2YzEAAAAAAAAAAQAAAAAAAAAAAAAAAAAAAAAAQABAAEgAAABIAAAAAAAAAAEVTGF2YzYyLjI4LjEwMSBsaWJ4MjY1 + AAAAAAAAAAAAAAAY//8AAAmFaHZjQwECIAAAAJAAAAAAAB7wAPz9+voAAA8EoAABABhAAQwB//8CIAAAAwCQAAADAAADAB6VmAmh + AAEAKkIBAQIgAAADAJAAAAMAAAMAHqAggQTZZWZKTC8BaAgAAAMACAAAAwGQQKIAAQAGRAHAc8CJJwABCQpOAQX///////////8F + LKLeCbUXR9u7VaT+f8L8TngyNjUgKGJ1aWxkIDIxNikgLSA0LjIrMS1lNDQ0NzQ0OltNYWMgT1MgWF1bY2xhbmcgMjEuMC4wXVs2 + NCBiaXRdIDEwYml0IC0gSC4yNjUvSEVWQyBjb2RlYyAtIENvcHlyaWdodCAyMDEzLTIwMTggKGMpIE11bHRpY29yZXdhcmUsIElu + YyAtIGh0dHA6Ly94MjY1Lm9yZyAtIG9wdGlvbnM6IGNwdWlkPTM0IGZyYW1lLXRocmVhZHM9MSBuby13cHAgbm8tcG1vZGUgbm8t + cG1lIG5vLXBzbnIgbm8tc3NpbSBsb2ctbGV2ZWw9MCBiaXRkZXB0aD0xMCBpbnB1dC1jc3A9MSBmcHM9NTAvMSBpbnB1dC1yZXM9 + NjR4NjQgaW50ZXJsYWNlPTAgdG90YWwtZnJhbWVzPTAgbGV2ZWwtaWRjPTAgaGlnaC10aWVyPTEgdWhkLWJkPTAgcmVmPTEgbm8t + YWxsb3ctbm9uLWNvbmZvcm1hbmNlIG5vLXJlcGVhdC1oZWFkZXJzIGFubmV4YiBuby1hdWQgbm8tZW9iIG5vLWVvcyBuby1ocmQg + aW5mbyBoYXNoPTAgdGVtcG9yYWwtbGF5ZXJzPTAgb3Blbi1nb3AgbWluLWtleWludD0zMCBrZXlpbnQ9MzAgZ29wLWxvb2thaGVh + ZD0wIGJmcmFtZXM9NCBiLWFkYXB0PTAgYi1weXJhbWlkIGJmcmFtZS1iaWFzPTAgcmMtbG9va2FoZWFkPTUgbG9va2FoZWFkLXNs + aWNlcz0wIHNjZW5lY3V0PTAgbm8taGlzdC1zY2VuZWN1dCByYWRsPTAgbm8tc3BsaWNlIG5vLWludHJhLXJlZnJlc2ggY3R1PTMy + IG1pbi1jdS1zaXplPTE2IG5vLXJlY3Qgbm8tYW1wIG1heC10dS1zaXplPTMyIHR1LWludGVyLWRlcHRoPTEgdHUtaW50cmEtZGVw + dGg9MSBsaW1pdC10dT0wIHJkb3EtbGV2ZWw9MCBkeW5hbWljLXJkPTAuMDAgbm8tc3NpbS1yZCBuby1zaWduaGlkZSBuby10c2tp + cCBuci1pbnRyYT0wIG5yLWludGVyPTAgbm8tY29uc3RyYWluZWQtaW50cmEgc3Ryb25nLWludHJhLXNtb290aGluZyBtYXgtbWVy + Z2U9MiBsaW1pdC1yZWZzPTAgbm8tbGltaXQtbW9kZXMgbWU9MCBzdWJtZT0wIG1lcmFuZ2U9NTcgdGVtcG9yYWwtbXZwIG5vLWZy + YW1lLWR1cCBuby1obWUgbm8td2VpZ2h0cCBuby13ZWlnaHRiIG5vLWFuYWx5emUtc3JjLXBpY3MgZGVibG9jaz0wOjAgbm8tc2Fv + IG5vLXNhby1ub24tZGVibG9jayByZD0yIHNlbGVjdGl2ZS1zYW89MCBlYXJseS1za2lwIHJza2lwIGZhc3QtaW50cmEgbm8tdHNr + aXAtZmFzdCBuby1jdS1sb3NzbGVzcyBuby1iLWludHJhIG5vLXNwbGl0cmQtc2tpcCByZHBlbmFsdHk9MCBwc3ktcmQ9Mi4wMCBw + c3ktcmRvcT0wLjAwIG5vLXJkLXJlZmluZSBuby1sb3NzbGVzcyBjYnFwb2Zmcz0wIGNycXBvZmZzPTAgcmM9Y3JmIGNyZj0yOC4w + IHFjb21wPTAuNjAgcXBzdGVwPTQgc3RhdHMtd3JpdGU9MCBzdGF0cy1yZWFkPTAgaXByYXRpbz0xLjQwIHBicmF0aW89MS4zMCBh + cS1tb2RlPTEgYXEtc3RyZW5ndGg9MC4wMCBjdXRyZWUgem9uZS1jb3VudD0wIG5vLXN0cmljdC1jYnIgcWctc2l6ZT0zMiBuby1y + Yy1ncmFpbiBxcG1heD02OSBxcG1pbj0wIG5vLWNvbnN0LXZidiBzYXI9MSBvdmVyc2Nhbj0wIHZpZGVvZm9ybWF0PTUgcmFuZ2U9 + MCBjb2xvcnByaW09MiB0cmFuc2Zlcj0yIGNvbG9ybWF0cml4PTIgY2hyb21hbG9jPTAgZGlzcGxheS13aW5kb3c9MCBjbGw9MCww + IG1pbi1sdW1hPTAgbWF4LWx1bWE9MTAyMyBsb2cyLW1heC1wb2MtbHNiPTggdnVpLXRpbWluZy1pbmZvIHZ1aS1ocmQtaW5mbyBz + bGljZXM9MSBuby1vcHQtcXAtcHBzIG5vLW9wdC1yZWYtbGlzdC1sZW5ndGgtcHBzIG5vLW11bHRpLXBhc3Mtb3B0LXJwcyBzY2Vu + ZWN1dC1iaWFzPTAuMDUgbm8tb3B0LWN1LWRlbHRhLXFwIG5vLWFxLW1vdGlvbiBuby1oZHIxMCBuby1oZHIxMC1vcHQgbm8tZGhk + cjEwLW9wdCBuby1pZHItcmVjb3Zlcnktc2VpIGFuYWx5c2lzLXJldXNlLWxldmVsPTAgYW5hbHlzaXMtc2F2ZS1yZXVzZS1sZXZl + bD0wIGFuYWx5c2lzLWxvYWQtcmV1c2UtbGV2ZWw9MCBzY2FsZS1mYWN0b3I9MCByZWZpbmUtaW50cmE9MCByZWZpbmUtaW50ZXI9 + MCByZWZpbmUtbXY9MSByZWZpbmUtY3R1LWRpc3RvcnRpb249MCBuby1saW1pdC1zYW8gY3R1LWluZm89MCBuby1sb3dwYXNzLWRj + dCByZWZpbmUtYW5hbHlzaXMtdHlwZT0wIGNvcHktcGljPTEgbWF4LWF1c2l6ZS1mYWN0b3I9MS4wIG5vLWR5bmFtaWMtcmVmaW5l + IG5vLXNpbmdsZS1zZWkgbm8taGV2Yy1hcSBuby1zdnQgbm8tZmllbGQgcXAtYWRhcHRhdGlvbi1yYW5nZT0xLjAwIHNjZW5lY3V0 + LWF3YXJlLXFwPTBjb25mb3JtYW5jZS13aW5kb3ctb2Zmc2V0cyByaWdodD0wIGJvdHRvbT0wIGRlY29kZXItbWF4LXJhdGU9MCBu + by12YnYtbGl2ZS1tdWx0aS1wYXNzIG5vLW1jc3RmIG5vLXNicmMgbm8tZnJhbWUtcmOAAAAACmZpZWwBAAAAABBwYXNwAAAAAQAA + AAEAAAAUYnRydAAAAAAAAFZoAABWaAAAABhzdHRzAAAAAAAAAAEAAAAeAAABAAAAABRzdHNzAAAAAAAAAAEAAAABAAAAKnNkdHAA + AAAAIBAQGBgYEBAYGBgQEBgYGBAQGBgYEBAYGBgQEBgYAAAAHHN0c2MAAAAAAAAAAQAAAAEAAAAeAAAAAQAAAIxzdHN6AAAAAAAA + AAAAAAAeAAAEFQAAABUAAAAOAAAADwAAAA8AAAAOAAAALAAAAA8AAAAQAAAAEAAAAA8AAAAsAAAAEAAAABAAAAAQAAAADwAAADoA + AAAQAAAAEAAAABAAAAAPAAAALgAAABAAAAAQAAAAEAAAAA8AAAAuAAAAEAAAAA8AAAAPAAAAFHN0Y28AAAAAAAAAAQAADVIAAABi + dWR0YQAAAFptZXRhAAAAAAAAACFoZGxyAAAAAAAAAABtZGlyYXBwbAAAAAAAAAAAAAAAAC1pbHN0AAAAJal0b28AAAAdZGF0YQAA + AAEAAAAATGF2ZjYyLjEyLjEwMQAAAAhmcmVlAAAGg21kYXQAAAQRKAGsKYCoC2PeLG++3GQqYwPd5f4iYh0irT8tB7IqSXHt9aYS + 3pMd85e6qfFPLmPrzOtW5kKQFnIBbc1MydWMNZHnyNpet0rgxNXjNkPiSCdRctOJmNboZXO3x//WaYP3Aq3x+xuQhmV0stHPZdcm + t5Jbna/QezfTDUUKoJwPS+5DmWJq9yw6gPeCyy1Rli+B4Uaxa0tsVKRIN5YnsFwCI3/P3LFhWQ1aulZViQoT/4lpsoXQXXUKsqQ1 + G/jmT98sRqYV9OoAjm8GpBUioTmj3bkTXiDhlAX6da2TWpoxFZOqcNETPTflTcnZRJjTcqAyvaoYMPHFMywXDq05X6Klw+5LUC0m + /IRo1LpwAEX6dQqd6AMic6AKZUI9Znm7vmJQUJlPMTBWdxUiVdl5FhUt+Aukxa3ZF8pAQW5ql0lJa7DCNvwtqZKJviO6bfAN+tV0 + zaPRskCedm8M7ojLfJp4/boaEq///9trlDjx6oCNe2I8TiPY/CgD6qXDF/1Ns/Pjv2jYMkPN+buRIG1PElW37KjK5VLSI8fBS/Iw + NmfWzMlJIfMPoQm3zMBouaAJCeUOYHTE9xl2KRPwOcs6TCcSW0VCQhefFSlomBwBIgI3UnQQz1HyVVmMr6tMSVFPe2zKSCztBKkO + 3sx/RLk88SA0dTCYwwefoazaTXOldBEPrQO1yAxZ+iqmsAslyNmichJ397ob6lWVj4HKIoH5dM3hA8rrhOqTQ5tqehE2pajPeVnr + YlxmlH81gtpi9MT6bA16aeJ9eY2gdP3A+QLrPLMTgHsgqa3k6rkOOpyrF15SswW9JNzerZOCRpfqw5AXp6y1aKyNhsYUJsDfwWkR + RGgWlC5zzB1jy5gOFuMIIM07Ew6eKOwU2teliFSzdk1MDHXcAwI+F3vaFsNEYJmk7zRnh873ftuThZQx28rwlmuf4+Ab8qv6amuM + d/sC8/zxhfF0UbCQSENNgeDzCZVTj8B8T0BZUoKHPn//KdmxoyikLbd2e3sEJyR/0HOhkOZnEKzLIqflcKN10SX5bgj9Q7at1ZCO + MhQ5AzS+F+iLb4qsLQJQeIApndqD3H7Llw14c3T0vZL1cAAsiSs5yFCuCwHYC8T8A7WdBrQM7As8BpBZgSAlXnJN7TxLfvXv3wwH + VKsXaNTzxIiV4s0Xtyhve/TUT6JuZtAZMJ3cWxXsJluym+z1c6poBR/aMnWSyhTuqOLa/5vU443t7Y4MwVpe6gBypmgX9uCJmCBk + me8et61FbpkZ+Mgc4AQJVenBce5b8yAQWHWd5zBk4zRu2L0VK9vuSu7ZgHYo51UxzwMQCNJQ7a92ZThKFLyDmFT5gqeltYfIjka3 + uHJDxujXGscwFPTQRj4LKfyFRuPy3xCSsS8YMYm+AAAAEQIB0ClLiBRAs1pEo6u+ZSuXAAAACgIB4GSdYIFkkngAAAALAAHgJPVe + iQMYkPAAAAALAAHgRNdeiQMIkPAAAAAKAAHghrfggYyQ8AAAACgCAdBQktXiBRCzWkFey1+VMBHdPUlsBUhuilioNQ9hOPghYTzW + H99wAAAACwIB4QInV1ggWZJ4AAAADAAB4Mb1VeiQMYCQ8AAAAAwAAeDm1XXokDCAkPAAAAALAAHhIi3XggYwkPAAAAAoAgHQeLLV + 1iBRs1fdRSLNr7qLgw9k6qz4+0LjUk+0NRRyL70QOwEQgAAAAAwCAeGiJ1LWCBZAkngAAAAMAAHhZvVV6JAxgJDwAAAADAAB4YbV + deiQMICQ8AAAAAsAAeHCLdeCBjCQ8AAAADYCAdCgstXWIFGzWkXwn0I9zhHfu+OUlQywM4hhFO/Rgmzzdyv0TypArT007GQJ1LjP + L7Vu2TAAAAAMAgHiQidS1ggWwJJ4AAAADAAB4gb1VeiQMICQ8AAAAAwAAeIm1XXokDGAkPAAAAALAAHiYi3XggYwkPAAAAAqAgHQ + yLLV1iBTs1pF2UwP6z4eiZizCiXuyD+VkAZ8PmIohoKUP1a5rFLYAAAADAIB4uInUtYIFkCSeAAAAAwAAeKm9VXokDCAkPAAAAAM + AAHixtV16JAxgJDwAAAACwAB4wIt14IGMJDwAAAAKgIB0OiyVdYgUbNX33n2/kixaxBTSBJOYLe+rSsDXZK4kET9EzPm7DExEgAA + AAwCAeNiJVLWCBbAkngAAAALAAHjRvXXokDGkPAAAAALAAHjgi1XggYwkPA= + """ +} diff --git a/Tests/AetherEngineTests/HLSDurationCeilingTests.swift b/Tests/AetherEngineTests/HLSDurationCeilingTests.swift new file mode 100644 index 000000000..e4537efac --- /dev/null +++ b/Tests/AetherEngineTests/HLSDurationCeilingTests.swift @@ -0,0 +1,92 @@ +import Testing +@testable import AetherEngine + +/// Audit NET-101, FEA-102, HLS-102: `Double(_:)` accepts `inf`, `nan` and any magnitude, and the playlist +/// parser stored the result unchecked. The live join log traps on `Int(inf)`, the live subtitle poll on +/// `UInt64(inf * 1e9)`, and the VOD ingest's EXTINF sum reaches the segment-plan builders unbounded. +@Suite("HLS durations stay inside the one-week program ceiling") +struct HLSDurationCeilingTests { + + private func media(targetDuration: String = "6", extinf: String) -> String { + """ + #EXTM3U + #EXT-X-TARGETDURATION:\(targetDuration) + #EXTINF:\(extinf), + seg0.ts + #EXT-X-ENDLIST + """ + } + + private func isPlaylistInvalid(_ text: String) -> Bool { + do { + _ = try HLSPlaylistParser.parse(text) + return false + } catch HLSIngestError.playlistInvalid { + return true + } catch { + return false + } + } + + @Test("A non-finite or absurd EXTINF is refused", arguments: [ + "inf", "infinity", "nan", "1e300", "604801", "200000000000000", + ]) + func rejectsHostileExtinf(value: String) { + #expect(isPlaylistInvalid(media(extinf: value))) + } + + @Test("A non-finite or absurd TARGETDURATION is refused", arguments: ["inf", "nan", "1e300", "20000000000"]) + func rejectsHostileTargetDuration(value: String) { + #expect(isPlaylistInvalid(media(targetDuration: value, extinf: "6"))) + } + + @Test("A whole program in one EXTINF, up to a week, still parses") + func weekLongEntryParses() throws { + guard case .media(let playlist) = try HLSPlaylistParser.parse(media(extinf: "604800")) else { + Issue.record("expected a media playlist") + return + } + #expect(playlist.segments.first?.duration == 604_800) + } + + @Test("The subtitle proxy's program ceiling is the parser's") + func proxySharesTheCeiling() { + #expect(RemoteHLSSubtitleProxy.maxProgramDurationSeconds == MediaDurationCeiling.seconds) + #expect(MediaDurationCeiling.seconds == 604_800) + } + + private func segments(_ durations: [Double]) -> [HLSMediaSegment] { + durations.enumerated().map { HLSMediaSegment(uri: "s\($0.offset).ts", duration: $0.element, + discontinuityBefore: false) } + } + + @Test("A VOD playlist summing past a week is refused, however small its entries") + func vodTotalCeiling() { + // Each entry passes the parser's per-entry ceiling; the sum does not. + #expect(throws: HLSIngestError.self) { + _ = try HLSVODIngestReader.segmentTimeline(segments([604_800, 604_800])) + } + // The Double-sum vanish case: a tiny entry after a large sum does not move the total, so the + // starts stop being monotonic and the plan falls back to a uniform grid over the whole sum. + #expect(throws: HLSIngestError.self) { + _ = try HLSVODIngestReader.segmentTimeline(segments(Array(repeating: 604_800, count: 1000) + [1e-9])) + } + } + + @Test("An ordinary VOD playlist keeps its starts and total") + func vodTimelineUnchanged() throws { + let timeline = try HLSVODIngestReader.segmentTimeline(segments([6, 6, 4.5])) + #expect(timeline.starts == [0, 6, 12]) + #expect(timeline.duration == 16.5) + #expect(throws: HLSIngestError.self) { + _ = try HLSVODIngestReader.segmentTimeline(segments([6, 0, 6])) + } + } + + @Test("The live subtitle poll interval stays between one and thirty seconds", arguments: [ + (6.0, 6.0), (0.2, 1.0), (604_800.0, 30.0), (Double.infinity, 30.0), (Double.nan, 1.0), (-5.0, 1.0), + ]) + func liveSubtitlePollInterval(targetDuration: Double, expected: Double) { + #expect(AetherEngine.liveSubtitlePollInterval(targetDuration: targetDuration) == expected) + } +} diff --git a/Tests/AetherEngineTests/HLSLocalServerSessionTokenTests.swift b/Tests/AetherEngineTests/HLSLocalServerSessionTokenTests.swift index 21234fb16..e01c39837 100644 --- a/Tests/AetherEngineTests/HLSLocalServerSessionTokenTests.swift +++ b/Tests/AetherEngineTests/HLSLocalServerSessionTokenTests.swift @@ -69,10 +69,105 @@ struct HLSLocalServerSessionTokenTests { #expect(Self.status(port: server.port, path: "/seg0.mp4") == 404) } + // MARK: - The token in the log (audit SUB-107) + + @Test("A running server's token is redacted from every log line, and released on stop") + func tokenIsRedactedWhileTheServerRuns() throws { + let server = HLSLocalServer(provider: StubProvider()) + try server.start() + let token = server.pathToken + let line = "[NativeAVPlayerHost] #2 load url=http://127.0.0.1:\(server.port)/\(token)/master.m3u8" + #expect(LogRedaction.isRegistered(token)) + #expect(!LogRedaction.redact(line).contains(token)) + #expect(LogRedaction.redact("[HLSLocalServer] GET /\(token)/seg_1.m4s HTTP/1.1") + == "[HLSLocalServer] GET //seg_1.m4s HTTP/1.1") + + server.stop() + #expect(!LogRedaction.isRegistered(token)) + server.stop() + #expect(!LogRedaction.isRegistered(token)) + } + + @Test("Stopping one server leaves the other's token redacted") + func twoServersKeepTheirOwnRegistration() throws { + let first = HLSLocalServer(provider: StubProvider()) + let second = HLSLocalServer(provider: StubProvider()) + try first.start() + try second.start() + defer { second.stop() } + + first.stop() + #expect(!LogRedaction.isRegistered(first.pathToken)) + #expect(LogRedaction.isRegistered(second.pathToken)) + } + + @Test("The logged request line names the route, not the token") + func requestLineOmitsTheToken() { + #expect(HLSLocalServer.requestLineForLog( + method: "GET", routePath: "/seg_1.m4s", query: "", version: "HTTP/1.1") + == "GET /seg_1.m4s HTTP/1.1") + #expect(HLSLocalServer.requestLineForLog( + method: "GET", routePath: "/media.m3u8", query: "_HLS_msn=12", version: "HTTP/1.1") + == "GET /media.m3u8?_HLS_msn=12 HTTP/1.1") + } + + // MARK: - Credential headers in the log (audit Vcred-102) + + /// On the #316 / AE#495 stand-in route AVPlayer carries the host's own headers to this server, so + /// the once-per-session header dump printed whatever credential the host passed. + @Test("The first-request header dump names a credential header but never prints its value") + func headerDumpOmitsCredentialValues() throws { + let tap = EngineLogCapture() + defer { tap.end() } + let server = HLSLocalServer(provider: StubProvider()) + try server.start() + defer { server.stop() } + let marker = "X-Probe-\(UUID().uuidString.prefix(8))" + + let status = Self.status( + port: server.port, path: "/\(server.pathToken)/media.m3u8", + extraHeaders: [#"Authorization: Digest username="bob", response="6629fae49393a05397450978507c4ef1""#, + "X-Portal-Auth: SECRETportal123", "\(marker): 1", "Range: bytes=0-1"]) + #expect(status == 200) + + // Header names no redactor rule knows, which is the point: the dump must not depend on one. + let dumped = tap.lines.filter { $0.contains("first request headers") && $0.contains(marker) } + #expect(dumped.count == 1) + for line in dumped { + #expect(!line.contains("6629fae49393a05397450978507c4ef1"), "\(line)") + #expect(!line.contains("SECRETportal123"), "\(line)") + #expect(line.contains("Authorization")) + #expect(line.contains("X-Portal-Auth")) + #expect(line.contains("Range: bytes=0-1")) + } + } + + @Test("Only the capability headers keep their values in the dump") + func headerDumpFormat() { + let dumped = HLSLocalServer.requestHeadersForLog([ + "Host: 127.0.0.1:50123", "Authorization: Bearer abcdefghijklmnopqrstuvwxyz0123", + "Cookie: session=SECRETsess123", "Range: bytes=0-1", "X-Playback-Session-Id: 5A0C", + "Accept: */*", "User-Agent: AppleCoreMedia/1.0", "garbage", + ]) + #expect(dumped == "Host: 127.0.0.1:50123 | Authorization | Cookie | Range: bytes=0-1 | " + + "X-Playback-Session-Id: 5A0C | Accept: */* | User-Agent: AppleCoreMedia/1.0 | ?") + } + // MARK: - Helpers - /// Status line of a plain GET, or 0 when the request could not be completed. - private static func status(port: UInt16, path: String) -> Int { + /// Status line of a plain GET, or 0 when the request could not be completed. A 0 is never the + /// answer under test, and a loaded CI runner has dropped one exchange mid-suite while the + /// requests around it on the same server were answered, so it is retried before it counts. + private static func status(port: UInt16, path: String, extraHeaders: [String] = []) -> Int { + for attempt in 0..<3 { + if attempt > 0 { usleep(100_000) } + let code = singleStatus(port: port, path: path, extraHeaders: extraHeaders) + if code != 0 { return code } + } + return 0 + } + + private static func singleStatus(port: UInt16, path: String, extraHeaders: [String]) -> Int { let fd = socket(AF_INET, SOCK_STREAM, IPPROTO_TCP) guard fd >= 0 else { return 0 } defer { close(fd) } @@ -87,7 +182,8 @@ struct HLSLocalServerSessionTokenTests { } } guard connected == 0 else { return 0 } - let request = "GET \(path) HTTP/1.1\r\nHost: 127.0.0.1\r\nConnection: close\r\n\r\n" + let extra = extraHeaders.map { "\($0)\r\n" }.joined() + let request = "GET \(path) HTTP/1.1\r\nHost: 127.0.0.1\r\n\(extra)Connection: close\r\n\r\n" let sent = Array(request.utf8).withUnsafeBytes { send(fd, $0.baseAddress, $0.count, 0) } guard sent > 0 else { return 0 } var buffer = [UInt8](repeating: 0, count: 256) diff --git a/Tests/AetherEngineTests/HLSLocalServerSlowPeerTests.swift b/Tests/AetherEngineTests/HLSLocalServerSlowPeerTests.swift new file mode 100644 index 000000000..06cdf8746 --- /dev/null +++ b/Tests/AetherEngineTests/HLSLocalServerSlowPeerTests.swift @@ -0,0 +1,248 @@ +import Foundation +import Testing +@testable import AetherEngine + +/// The listener answers the whole LAN while AirPlay needs it (#86), and the session token is read only +/// after a complete request head. These pin what keeps a peer that never presents one from holding the +/// slots the local player needs (audit NET-6), plus the two accept-loop diagnostics (NET-12, NET-13). +@Suite("HLS local server against slow and silent peers", .serialized, .timeLimit(.minutes(2))) +struct HLSLocalServerSlowPeerTests { + + @Test("A connection that never sends a request head is dropped at the head deadline") + func idleStrangerIsDropped() async throws { + let server = HLSLocalServer(provider: StubProvider(), unauthenticatedHeadSeconds: 0.5) + try server.start() + defer { server.stop() } + + let fd = try #require(Self.connect(port: server.port, host: "127.0.0.1")) + defer { close(fd) } + + // Observed from the client side: the server may drop the connection before a poll of its + // count ever sees it, which on a loaded run left a wait for "count == 1" parked forever. + // The old per-recv timeout was 60 s, so closing well inside that is the fix. + #expect(await Self.onOwnThread { Self.peerCloses(fd: fd, within: 20) }, + "an idle unauthenticated connection kept its slot") + let released = try await waitFor(upTo: .seconds(20)) { server.activeConnectionCount == 0 } + #expect(released) + } + + @Test("A peer trickling its head one byte at a time is dropped at the head deadline") + func tricklingStrangerIsDropped() async throws { + let server = HLSLocalServer(provider: StubProvider(), unauthenticatedHeadSeconds: 1) + try server.start() + defer { server.stop() } + + let fd = try #require(Self.connect(port: server.port, host: "127.0.0.1")) + let trickle = Trickle(fd: fd) + defer { trickle.stop(); close(fd) } + + #expect(await Self.onOwnThread { Self.peerCloses(fd: fd, within: 20) }, + "a byte every 200 ms kept an unauthenticated connection open") + let released = try await waitFor(upTo: .seconds(20)) { server.activeConnectionCount == 0 } + #expect(released) + } + + @Test("A connection that presented the token keeps its keep-alive idle past the stranger deadline") + func authenticatedKeepAliveSurvives() async throws { + // The stranger deadline runs from accept, so it must leave room for a worker thread that a + // loaded runner schedules late, or the first request is dropped before it is read. + let server = HLSLocalServer(provider: StubProvider(), unauthenticatedHeadSeconds: 2) + try server.start() + defer { server.stop() } + + let fd = try #require(Self.connect(port: server.port, host: "127.0.0.1")) + defer { close(fd) } + let path = "/\(server.pathToken)/media.m3u8" + #expect(await Self.onOwnThread { Self.requestStatus(fd: fd, path: path) } == 200) + try await Task.sleep(for: .milliseconds(3000)) + #expect(await Self.onOwnThread { Self.requestStatus(fd: fd, path: path) } == 200, + "the second request on an authenticated keep-alive connection was not answered") + } + + @Test("LAN peers cannot take the slots kept for loopback", + .enabled(if: HLSLocalServer.localActiveIPAddress() != nil)) + func lanPeersLeaveLoopbackSlots() async throws { + let lanIP = try #require(HLSLocalServer.localActiveIPAddress()) + let server = HLSLocalServer(provider: StubProvider(), unauthenticatedHeadSeconds: 60) + try server.start() + defer { server.stop() } + + var held: [Int32] = [] + defer { held.forEach { close($0) } } + for _ in 0 ..< HLSLocalServer.maxConcurrentConnections + 2 { + if let fd = Self.connect(port: server.port, host: lanIP) { held.append(fd) } + } + try await waitFor { server.activeConnectionCount >= HLSLocalServer.maxNonLoopbackConnections } + #expect(server.activeConnectionCount == HLSLocalServer.maxNonLoopbackConnections) + + let fd = try #require(Self.connect(port: server.port, host: "127.0.0.1")) + defer { close(fd) } + let path = "/\(server.pathToken)/media.m3u8" + #expect(await Self.onOwnThread { Self.requestStatus(fd: fd, path: path) } == 200, + "a loopback player was refused while LAN peers held their share") + } + + @Test("The stop line names the port that was released") + func stopLineNamesThePort() throws { + let tap = EngineLogCapture() + defer { tap.end() } + let server = HLSLocalServer(provider: StubProvider()) + try server.start() + let port = server.port + server.stop() + #expect(!tap.matching("[HLSLocalServer] stop: port \(port) released").isEmpty) + #expect(tap.matching("[HLSLocalServer] stop: port 0 released").isEmpty) + } + + /// Audit NET-111: every tokenless request cost the host's log two unthrottled lines, so a LAN + /// peer looping short connections could scroll a 300-line ring in a fraction of a second. + @Test("A flood of tokenless requests costs the log a handful of lines") + func tokenlessFloodIsThrottled() async throws { + let tap = EngineLogCapture() + defer { tap.end() } + let server = HLSLocalServer(provider: StubProvider()) + try server.start() + defer { server.stop() } + let marker = "flood-\(UUID().uuidString)" + let port = server.port + + let answered = await Self.onOwnThread { () -> Int in + var answered = 0 + for _ in 0 ..< 1000 { + guard let fd = Self.connect(port: port, host: "127.0.0.1") else { continue } + if Self.requestStatus(fd: fd, path: "/\(marker)/media.m3u8") == 404 { answered += 1 } + close(fd) + } + return answered + } + #expect(answered == 1000) + #expect(tap.matching(marker).count <= 3, "\(tap.matching(marker).count) lines for 1000 requests") + } + + @Test("Attacker text is logged with its control characters escaped") + func controlCharactersAreEscaped() { + #expect(HLSLocalServer.escapedForLog("GET /x\n[HLSLocalServer] GET /forged HTTP/1.1") + == "GET /x\\x0A[HLSLocalServer] GET /forged HTTP/1.1") + #expect(HLSLocalServer.escapedForLog("a\u{7F}b\tc\u{0}") == "a\\x7Fb\\x09c\\x00") + #expect(HLSLocalServer.escapedForLog("GET /seg_1.m4s HTTP/1.1") == "GET /seg_1.m4s HTTP/1.1") + #expect(HLSLocalServer.escapedForLog(String(repeating: "a", count: 300), limit: 256) + == String(repeating: "a", count: 256) + "...") + } + + @Test("A repeating failure line goes out once per interval with a tally of the rest") + func throttleCountsWhatItHeldBack() { + var throttle = LogThrottle(interval: 5) + #expect(throttle.admit(now: 100) == 0) + #expect(throttle.admit(now: 100.1) == nil) + #expect(throttle.admit(now: 104.9) == nil) + #expect(throttle.admit(now: 105) == 2) + #expect(throttle.admit(now: 106) == nil) + #expect(throttle.admit(now: 200) == 1) + } + + // MARK: - Helpers + + private static func connect(port: UInt16, host: String) -> Int32? { + let fd = socket(AF_INET, SOCK_STREAM, IPPROTO_TCP) + guard fd >= 0 else { return nil } + var on: Int32 = 1 + _ = setsockopt(fd, SOL_SOCKET, SO_NOSIGPIPE, &on, socklen_t(MemoryLayout.size)) + var addr = sockaddr_in() + addr.sin_len = UInt8(MemoryLayout.size) + addr.sin_family = sa_family_t(AF_INET) + addr.sin_port = port.bigEndian + addr.sin_addr.s_addr = inet_addr(host) + let connected = withUnsafePointer(to: &addr) { ptr -> Int32 in + ptr.withMemoryRebound(to: sockaddr.self, capacity: 1) { + Darwin.connect(fd, $0, socklen_t(MemoryLayout.size)) + } + } + guard connected == 0 else { + close(fd) + return nil + } + return fd + } + + /// One keep-alive GET on an open connection; the status, or 0. Reads until the declared body + /// has arrived so the next request on the same socket starts clean. + private static func requestStatus(fd: Int32, path: String) -> Int { + var timeout = timeval(tv_sec: 10, tv_usec: 0) + _ = setsockopt(fd, SOL_SOCKET, SO_RCVTIMEO, &timeout, socklen_t(MemoryLayout.size)) + let request = "GET \(path) HTTP/1.1\r\nHost: 127.0.0.1\r\n\r\n" + let sent = Array(request.utf8).withUnsafeBytes { send(fd, $0.baseAddress, $0.count, 0) } + guard sent > 0 else { return 0 } + var received = Data() + var chunk = [UInt8](repeating: 0, count: 4096) + while true { + let n = recv(fd, &chunk, chunk.count, 0) + guard n > 0 else { return 0 } + received.append(chunk, count: n) + guard let text = String(data: received, encoding: .utf8), + let headEnd = text.range(of: "\r\n\r\n") else { continue } + let head = text[..= length else { continue } + let parts = (head.components(separatedBy: "\r\n").first ?? "").split(separator: " ") + return parts.count >= 2 ? (Int(parts[1]) ?? 0) : 0 + } + } + + /// True once the server has closed or reset `fd`, false if it is still open after `seconds`. + private static func peerCloses(fd: Int32, within seconds: Int) -> Bool { + var timeout = timeval(tv_sec: seconds, tv_usec: 0) + _ = setsockopt(fd, SOL_SOCKET, SO_RCVTIMEO, &timeout, socklen_t(MemoryLayout.size)) + var chunk = [UInt8](repeating: 0, count: 512) + while true { + let n = recv(fd, &chunk, chunk.count, 0) + if n == 0 { return true } + if n < 0 { return errno == ECONNRESET } + } + } + + private static func onOwnThread(_ body: @escaping @Sendable () -> T) async -> T { + await withCheckedContinuation { continuation in + Thread.detachNewThread { continuation.resume(returning: body()) } + } + } +} + +/// Sends one byte of a request head every 200 ms from its own thread until stopped or refused. +private final class Trickle: @unchecked Sendable { + private let lock = NSLock() + private var stopped = false + + init(fd: Int32) { + Thread.detachNewThread { [self] in + let head = Array("GET / HTTP/1.1\r\nHost: x\r\nX-Pad: \(String(repeating: "a", count: 400))\r\n\r\n".utf8) + for byte in head { + if !sendUnlessStopped(byte, fd: fd) { return } + Thread.sleep(forTimeInterval: 0.2) + } + } + } + + /// The check and the send hold the lock together, so once `stop()` returns no send is in flight + /// and the caller's `close(fd)` cannot let a reused descriptor receive a stray byte. + private func sendUnlessStopped(_ byte: UInt8, fd: Int32) -> Bool { + lock.lock(); defer { lock.unlock() } + if stopped { return false } + var one = byte + return send(fd, &one, 1, 0) == 1 + } + + func stop() { + lock.lock(); stopped = true; lock.unlock() + } +} + +private final class StubProvider: HLSSegmentProvider, @unchecked Sendable { + func initSegment() -> Data? { Data([0x00]) } + func mediaSegment(at index: Int) -> Data? { Data([0x00]) } + var segmentCount: Int { 1 } + func segmentDuration(at index: Int) -> Double { 4.0 } + var playlistType: HLSPlaylistType { .vod } +} diff --git a/Tests/AetherEngineTests/HLSOriginRelayStalledConsumerTests.swift b/Tests/AetherEngineTests/HLSOriginRelayStalledConsumerTests.swift new file mode 100644 index 000000000..6c0c1919d --- /dev/null +++ b/Tests/AetherEngineTests/HLSOriginRelayStalledConsumerTests.swift @@ -0,0 +1,187 @@ +// A relayed fetch whose consumer stopped reading must cost the session nothing but its own +// memory, bounded. Everything here drives a real origin subprocess, which exists on macOS only. +#if os(macOS) + +import Foundation +import Testing + +@testable import AetherEngine + +/// Audit NET-107: the pump used to park the session's serial delegate queue inside `didReceive` +/// once a fetch held 4 MB its consumer had not taken, so one receiver that stopped reading held up +/// the head and the body of every other relayed fetch for as long as the stall lasted. +@Suite("HLS origin relay with a consumer that stops reading", .timeLimit(.minutes(3))) +struct HLSOriginRelayStalledConsumerTests { + + /// A fixed-length body written as fast as the socket takes it, with a marker file once the last + /// byte has gone out. The marker is what says the relay kept reading the origin while its own + /// consumer did not. + private final class FastBodyOrigin { + let port: UInt16 + private let process: Process + private let workDir: URL + + var finishedSending: Bool { + FileManager.default.fileExists(atPath: workDir.appendingPathComponent("done").path) + } + + init?(megabytes: Int) async { + guard let launched = await PythonOrigin.launch( + prefix: "aether-fast-body-origin", script: Self.serverPy(bytes: megabytes << 20)) + else { return nil } + process = launched.process + port = launched.port + workDir = launched.workDir + } + + func stop() { + process.terminate() + try? FileManager.default.removeItem(at: workDir) + } + + private static func serverPy(bytes: Int) -> String { + """ + import http.server + + TOTAL = \(bytes) + CHUNK = 65536 + + class Handler(http.server.BaseHTTPRequestHandler): + protocol_version = "HTTP/1.1" + + def log_message(self, *args): + pass + + def do_GET(self): + self.send_response(200) + self.send_header("Content-Length", str(TOTAL)) + self.send_header("Content-Type", "video/mp2t") + self.end_headers() + sent = 0 + try: + while sent < TOTAL: + n = min(CHUNK, TOTAL - sent) + self.wfile.write(b"\\x47" * n) + sent += n + self.wfile.flush() + open("done", "w").close() + except OSError: + return + + server = http.server.ThreadingHTTPServer(("127.0.0.1", 0), Handler) + print("READY", server.server_address[1], flush=True) + server.serve_forever() + """ + } + } + + /// A client that sends its request on a raw socket with a tiny receive buffer and then never + /// reads, which is what a receiver on a link slower than the stream looks like to the server. + private final class StalledReader { + private var fd: Int32 + + init?(url: URL) { + guard let host = url.host, let port = url.port else { return nil } + fd = socket(AF_INET, SOCK_STREAM, 0) + guard fd >= 0 else { return nil } + var small: Int32 = 4096 + setsockopt(fd, SOL_SOCKET, SO_RCVBUF, &small, socklen_t(MemoryLayout.size)) + + var address = sockaddr_in() + address.sin_family = sa_family_t(AF_INET) + address.sin_port = UInt16(port).bigEndian + guard inet_pton(AF_INET, host, &address.sin_addr) == 1 else { Darwin.close(fd); return nil } + let connected = withUnsafePointer(to: &address) { + $0.withMemoryRebound(to: sockaddr.self, capacity: 1) { + connect(fd, $0, socklen_t(MemoryLayout.size)) + } + } + guard connected == 0 else { Darwin.close(fd); return nil } + + let target = url.path + (url.query.map { "?\($0)" } ?? "") + let wire = Array("GET \(target) HTTP/1.1\r\nHost: \(host):\(port)\r\n\r\n".utf8) + let sent = wire.withUnsafeBufferPointer { send(fd, $0.baseAddress, $0.count, 0) } + guard sent == wire.count else { Darwin.close(fd); return nil } + } + + func close() { + guard fd >= 0 else { return } + Darwin.close(fd) + fd = -1 + } + } + + private func entry(_ server: HLSLocalServer, port: UInt16, path: String) throws -> URL { + try #require(server.relayURL(for: URL(string: "http://127.0.0.1:\(port)\(path)")!)) + } + + @Test("A player that stops reading one segment does not hold up another relayed fetch") + func stalledConsumerDoesNotBlockOtherFetches() async throws { + let big = try #require(await FastBodyOrigin(megabytes: 16)) + let quick = try #require(await TricklingOrigin(slices: 2, pauseSeconds: 0)) + defer { big.stop(); quick.stop() } + let relay = HLSOriginRelay() + let server = HLSLocalServer(relay: relay) + try server.start() + defer { server.stop(); relay.stop() } + + let stalled = try #require(StalledReader(url: try entry(server, port: big.port, path: "/big.ts"))) + defer { stalled.close() } + + // The relay keeps taking the body off the origin while nobody reads it downstream. Parked at + // 4 MB it never does, and the origin's last write never completes. + let absorbed = try await waitFor(upTo: .seconds(30)) { big.finishedSending } + #expect(absorbed, "the relay stopped reading the origin because its consumer stalled") + + var request = URLRequest(url: try entry(server, port: quick.port, path: "/quick.ts")) + request.timeoutInterval = 20 + let (body, response) = try await URLSession.shared.data(for: request) + #expect((response as? HTTPURLResponse)?.statusCode == 200) + #expect(body.count == TricklingOrigin.totalBytes(slices: 2)) + #expect(relay.cappedFetchCount == 0, "16 MB is under the 32 MiB cap") + } + + @Test("A consumer that falls past the cap is cut off and the fetch is cancelled, not parked") + func consumerPastTheCapIsCutOff() async throws { + let big = try #require(await FastBodyOrigin(megabytes: 16)) + let quick = try #require(await TricklingOrigin(slices: 2, pauseSeconds: 0)) + defer { big.stop(); quick.stop() } + let relay = HLSOriginRelay(maximumPendingBytes: 1 << 20) + let server = HLSLocalServer(relay: relay) + try server.start() + defer { server.stop(); relay.stop() } + + let stalled = try #require(StalledReader(url: try entry(server, port: big.port, path: "/big.ts"))) + defer { stalled.close() } + + try await waitFor { relay.cappedFetchCount == 1 } + + var request = URLRequest(url: try entry(server, port: quick.port, path: "/quick.ts")) + request.timeoutInterval = 20 + let (body, response) = try await URLSession.shared.data(for: request) + #expect((response as? HTTPURLResponse)?.statusCode == 200) + #expect(body.count == TricklingOrigin.totalBytes(slices: 2)) + #expect(relay.cappedFetchCount == 1) + } + + @Test("A body that fits under the cap arrives whole however late its consumer starts") + func bodyUnderTheCapArrivesWhole() async throws { + // Four megabytes against an eight megabyte cap: even if the consumer took nothing until the + // origin had finished, the fetch holds at most what the origin sent, so no timing decides this. + let origin = try #require(await TricklingOrigin(slices: 64, pauseSeconds: 0)) + defer { origin.stop() } + let relay = HLSOriginRelay(maximumPendingBytes: 8 << 20) + let server = HLSLocalServer(relay: relay) + try server.start() + defer { server.stop(); relay.stop() } + + var request = URLRequest(url: try entry(server, port: origin.port, path: "/seg.ts")) + request.timeoutInterval = 30 + let (body, response) = try await URLSession.shared.data(for: request) + #expect((response as? HTTPURLResponse)?.statusCode == 200) + #expect(body.count == TricklingOrigin.totalBytes(slices: 64)) + #expect(relay.cappedFetchCount == 0) + } +} + +#endif diff --git a/Tests/AetherEngineTests/HLSOriginRelayTests.swift b/Tests/AetherEngineTests/HLSOriginRelayTests.swift index f870d5479..cefc73add 100644 --- a/Tests/AetherEngineTests/HLSOriginRelayTests.swift +++ b/Tests/AetherEngineTests/HLSOriginRelayTests.swift @@ -14,22 +14,71 @@ struct HLSOriginRelayAddressingTests { @Test("An origin survives the round trip through a relay URL") func roundTrip() throws { + let relay = HLSOriginRelay() let origin = URL( string: "https://media.example.com:8920/videos/1/master.m3u8" + "?ApiKey=abc123&PlaySessionId=x%20y&tag=a+b")! - let local = try #require(HLSOriginRelay.localURL(for: origin, port: 51234, token: token)) + let local = try #require(relay.localURL(for: origin, port: 51234, token: token)) #expect(local.host == "127.0.0.1") #expect(local.port == 51234) #expect(local.path == "/\(token)\(HLSOriginRelay.route)") - let recovered = try #require(HLSOriginRelay.originURL(fromQuery: local.query ?? "")) + let recovered = try #require(relay.originURL(fromQuery: local.query ?? "")) #expect(recovered == origin, "recovered \(recovered) from \(origin)") } + @Test("A relay URL names the origin by reference, never by its host, path or token (audit NET-1)") + func localURLCarriesNoOriginText() throws { + // The local URL is logged on every request, handed to AirPlay receivers and written into + // every rewritten playlist, so nothing of the origin may be readable in it, in any encoding. + let relay = HLSOriginRelay() + let secret = "0123456789abcdef0123456789abcdef" + let origin = URL( + string: "https://jf.example.com/Videos/abc/master.m3u8?MediaSourceId=x&api_key=\(secret)")! + let local = try #require(relay.localURL(for: origin, port: 51234, token: token)) + let text = local.absoluteString + // Fragments long enough to mean something: the reference is random base64, and a two-letter + // fragment ("jf") turns up in it by chance on about one run in twenty (CI, 2026-09-28). + for fragment in [secret, "jf.example", "example", "Videos", "master.m3u8", "api_key", "MediaSourceId"] { + #expect(!text.contains(fragment), "\(fragment) is readable in \(text)") + } + #expect(!text.contains("%"), "the reference should need no escaping: \(text)") + } + + @Test("One origin always seals to the same local URL, so a refreshed playlist names a segment the same way") + func sealingIsStable() throws { + let relay = HLSOriginRelay() + let segment = URL(string: "https://media.example.com/hls/seg42.ts?api_key=k")! + let first = try #require(relay.localURL(for: segment, port: 51234, token: token)) + let again = try #require(relay.localURL(for: segment, port: 51234, token: token)) + let other = try #require(relay.localURL( + for: URL(string: "https://media.example.com/hls/seg43.ts?api_key=k")!, port: 51234, token: token)) + #expect(first == again) + #expect(first != other) + } + + @Test("A reference from another session, or one that was altered, opens to nothing") + func foreignOrTamperedReferenceIsRefused() throws { + let origin = URL(string: "https://media.example.com/hls/media.m3u8?ApiKey=k")! + let previousSession = HLSOriginRelay() + let stale = try #require(previousSession.localURL(for: origin, port: 51234, token: token)) + let relay = HLSOriginRelay() + #expect(relay.originURL(fromQuery: stale.query ?? "") == nil) + + let own = try #require(relay.localURL(for: origin, port: 51234, token: token)) + var query = Array(own.query ?? "") + let flipAt = query.count - 3 + query[flipAt] = query[flipAt] == "A" ? "B" : "A" + #expect(relay.originURL(fromQuery: String(query)) == nil) + } + @Test("A query that names no origin yields none") func rejectsForeignQuery() { - #expect(HLSOriginRelay.originURL(fromQuery: "") == nil) - #expect(HLSOriginRelay.originURL(fromQuery: "other=https%3A%2F%2Fa.b") == nil) + let relay = HLSOriginRelay() + #expect(relay.originURL(fromQuery: "") == nil) + #expect(relay.originURL(fromQuery: "other=https%3A%2F%2Fa.b") == nil) + #expect(relay.originURL(fromQuery: "origin=https%3A%2F%2Fa.b") == nil, + "the pre-NET-1 plain origin form must not be honoured") } @Test("The origin key keeps scheme, host and port apart") @@ -56,7 +105,7 @@ struct HLSOriginRelayAddressingTests { func rewritesMediaPlaylist() throws { let relay = HLSOriginRelay() let origin = URL(string: "https://media.example.com/hls/media.m3u8?ApiKey=k")! - relay.admit(origin) + relay.grantCredentials(to: origin) let playlist = """ #EXTM3U @@ -90,8 +139,10 @@ struct HLSOriginRelayAddressingTests { // A relative segment resolves against the playlist it came from. let segLine = try #require(lines.first { !$0.hasPrefix("#") && !$0.isEmpty }) let query = try #require(URL(string: segLine)?.query) - let recovered = try #require(HLSOriginRelay.originURL(fromQuery: query)) + let recovered = try #require(relay.originURL(fromQuery: query)) #expect(recovered.absoluteString == "https://media.example.com/hls/seg0.ts") + #expect(!rewritten.contains("ApiKey"), "the origin's query leaked into the rewritten playlist") + #expect(!rewritten.contains("example.com"), "an origin host leaked into the rewritten playlist") } @Test("An injected rendition stays with the server that owns it") @@ -102,7 +153,7 @@ struct HLSOriginRelayAddressingTests { // engine has. let relay = HLSOriginRelay() let origin = URL(string: "https://media.example.com/hls/master.m3u8")! - relay.admit(origin) + relay.grantCredentials(to: origin) let master = """ #EXTM3U @@ -130,14 +181,14 @@ struct HLSOriginRelayAddressingTests { // fetch to time out. Nothing here waits on a name resolving, and the three // differ only by port, which the allow list treats as part of the origin. let origin = URL(string: "https://127.0.0.1:9/hls/media.m3u8")! - relay.admit(origin) + relay.grantCredentials(to: origin) _ = relay.rewritePlaylist( "#EXTM3U\n#EXTINF:6.0,\nhttps://127.0.0.1:10/seg1.ts\n", relativeTo: origin, port: server.port, token: server.pathToken) let named = try #require(server.relayURL(for: URL(string: "https://127.0.0.1:10/seg1.ts")!)) - let stranger = try #require(HLSOriginRelay.localURL( + let stranger = try #require(relay.localURL( for: URL(string: "https://127.0.0.1:11/x.ts")!, port: server.port, token: server.pathToken)) @@ -152,9 +203,9 @@ struct HLSOriginRelayAddressingTests { try server.start() defer { server.stop(); relay.stop() } let origin = URL(string: "https://127.0.0.1:9/hls/media.m3u8")! - relay.admit(origin) + relay.grantCredentials(to: origin) - let stranger = try #require(HLSOriginRelay.localURL( + let stranger = try #require(relay.localURL( for: origin, port: server.port, token: String(repeating: "cd", count: 16))) #expect(try await status(of: stranger) == 404, "a stale or scanned token was answered") } @@ -162,13 +213,14 @@ struct HLSOriginRelayAddressingTests { @Test("A blocking-reload request keeps the parameters that make it block") func blockingReloadParametersReachTheOrigin() throws { // AVPlayer appends _HLS_msn / _HLS_part to a playlist URL that advertises CAN-BLOCK-RELOAD - // (#441). The relay URL already carries a query, so they arrive alongside `origin`; dropped, - // the reload answers at once and the player asks again immediately. + // (#441). The relay URL already carries a query, so they arrive alongside the reference; + // dropped, the reload answers at once and the player asks again immediately. + let relay = HLSOriginRelay() let origin = URL(string: "https://media.example.com/hls/media.m3u8?ApiKey=k")! - let local = try #require(HLSOriginRelay.localURL(for: origin, port: 51234, token: token)) + let local = try #require(relay.localURL(for: origin, port: 51234, token: token)) let asAVPlayerAsks = "\(local.query ?? "")&_HLS_msn=42&_HLS_part=3" - let upstream = try #require(HLSOriginRelay.originURL(fromQuery: asAVPlayerAsks)) + let upstream = try #require(relay.originURL(fromQuery: asAVPlayerAsks)) #expect(upstream.path == "/hls/media.m3u8") let query = try #require(upstream.query) #expect(query.contains("ApiKey=k"), "the origin's own query was dropped: \(query)") @@ -182,6 +234,41 @@ struct HLSOriginRelayAddressingTests { // which a macOS `swift build` never shows. #if os(macOS) + @Test("A held body that outgrows its cap is refused instead of buffered whole (audit NET-10)") + func heldBodyIsBounded() async throws { + // No length stated, so the relay has to hold it; eight 64 KB slices against a 256 KB cap. + let upstream = try #require(await TricklingOrigin(slices: 8, pauseSeconds: 0, declaresLength: false)) + defer { upstream.stop() } + let relay = HLSOriginRelay(maximumHeldBodyBytes: 256 * 1024) + let server = HLSLocalServer(relay: relay) + try server.start() + defer { server.stop(); relay.stop() } + + let entry = try #require(server.relayURL(for: URL(string: "http://127.0.0.1:\(upstream.port)/seg.ts")!)) + var request = URLRequest(url: entry) + request.timeoutInterval = 30 + let (body, response) = try await URLSession.shared.data(for: request) + let http = try #require(response as? HTTPURLResponse) + #expect(http.statusCode == 502, "an over-cap held body was relayed (\(body.count) bytes)") + } + + @Test("A held body under its cap still arrives whole") + func heldBodyUnderCapArrives() async throws { + let upstream = try #require(await TricklingOrigin(slices: 2, pauseSeconds: 0, declaresLength: false)) + defer { upstream.stop() } + let relay = HLSOriginRelay(maximumHeldBodyBytes: 256 * 1024) + let server = HLSLocalServer(relay: relay) + try server.start() + defer { server.stop(); relay.stop() } + + let entry = try #require(server.relayURL(for: URL(string: "http://127.0.0.1:\(upstream.port)/seg.ts")!)) + var request = URLRequest(url: entry) + request.timeoutInterval = 30 + let (body, response) = try await URLSession.shared.data(for: request) + #expect((response as? HTTPURLResponse)?.statusCode == 200) + #expect(body.count == TricklingOrigin.totalBytes(slices: 2)) + } + @Test("A range is forwarded verbatim and its framing comes back") func rangesPassThroughUntouched() async throws { let upstream = try #require(await RangeEchoOrigin()) diff --git a/Tests/AetherEngineTests/HLSPlaylistHostileInputTests.swift b/Tests/AetherEngineTests/HLSPlaylistHostileInputTests.swift new file mode 100644 index 000000000..93a5760db --- /dev/null +++ b/Tests/AetherEngineTests/HLSPlaylistHostileInputTests.swift @@ -0,0 +1,195 @@ +import Testing +import Foundation +@testable import AetherEngine + +/// Audit NET-105 / NAT-102 / NET-106: `HLSPlaylistParser.attribute` recounted the quotes from the line +/// start for every `KEY=` it met (a 64 KiB line took 9.6 s, 4x per doubling), and every +/// EXT-X-PROGRAM-DATE-TIME line built two `ISO8601DateFormatter`s. +struct HLSPlaylistHostileInputTests { + + /// The implementation `attribute` had before the single pass, kept as the oracle. + private func legacyAttribute(_ key: String, in line: String) -> String? { + let needle = "\(key)=" + var searchStart = line.startIndex + while let range = line.range(of: needle, range: searchStart.. UInt64 { + state &+= 0x9E37_79B9_7F4A_7C15 + var z = state + z = (z ^ (z >> 30)) &* 0xBF58_476D_1CE4_E5B9 + z = (z ^ (z >> 27)) &* 0x94D0_49BB_1331_11EB + return z ^ (z >> 31) + } + } + + private static let keys = ["BANDWIDTH", "AVERAGE-BANDWIDTH", "URI", "TYPE", "GROUP-ID", "NAME", "DEFAULT", + "SUBTITLES", "AUDIO", "METHOD", "IV", "LANGUAGE", "FORCED"] + + private static let ordinaryLines = [ + "#EXT-X-STREAM-INF:BANDWIDTH=2000000,AVERAGE-BANDWIDTH=1500000,AUDIO=\"aud\",SUBTITLES=\"subs\"", + "#EXT-X-STREAM-INF:AVERAGE-BANDWIDTH=1500000,BANDWIDTH=2000000", + "#EXT-X-STREAM-INF:AVERAGE-BANDWIDTH=1500000", + "#EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID=\"subs\",NAME=\"Deutsch, SDH\",LANGUAGE=\"de\",DEFAULT=YES,URI=\"de.m3u8\"", + "#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID=\"a\",NAME=\"TYPE=VIDEO\",URI=\"a.m3u8\"", + "#EXT-X-KEY:METHOD=AES-128,URI=\"https://k.example/key?a=1,b=2\",IV=0x00000000000000000000000000000001", + "#EXT-X-KEY:METHOD=NONE", + "#EXT-X-MEDIA:NAME=\"unterminated,TYPE=AUDIO", + "#EXT-X-MEDIA:URI=\"", + "#EXT-X-MEDIA:URI=", + "URI=first,TYPE=x", + "xURI=nope,URI=yes", + "", + "=", + "#EXT-X-MEDIA:NAME=\"caf\u{00E9} \u{4E2D}\u{6587}\",URI=\"\u{1F600}.m3u8\"", + ] + + @Test("the single pass matches the old scan on ordinary tag lines") + func matchesOldScanOnOrdinaryLines() { + for line in Self.ordinaryLines { + for key in Self.keys { + #expect(HLSPlaylistParser.attribute(key, in: line) == legacyAttribute(key, in: line), + "key \(key) line \(line.debugDescription)") + } + } + } + + @Test("the single pass matches the old scan on generated quote and comma soup") + func matchesOldScanOnGeneratedLines() { + let pieces = ["\"", "\"", ",", ",", ":", "=", "#EXT-X-MEDIA:", "URI=", "TYPE=", "BANDWIDTH=", "AVERAGE-", + "NAME=", "a", "b", " ", "x=y", "GROUP-ID=", "\u{00E9}"] + var rng = SplitMix64(state: 0x1105_0105) + for _ in 0..<4000 { + var line = "" + for _ in 0.. Double? { + let text = "#EXTM3U\n#EXT-X-TARGETDURATION:9\n\(tag)\nseg.ts\n" + guard case .media(let media) = try HLSPlaylistParser.parse(text) else { return nil } + return media.segments.first?.duration + } + #expect(try duration("#EXTINF:6.5,Title, with, commas") == 6.5) + #expect(try duration("#EXTINF:6.5") == 6.5) + #expect(try duration("#EXTINF:,7,x") == 7) + #expect(try duration("#EXTINF:,") == 9) + #expect(try duration("#EXTINF:abc,x") == 9) + } + + // MARK: - EXT-X-PROGRAM-DATE-TIME + + private func legacyProgramDateTime(_ raw: String) -> Date? { + let text = raw.trimmingCharacters(in: .whitespaces) + let withFraction = ISO8601DateFormatter() + withFraction.formatOptions = [.withInternetDateTime, .withFractionalSeconds] + if let date = withFraction.date(from: text) { return date } + let plain = ISO8601DateFormatter() + plain.formatOptions = [.withInternetDateTime] + return plain.date(from: text) + } + + private func firstSegmentDate(_ pdt: String) throws -> Date? { + let text = "#EXTM3U\n#EXT-X-TARGETDURATION:6\n#EXT-X-PROGRAM-DATE-TIME:\(pdt)\n#EXTINF:6.0,\nseg.ts\n" + guard case .media(let media) = try HLSPlaylistParser.parse(text) else { return nil } + return media.segments.first?.programDateTime + } + + @Test("fractional and plain program dates parse as the ISO8601DateFormatters did") + func programDatesMatchTheOldFormatters() throws { + let shapes = ["2026-09-30T10:00:00.000Z", "2026-09-30T10:00:00Z", "2026-09-30T10:00:00.123Z", + "2026-09-30T10:00:00.123456Z", "2026-09-30T10:00:00.000+00:00", "2026-09-30T10:00:00.000+0000", + "2026-09-30T12:00:00+02:00", "2026-09-30T12:00:00.250+02:00", "2026-09-30T10:00:00.5Z", + " 2026-09-30T10:00:00.000Z ", "2026-09-30 10:00:00Z", "garbage", "", "2026-13-45T99:00:00Z"] + for shape in shapes { + let expected = legacyProgramDateTime(shape) + let actual = try firstSegmentDate(shape) + switch (expected, actual) { + case (nil, nil): break + case let (expected?, actual?): + #expect(abs(expected.timeIntervalSince(actual)) < 0.001, "shape \(shape.debugDescription)") + default: + Issue.record("shape \(shape.debugDescription): old \(String(describing: expected)) new \(String(describing: actual))") + } + } + } + + @Test("a playlist with a program date on every segment parses in linear time") + func programDatePerSegmentIsLinear() throws { + var text = "#EXTM3U\n#EXT-X-TARGETDURATION:2\n#EXT-X-MEDIA-SEQUENCE:1\n" + for index in 0..<7_200 { + let second = index * 2 + let stamp = String(format: "2026-09-30T%02d:%02d:%02d.000Z", (second / 3600) % 24, (second / 60) % 60, second % 60) + text += "#EXT-X-PROGRAM-DATE-TIME:\(stamp)\n#EXTINF:2.0,\nseg\(index).ts\n" + } + let started = ContinuousClock.now + let parsed = try HLSPlaylistParser.parse(text) + let elapsed = ContinuousClock.now - started + guard case .media(let media) = parsed else { Issue.record("not a media playlist"); return } + #expect(media.segments.count == 7_200) + #expect(media.segments.allSatisfy { $0.programDateTime != nil }) + #expect(elapsed < .seconds(1), "took \(elapsed)") + } +} diff --git a/Tests/AetherEngineTests/HLSPlaylistTests.swift b/Tests/AetherEngineTests/HLSPlaylistTests.swift index dbbe00aa9..086257d6d 100644 --- a/Tests/AetherEngineTests/HLSPlaylistTests.swift +++ b/Tests/AetherEngineTests/HLSPlaylistTests.swift @@ -204,6 +204,48 @@ final class HLSPlaylistTests: XCTestCase { XCTAssertThrowsError(try HLSPlaylistParser.parse("#EXTM3U\n#EXT-X-TARGETDURATION:6\n")) } + // audit NET-2 / NAT-3: "\r\n" is one Character in Swift, so `split(separator: "\n")` read a + // whole CRLF playlist (Windows-hosted IPTV panels, some packagers) as a single line. + func testParsesCRLFMediaPlaylist() throws { + let text = "#EXTM3U\r\n#EXT-X-TARGETDURATION:6\r\n#EXT-X-MEDIA-SEQUENCE:147\r\n" + + "#EXTINF:6.000,\r\nseg147.ts\r\n" + guard case .media(let media) = try HLSPlaylistParser.parse(text) else { + return XCTFail("expected media playlist") + } + XCTAssertEqual(media.targetDuration, 6.0) + XCTAssertEqual(media.mediaSequence, 147) + XCTAssertEqual(media.segments.map(\.uri), ["seg147.ts"]) + } + + func testParsesCRLFMasterPlaylist() throws { + let text = "#EXTM3U\r\n#EXT-X-STREAM-INF:BANDWIDTH=1000\r\nv.m3u8\r\n" + guard case .master(let master) = try HLSPlaylistParser.parse(text) else { + return XCTFail("expected master playlist") + } + XCTAssertEqual(master.variants.map(\.uri), ["v.m3u8"]) + } + + // audit NET-3: a hostile or MITM MEDIA-SEQUENCE near Int.max used to reach + // `mediaSequence + segments.count` unchecked in HLSPlaylistTracker and trap. + func testRejectsMediaSequenceNearIntMax() { + let text = "#EXTM3U\n#EXT-X-TARGETDURATION:6\n#EXT-X-MEDIA-SEQUENCE:9223372036854775807\n" + + "#EXTINF:6.000,\nseg.ts\n" + XCTAssertThrowsError(try HLSPlaylistParser.parse(text)) { error in + guard case HLSIngestError.playlistInvalid = error else { + return XCTFail("expected playlistInvalid, got \(error)") + } + } + } + + func testRejectsNegativeMediaSequence() { + let text = "#EXTM3U\n#EXT-X-TARGETDURATION:6\n#EXT-X-MEDIA-SEQUENCE:-1\n#EXTINF:6.000,\nseg.ts\n" + XCTAssertThrowsError(try HLSPlaylistParser.parse(text)) { error in + guard case HLSIngestError.playlistInvalid = error else { + return XCTFail("expected playlistInvalid, got \(error)") + } + } + } + // AE#359: the master's SUBTITLES group was parsed away, so a live channel that offers WebVTT // renditions (every public German broadcaster does) had no subtitle track to select. Fixture is // MDR Sachsen's real master, trimmed to one variant and its rendition line. diff --git a/Tests/AetherEngineTests/HLSPlaylistTrackerTests.swift b/Tests/AetherEngineTests/HLSPlaylistTrackerTests.swift index 32d46b2ac..836589b71 100644 --- a/Tests/AetherEngineTests/HLSPlaylistTrackerTests.swift +++ b/Tests/AetherEngineTests/HLSPlaylistTrackerTests.swift @@ -16,10 +16,11 @@ final class HLSPlaylistTrackerTests: XCTestCase { } func testPrimesAtLiveEdgeWithCoverageTarget() { - // 4s segments: coverage = max(8, 1.5*4) = 8s -> join takes exactly two segments. + // 4s segments: the loopback cushion wants 3 x ceil(4 / 1.5) = 9s plus a 4s GOP margin, so + // 13s; edgeOffset caps the join at three segments. var tracker = HLSPlaylistTracker(edgeOffset: 3, minJoinCoverageSeconds: 8) let new = tracker.newSegments(in: playlist(sequence: 100, uris: ["a", "b", "c", "d", "e", "f"])) - XCTAssertEqual(new.map(\.uri), ["e", "f"]) + XCTAssertEqual(new.map(\.uri), ["d", "e", "f"]) XCTAssertEqual(tracker.stallCount, 0) } @@ -31,17 +32,19 @@ final class HLSPlaylistTrackerTests: XCTestCase { } func testPrimeCoversUpstreamCadenceForLongSegments() { - // 12s segments: coverage = max(8, 1.5*12) = 18s -> two segments (24s) cover a full upstream gap. + // 12s segments: 1.5 x 12 = 18s covers one upstream gap, but the loopback seals TD 8 and wants + // 24s + 4s of margin, so the whole three-segment window. var tracker = HLSPlaylistTracker(edgeOffset: 3, minJoinCoverageSeconds: 8) let new = tracker.newSegments(in: playlist(sequence: 50, uris: ["a", "b", "c"], duration: 12)) - XCTAssertEqual(new.map(\.uri), ["b", "c"]) + XCTAssertEqual(new.map(\.uri), ["a", "b", "c"]) } func testPrimeCoversBurstyTenSecondUpstream() { - // Device-repro shape: 10s segments. Coverage = max(8, 1.5*10) = 15s -> two segments / 20s. + // Device-repro shape: 10s segments. 1.5 x 10 = 15s took two segments / 20s, one short of the + // 21s holdback TD 7 seals (AE#678); 21s + 4s of margin takes three. var tracker = HLSPlaylistTracker(edgeOffset: 3, minJoinCoverageSeconds: 8) let new = tracker.newSegments(in: playlist(sequence: 7, uris: ["a", "b", "c", "d"], duration: 10)) - XCTAssertEqual(new.map(\.uri), ["c", "d"]) + XCTAssertEqual(new.map(\.uri), ["b", "c", "d"]) } func testPrimesAtWindowStartWhenWindowIsShort() { @@ -71,9 +74,9 @@ final class HLSPlaylistTrackerTests: XCTestCase { func testWindowSlidePastCursorRejoinsAtEdgeWithDiscontinuity() { var tracker = HLSPlaylistTracker(edgeOffset: 3, minJoinCoverageSeconds: 8) _ = tracker.newSegments(in: playlist(sequence: 100, uris: ["a", "b", "c"])) - // Window slid past cursor: rejoin at edge (duration-capped to two 4s segments). + // Window slid past cursor: rejoin at edge with the same depth as a join (three 4s segments). let new = tracker.newSegments(in: playlist(sequence: 500, uris: ["x", "y", "z", "w", "v", "u"])) - XCTAssertEqual(new.map(\.uri), ["v", "u"]) + XCTAssertEqual(new.map(\.uri), ["w", "v", "u"]) XCTAssertTrue(new[0].discontinuityBefore, "rejoin must be marked as a discontinuity") } @@ -87,7 +90,7 @@ final class HLSPlaylistTrackerTests: XCTestCase { XCTAssertTrue(tracker.newSegments(in: playlist(sequence: 0, uris: ["x", "y", "z"])).isEmpty) XCTAssertTrue(tracker.newSegments(in: playlist(sequence: 0, uris: ["x", "y", "z"])).isEmpty) let rejoined = tracker.newSegments(in: playlist(sequence: 0, uris: ["x", "y", "z"])) - XCTAssertEqual(rejoined.map(\.uri), ["y", "z"], "third consecutive regression rejoins at the new edge") + XCTAssertEqual(rejoined.map(\.uri), ["x", "y", "z"], "third consecutive regression rejoins at the new edge") XCTAssertTrue(rejoined[0].discontinuityBefore, "reset rejoin must be marked as a discontinuity") // Cursor continues normally on the new sequence axis. let next = tracker.newSegments(in: playlist(sequence: 1, uris: ["y", "z", "w"])) @@ -151,14 +154,36 @@ final class HLSPlaylistTrackerTests: XCTestCase { XCTAssertEqual(new.map(\.uri), ["a", "b", "c"]) } - func testDefaultJoinIsUnchangedForLongSegments() { - // The raised cap moves nothing here: the coverage term breaks a 6s-segment provider at 12s - // on its own, which is why the count was only ever binding on short segments. + func testDefaultJoinCoversTheLoopbackCushionForLongSegments() { + // AE#678: a 6s provider seals TD 4 downstream, so the first serve wants 12s, and two joined + // segments finalize only 12s minus the open GOP. The cushion term takes a third. var tracker = HLSPlaylistTracker() let new = tracker.newSegments( in: playlist(sequence: 3, uris: ["a", "b", "c", "d", "e", "f"], duration: 6) ) - XCTAssertEqual(new.map(\.uri), ["e", "f"]) + XCTAssertEqual(new.map(\.uri), ["d", "e", "f"]) + } + + func testLoopbackCushionCoverageIsTheSealedHoldbackPlusAGOP() { + func coverage(_ durations: [Double]) -> Double { + HLSPlaylistTracker.loopbackCushionCoverageSeconds(segments: durations.enumerated().map { + HLSMediaSegment(uri: "s\($0.offset)", duration: $0.element, discontinuityBefore: false) + }) + } + XCTAssertEqual(coverage([10, 10, 10]), 25) // TD 7, 21s holdback, 4s margin + XCTAssertEqual(coverage([12, 13.5, 12]), 31) // TD 9 from the longest, 27s + 4s + XCTAssertEqual(coverage([2, 2, 2]), 8) // TD 2, 6s + a 2s GOP bounded by the segment + XCTAssertEqual(coverage([1, 1, 1]), 4) // below the 8s floor, which then decides + XCTAssertEqual(coverage([]), 0) + } + + // audit NET-3: a MEDIA-SEQUENCE the parser somehow let through near Int.max used to trap + // `mediaSequence + segments.count` here. `&+`/`&-` compute mod 2^64, so relative distances + // (and therefore which segments are new) come out the same as at a small sequence number. + func testDoesNotTrapOnMediaSequenceNearIntMax() { + var tracker = HLSPlaylistTracker(edgeOffset: 3, minJoinCoverageSeconds: 8) + let new = tracker.newSegments(in: playlist(sequence: Int.max - 1, uris: ["a", "b", "c"])) + XCTAssertEqual(new.map(\.uri), ["a", "b", "c"]) } func testJoinSegmentLimitAppliesTheMarginOnlyBelowTheCap() { diff --git a/Tests/AetherEngineTests/HTTPDiscIOReaderTests.swift b/Tests/AetherEngineTests/HTTPDiscIOReaderTests.swift index 2d3695ec6..e17b0ad53 100644 --- a/Tests/AetherEngineTests/HTTPDiscIOReaderTests.swift +++ b/Tests/AetherEngineTests/HTTPDiscIOReaderTests.swift @@ -112,6 +112,34 @@ struct HTTPDiscIOReaderTests { #expect(Array(out[0..<100]) == Array(src[0..<100])) } + // audit NET-9: a 206 that claims a different start than what was requested must never be + // accepted; the reader would otherwise place those bytes at `position` and corrupt whatever + // structure or stream is read through it. + @Test("A 206 whose Content-Range start does not match the request is refused, not served") + func misrangedResponseIsRefused() throws { + let src = (0..<2000).map { UInt8($0 & 0xff) } + let r = try #require(makeReader(src)) // the offset-0 probe is unaffected and succeeds + MockRangeURLProtocol.misrangedFor.insert("http://disc.test/test.iso") + #expect(r.seek(offset: 1000, whence: SEEK_SET) == 1000) + var out = [UInt8](repeating: 0, count: 100) + let n = out.withUnsafeMutableBufferPointer { r.read($0.baseAddress, size: 100) } + #expect(n == -1) + } + + // audit NET-9: a proxy that falls back to a full 200 mid-session (cache miss, range + // coalescing, a Range-stripping origin) must be treated as fatal for this reader, not read as + // if byte 0 of the body were the requested offset. + @Test("A 200 answer mid-session is refused, not served as if it started at the requested offset") + func plain200MidSessionIsRefused() throws { + let src = (0..<2000).map { UInt8($0 & 0xff) } + let r = try #require(makeReader(src)) // the offset-0 probe is unaffected and succeeds + MockRangeURLProtocol.plain200AfterProbeFor.insert("http://disc.test/test.iso") + #expect(r.seek(offset: 1000, whence: SEEK_SET) == 1000) + var out = [UInt8](repeating: 0, count: 100) + let n = out.withUnsafeMutableBufferPointer { r.read($0.baseAddress, size: 100) } + #expect(n == -1) + } + @Test("A server without range support fails init (caller falls back to streaming)") func noRangeSupport() { let url = URL(string: "http://disc.test/noRange.iso")! @@ -143,9 +171,17 @@ final class MockRangeURLProtocol: URLProtocol, @unchecked Sendable { nonisolated(unsafe) static var disableRangeFor: Set = [] /// Number of leading requests per URL that should fail with a network error (retry testing). nonisolated(unsafe) static var failFirstForURL: [String: Int] = [:] + /// audit NET-9: any request at a non-zero offset gets a 206 that claims to start at 0 (a + /// mis-ranged response) instead of the requested offset, with junk bytes so a caller that + /// trusted it would read garbage rather than the source. + nonisolated(unsafe) static var misrangedFor: Set = [] + /// audit NET-9: any request at a non-zero offset gets a full 200 instead of a 206 (a + /// Range-ignoring proxy or a cache-miss fallback), body starting at byte 0 of the source. + nonisolated(unsafe) static var plain200AfterProbeFor: Set = [] static func reset() { bytesByURL = [:]; disableRangeFor = []; failFirstForURL = [:] + misrangedFor = []; plain200AfterProbeFor = [] } override class func canInit(with request: URLRequest) -> Bool { true } @@ -180,7 +216,16 @@ final class MockRangeURLProtocol: URLProtocol, @unchecked Sendable { status = 206 headers["Content-Range"] = "bytes \(lower)-\(upper)/\(total)" } - let slice = (lower <= upper && lower < total) ? data.subdata(in: lower..<(upper + 1)) : Data() + var slice = (lower <= upper && lower < total) ? data.subdata(in: lower..<(upper + 1)) : Data() + if lower > 0, status == 206, Self.misrangedFor.contains(url.absoluteString) { + headers["Content-Range"] = "bytes 0-\(slice.count - 1)/\(total)" + slice = Data(repeating: 0xEE, count: slice.count) + } + if lower > 0, status == 206, Self.plain200AfterProbeFor.contains(url.absoluteString) { + status = 200 + headers["Content-Range"] = nil + slice = data + } headers["Content-Length"] = String(slice.count) let resp = HTTPURLResponse(url: url, statusCode: status, httpVersion: "HTTP/1.1", headerFields: headers)! client?.urlProtocol(self, didReceive: resp, cacheStoragePolicy: .notAllowed) diff --git a/Tests/AetherEngineTests/IFrameFragmentBuilderTests.swift b/Tests/AetherEngineTests/IFrameFragmentBuilderTests.swift new file mode 100644 index 000000000..7dc62f572 --- /dev/null +++ b/Tests/AetherEngineTests/IFrameFragmentBuilderTests.swift @@ -0,0 +1,188 @@ +// Tests/AetherEngineTests/IFrameFragmentBuilderTests.swift +import Foundation +import Testing +import AetherLibavcodec +@testable import AetherEngine + +private func fixtureURL(_ name: String) -> URL { + URL(fileURLWithPath: #filePath) + .deletingLastPathComponent().deletingLastPathComponent().deletingLastPathComponent() + .appendingPathComponent("Fixtures/\(name)") +} +private func fixtureExists(_ name: String) -> Bool { + FileManager.default.fileExists(atPath: fixtureURL(name).path) +} + +/// Just enough ISO BMFF to read a track timescale, a fragment's base decode time and its sample count. +enum IFrameTestBoxes { + static func u32(_ d: Data, _ off: Int) -> UInt32 { + d.withUnsafeBytes { UInt32(bigEndian: $0.loadUnaligned(fromByteOffset: off, as: UInt32.self)) } + } + static func u64(_ d: Data, _ off: Int) -> UInt64 { + d.withUnsafeBytes { UInt64(bigEndian: $0.loadUnaligned(fromByteOffset: off, as: UInt64.self)) } + } + /// Body range of the first box of `path` nested from the top of `range`. + static func find(_ path: [String], in d: Data, range: Range? = nil) -> Range? { + var range = range ?? 0..? + while off + 8 <= range.upperBound { + let size = Int(u32(d, off)) + guard size >= 8, off + size <= range.upperBound else { break } + if String(bytes: d[(d.startIndex + off + 4)..<(d.startIndex + off + 8)], encoding: .isoLatin1) == name { + found = (off + 8)..<(off + size); break + } + off += size + } + guard let f = found else { return nil } + range = f + } + return range + } + static func topLevelTypes(_ d: Data) -> [String] { + var out: [String] = []; var off = 0 + while off + 8 <= d.count { + let size = Int(u32(d, off)); guard size >= 8 else { break } + out.append(String(bytes: d[(d.startIndex + off + 4)..<(d.startIndex + off + 8)], encoding: .isoLatin1) ?? "?") + off += size + } + return out + } + static func timescale(initSegment d: Data) -> UInt32? { + guard let mdhd = find(["moov", "trak", "mdia", "mdhd"], in: d) else { return nil } + let version = d[d.startIndex + mdhd.lowerBound] + return u32(d, mdhd.lowerBound + (version == 1 ? 20 : 12)) + } + static func baseDecodeTime(fragment d: Data) -> UInt64? { + guard let tfdt = find(["moof", "traf", "tfdt"], in: d) else { return nil } + let version = d[d.startIndex + tfdt.lowerBound] + return version == 1 ? u64(d, tfdt.lowerBound + 4) : UInt64(u32(d, tfdt.lowerBound + 4)) + } + static func sampleCount(fragment d: Data) -> UInt32? { + guard let trun = find(["moof", "traf", "trun"], in: d) else { return nil } + return u32(d, trun.lowerBound + 4) + } +} + +/// Opens a fixture and hands back its video config plus the first keyframe's bytes. +struct IFrameFixture { + let demuxer: Demuxer + let config: MP4SegmentMuxer.VideoConfig + let keyframe: Data + + init(_ name: String, codecTag: String) throws { + let dem = Demuxer() + try dem.open(url: fixtureURL(name)) + let index = dem.videoStreamIndex + let stream = try #require(dem.stream(at: index)) + config = MP4SegmentMuxer.VideoConfig( + codecpar: UnsafePointer(stream.pointee.codecpar), + timeBase: stream.pointee.time_base, + codecTagOverride: codecTag) + var found: Data? + for _ in 0..<200 { + guard let pkt = try dem.readPacket() else { break } + var owned: UnsafeMutablePointer? = pkt + defer { av_packet_free(&owned) } + if pkt.pointee.stream_index == index, (pkt.pointee.flags & AV_PKT_FLAG_KEY) != 0, + let bytes = pkt.pointee.data { + found = Data(bytes: bytes, count: Int(pkt.pointee.size)); break + } + } + keyframe = try #require(found) + demuxer = dem + } +} + +@Suite("I-frame fragment builder", .serialized) +struct IFrameFragmentBuilderTests { + private func stagingDir() throws -> URL { + let dir = FileManager.default.temporaryDirectory + .appendingPathComponent("iframe-builder-\(UUID().uuidString)", isDirectory: true) + try FileManager.default.createDirectory(at: dir, withIntermediateDirectories: true) + return dir + } + + @Test("one keyframe becomes one fragment with one sample at the requested time", + .enabled(if: fixtureExists("sdr-h264.mp4"))) + func oneSampleAtRequestedTime() throws { + let fx = try IFrameFixture("sdr-h264.mp4", codecTag: "avc1") + defer { fx.demuxer.close() } + let dir = try stagingDir() + defer { try? FileManager.default.removeItem(at: dir) } + let builder = IFrameFragmentBuilder(video: fx.config, stagingDir: dir) + + let out = try #require(builder.build(payload: fx.keyframe, index: 7, + startSeconds: 28.0, durationSeconds: 4.0)) + #expect(IFrameTestBoxes.topLevelTypes(out.initSegment) == ["ftyp", "moov"]) + #expect(IFrameTestBoxes.topLevelTypes(out.fragment).contains("moof")) + #expect(IFrameTestBoxes.topLevelTypes(out.fragment).contains("mdat")) + #expect(IFrameTestBoxes.sampleCount(fragment: out.fragment) == 1) + let timescale = Double(try #require(IFrameTestBoxes.timescale(initSegment: out.initSegment))) + let tfdt = Double(try #require(IFrameTestBoxes.baseDecodeTime(fragment: out.fragment))) + #expect(abs(tfdt / timescale - 28.0) < 0.002) + } + + @Test("the init is byte-identical across fragments built out of order", + .enabled(if: fixtureExists("sdr-h264.mp4"))) + func initIsStable() throws { + let fx = try IFrameFixture("sdr-h264.mp4", codecTag: "avc1") + defer { fx.demuxer.close() } + let dir = try stagingDir() + defer { try? FileManager.default.removeItem(at: dir) } + let builder = IFrameFragmentBuilder(video: fx.config, stagingDir: dir) + + let late = try #require(builder.build(payload: fx.keyframe, index: 40, + startSeconds: 160.0, durationSeconds: 4.0)) + let early = try #require(builder.build(payload: fx.keyframe, index: 2, + startSeconds: 8.0, durationSeconds: 4.0)) + #expect(late.initSegment == early.initSegment) + let timescale = Double(try #require(IFrameTestBoxes.timescale(initSegment: early.initSegment))) + let tfdt = Double(try #require(IFrameTestBoxes.baseDecodeTime(fragment: early.fragment))) + #expect(abs(tfdt / timescale - 8.0) < 0.002) + } + + @Test("nothing is left behind in the staging directory", + .enabled(if: fixtureExists("sdr-h264.mp4"))) + func stagingIsCleaned() throws { + let fx = try IFrameFixture("sdr-h264.mp4", codecTag: "avc1") + defer { fx.demuxer.close() } + let dir = try stagingDir() + defer { try? FileManager.default.removeItem(at: dir) } + let builder = IFrameFragmentBuilder(video: fx.config, stagingDir: dir) + _ = builder.build(payload: fx.keyframe, index: 0, startSeconds: 0, durationSeconds: 4.0) + #expect(try FileManager.default.contentsOfDirectory(atPath: dir.path).isEmpty) + } + + @Test("an empty payload builds nothing", .enabled(if: fixtureExists("sdr-h264.mp4"))) + func emptyPayload() throws { + let fx = try IFrameFixture("sdr-h264.mp4", codecTag: "avc1") + defer { fx.demuxer.close() } + let dir = try stagingDir() + defer { try? FileManager.default.removeItem(at: dir) } + let builder = IFrameFragmentBuilder(video: fx.config, stagingDir: dir) + #expect(builder.build(payload: Data(), index: 0, startSeconds: 0, durationSeconds: 4.0) == nil) + } + + @Test("building a fragment is cheap enough to do per request", + .enabled(if: fixtureExists("sdr-h264.mp4"))) + func buildCost() throws { + let fx = try IFrameFixture("sdr-h264.mp4", codecTag: "avc1") + defer { fx.demuxer.close() } + let dir = try stagingDir() + defer { try? FileManager.default.removeItem(at: dir) } + let builder = IFrameFragmentBuilder(video: fx.config, stagingDir: dir) + let clock = ContinuousClock() + let elapsed = clock.measure { + for i in 0..<50 { + _ = builder.build(payload: fx.keyframe, index: i, + startSeconds: Double(i) * 4, durationSeconds: 4) + } + } + let perFragmentMs = Double(elapsed.components.attoseconds) / 1e15 / 50 + + Double(elapsed.components.seconds) * 1000 / 50 + print("IFrameFragmentBuilder: \(String(format: "%.2f", perFragmentMs)) ms per fragment") + #expect(perFragmentMs < 25) + } +} diff --git a/Tests/AetherEngineTests/IFramePayloadCacheTests.swift b/Tests/AetherEngineTests/IFramePayloadCacheTests.swift new file mode 100644 index 000000000..62bed54ad --- /dev/null +++ b/Tests/AetherEngineTests/IFramePayloadCacheTests.swift @@ -0,0 +1,82 @@ +// Tests/AetherEngineTests/IFramePayloadCacheTests.swift +import Foundation +import Testing +@testable import AetherEngine + +struct IFramePayloadCacheTests { + private func makeCache(limit: Int) throws -> (IFramePayloadCache, URL) { + let dir = FileManager.default.temporaryDirectory + .appendingPathComponent("iframe-cache-\(UUID().uuidString)", isDirectory: true) + return (IFramePayloadCache(directory: dir, byteLimit: limit), dir) + } + + @Test("a stored payload comes back byte for byte") + func roundTrip() throws { + let (cache, dir) = try makeCache(limit: 1000) + defer { try? FileManager.default.removeItem(at: dir) } + cache.store(Data([1, 2, 3]), for: 5) + #expect(cache.payload(for: 5) == Data([1, 2, 3])) + #expect(cache.payload(for: 6) == nil) + } + + @Test("the least recently used payload goes first once the cap is passed") + func evictsLeastRecentlyUsed() throws { + let (cache, dir) = try makeCache(limit: 250) + defer { try? FileManager.default.removeItem(at: dir) } + cache.store(Data(count: 100), for: 0) + cache.store(Data(count: 100), for: 1) + _ = cache.payload(for: 0) + cache.store(Data(count: 100), for: 2) + #expect(cache.payload(for: 1) == nil) + #expect(cache.payload(for: 0) != nil) + #expect(cache.payload(for: 2) != nil) + } + + @Test("a payload larger than the cap is not kept and evicts nothing; one exactly at the cap is kept") + func oversizedPayloadIsNotKept() throws { + let (cache, dir) = try makeCache(limit: 100) + defer { try? FileManager.default.removeItem(at: dir) } + cache.store(Data(count: 60), for: 0) + cache.store(Data(count: 500), for: 1) + #expect(cache.payload(for: 1) == nil) + #expect(cache.payload(for: 0)?.count == 60) + #expect(cache.nearestIndex(to: 1) == 0) + let files = try FileManager.default.contentsOfDirectory(atPath: dir.path) + #expect(files == ["payload-0.bin"], "the oversized payload left a file over the cap") + + cache.store(Data(count: 100), for: 2) + #expect(cache.payload(for: 2)?.count == 100) + #expect(cache.payload(for: 0) == nil) + } + + @Test("nearest prefers the closest index and the lower one on a tie") + func nearest() throws { + let (cache, dir) = try makeCache(limit: 1000) + defer { try? FileManager.default.removeItem(at: dir) } + #expect(cache.nearestIndex(to: 10) == nil) + cache.store(Data([1]), for: 4) + cache.store(Data([1]), for: 20) + #expect(cache.nearestIndex(to: 10) == 4) + #expect(cache.nearestIndex(to: 13) == 20) + #expect(cache.nearestIndex(to: 12) == 4) + } + + @Test("removeAll forgets everything and deletes the directory") + func removeAll() throws { + let (cache, dir) = try makeCache(limit: 1000) + cache.store(Data([1]), for: 0) + cache.removeAll() + #expect(cache.payload(for: 0) == nil) + #expect(!FileManager.default.fileExists(atPath: dir.path)) + } + + @Test("a payload whose file vanished is a miss, not a crash") + func missingFileIsAMiss() throws { + let (cache, dir) = try makeCache(limit: 1000) + defer { try? FileManager.default.removeItem(at: dir) } + cache.store(Data([1]), for: 3) + try FileManager.default.removeItem(at: dir) + #expect(cache.payload(for: 3) == nil) + #expect(cache.nearestIndex(to: 3) == nil) + } +} diff --git a/Tests/AetherEngineTests/IFramePlaylistTextTests.swift b/Tests/AetherEngineTests/IFramePlaylistTextTests.swift new file mode 100644 index 000000000..27d320a1d --- /dev/null +++ b/Tests/AetherEngineTests/IFramePlaylistTextTests.swift @@ -0,0 +1,119 @@ +// Tests/AetherEngineTests/IFramePlaylistTextTests.swift +import Testing +import Foundation +@testable import AetherEngine + +private final class IFrameMockProvider: HLSSegmentProvider, @unchecked Sendable { + let served: Bool + let durations: [Double] + let supplemental: String? + init(served: Bool, durations: [Double] = [4.0, 4.0, 2.5], supplemental: String? = nil) { + self.served = served; self.durations = durations; self.supplemental = supplemental + } + func initSegment() -> Data? { Data([0x00]) } + func mediaSegment(at index: Int) -> Data? { Data([0x00]) } + var segmentCount: Int { durations.count } + func segmentDuration(at index: Int) -> Double { durations[index] } + var playlistType: HLSPlaylistType { .vod } + var masterCodecs: String? { "hvc1.2.4.L150.B0,ec-3" } + var masterSupplementalCodecs: String? { supplemental } + var masterResolution: (width: Int, height: Int)? { (3840, 2160) } + var masterVideoRange: HLSVideoRange? { .pq } + var masterBandwidth: Int? { 12_000_000 } + var masterFrameRate: Double? { 23.976 } + var iFrameRenditionServed: Bool { served } +} + +struct IFramePlaylistTextTests { + @Test("master carries one I-frame line, after the variant, with the video codec only") + func masterLine() { + let m = HLSLocalServer.buildMasterPlaylistText(provider: IFrameMockProvider(served: true)) + let lines = m.split(separator: "\n").map(String.init) + let tag = lines.filter { $0.hasPrefix("#EXT-X-I-FRAME-STREAM-INF:") } + #expect(tag.count == 1) + #expect(tag[0] == "#EXT-X-I-FRAME-STREAM-INF:BANDWIDTH=12000000,CODECS=\"hvc1.2.4.L150.B0\"," + + "RESOLUTION=3840x2160,VIDEO-RANGE=PQ,URI=\"iframe.m3u8\"") + let variantURI = try! #require(lines.firstIndex(of: "media.m3u8")) + #expect(lines.firstIndex(of: tag[0])! > variantURI) + } + + @Test("master is untouched when the rendition is not served") + func masterWithout() { + let m = HLSLocalServer.buildMasterPlaylistText(provider: IFrameMockProvider(served: false)) + #expect(!m.contains("I-FRAME")) + #expect(m.hasSuffix("media.m3u8\n")) + } + + @Test("SUPPLEMENTAL-CODECS rides on the primary master and is dropped on the reduced one") + func supplementalFollowsVariant() { + let p = IFrameMockProvider(served: true, supplemental: "dvh1.08.06/db1p") + let primary = HLSLocalServer.buildMasterPlaylistText(provider: p) + let reduced = HLSLocalServer.buildMasterPlaylistText(provider: p, variant: .reducedHDR) + let tag: (String) -> String = { text in + text.split(separator: "\n").map(String.init).first { $0.hasPrefix("#EXT-X-I-FRAME-STREAM-INF:") }! + } + #expect(tag(primary).contains("SUPPLEMENTAL-CODECS=\"dvh1.08.06/db1p\"")) + #expect(!tag(reduced).contains("SUPPLEMENTAL-CODECS")) + #expect(tag(reduced).contains("URI=\"iframe.m3u8\"")) + } + + @Test("I-frame playlist mirrors the plan one entry per segment") + func playlist() { + let p = IFrameMockProvider(served: true) + let text = HLSLocalServer.buildIFramePlaylistText(provider: p) + #expect(text == """ + #EXTM3U + #EXT-X-VERSION:7 + #EXT-X-I-FRAMES-ONLY + #EXT-X-TARGETDURATION:4 + #EXT-X-MEDIA-SEQUENCE:0 + #EXT-X-PLAYLIST-TYPE:VOD + #EXT-X-MAP:URI="iframe_init.mp4" + #EXTINF:4.000, + iframe0.mp4 + #EXTINF:4.000, + iframe1.mp4 + #EXTINF:2.500, + iframe2.mp4 + #EXT-X-ENDLIST + + """) + } + + @Test("I-frame playlist TARGETDURATION equals the media playlist's") + func targetDurationMatchesMedia() { + let p = IFrameMockProvider(served: true, durations: [4.0, 9.6, 4.0]) + let media = HLSLocalServer.buildMediaPlaylistText(provider: p) + let iframe = HLSLocalServer.buildIFramePlaylistText(provider: p) + let td: (String) -> String? = { t in + t.split(separator: "\n").map(String.init).first { $0.hasPrefix("#EXT-X-TARGETDURATION:") } + } + #expect(td(media) != nil) + #expect(td(media) == td(iframe)) + } + + @Test("absolute sub-resource base is honoured like in the media playlist") + func absoluteBase() { + let p = IFrameMockProvider(served: true, durations: [4.0]) + let text = HLSLocalServer.buildIFramePlaylistText( + provider: p, subResourceBaseURL: URL(string: "aether-hls://session/")!) + #expect(text.contains("#EXT-X-MAP:URI=\"aether-hls://session/iframe_init.mp4\"")) + #expect(text.contains("\naether-hls://session/iframe0.mp4\n")) + } + + @Test("I-frame segment paths parse to their index, and nothing else does", + arguments: [("/iframe0.mp4", 0), ("/iframe17.mp4", 17), ("/iframe_init.mp4", nil), + ("/iframe-1.mp4", nil), ("/iframeX.mp4", nil), ("/iframe.mp4", nil), + ("/iframe3.m4s", nil), ("/seg3.mp4", nil)] as [(String, Int?)]) + func parsePath(path: String, expected: Int?) { + #expect(HLSLocalServer.parseIFramePath(path) == expected) + } + + @Test("video codec extraction keeps the video entry and falls back to the whole string", + arguments: [("hvc1.2.4.L150.B0,ec-3", "hvc1.2.4.L150.B0"), ("mp4a.40.2,avc1.640028", "avc1.640028"), + ("dvh1.05.06,ec-3", "dvh1.05.06"), ("av01.0.12M.10", "av01.0.12M.10"), + ("xyz1.1", "xyz1.1")]) + func videoCodecs(codecs: String, expected: String) { + #expect(HLSLocalServer.videoCodecs(of: codecs) == expected) + } +} diff --git a/Tests/AetherEngineTests/IFrameRenditionEligibilityTests.swift b/Tests/AetherEngineTests/IFrameRenditionEligibilityTests.swift new file mode 100644 index 000000000..61c1457b4 --- /dev/null +++ b/Tests/AetherEngineTests/IFrameRenditionEligibilityTests.swift @@ -0,0 +1,56 @@ +// Tests/AetherEngineTests/IFrameRenditionEligibilityTests.swift +import Testing +@testable import AetherEngine + +struct IFrameRenditionEligibilityTests { + private func inputs(_ change: (inout IFrameRenditionEligibility.Inputs) -> Void = { _ in }) + -> IFrameRenditionEligibility.Inputs { + var i = IFrameRenditionEligibility.Inputs( + requested: true, isLive: false, planBoundariesClaimRandomAccess: true, + sequentialOrigin: false, heldSourceConnection: false, originIsSerial: false, + isDiscSource: false, secondReaderAvailable: true) + change(&i) + return i + } + + @Test("every requirement met is a candidate") + func allMet() { + #expect(IFrameRenditionEligibility.candidate(inputs()) == .served) + } + + @Test("each failed requirement names its own reason", + arguments: ["notRequested", "live", "planNotKeyframeAligned", "sequentialOrigin", + "heldSourceConnection", "serialOrigin", "discSource", "noSecondReader"]) + func eachReason(reason: String) throws { + let expected = try #require(IFrameRenditionEligibility.AbsentReason(rawValue: reason)) + let verdict = IFrameRenditionEligibility.candidate(inputs { i in + switch expected { + case .notRequested: i.requested = false + case .live: i.isLive = true + case .planNotKeyframeAligned: i.planBoundariesClaimRandomAccess = false + case .sequentialOrigin: i.sequentialOrigin = true + case .heldSourceConnection: i.heldSourceConnection = true + case .serialOrigin: i.originIsSerial = true + case .discSource: i.isDiscSource = true + case .noSecondReader: i.secondReaderAvailable = false + case .mediaPlaylistRouting: break + } + }) + #expect(verdict == .absent(expected)) + } + + @Test("not requested wins over every other reason, so an unflagged session logs nothing odd") + func notRequestedFirst() { + let v = IFrameRenditionEligibility.candidate(inputs { $0.requested = false; $0.isLive = true }) + #expect(v == .absent(.notRequested)) + } + + @Test("a candidate without a master is absent for routing") + func mediaRouting() { + #expect(IFrameRenditionEligibility.resolve(candidate: .served, servingMaster: false) + == .absent(.mediaPlaylistRouting)) + #expect(IFrameRenditionEligibility.resolve(candidate: .served, servingMaster: true) == .served) + #expect(IFrameRenditionEligibility.resolve(candidate: .absent(.live), servingMaster: true) + == .absent(.live)) + } +} diff --git a/Tests/AetherEngineTests/IFrameRenditionTests.swift b/Tests/AetherEngineTests/IFrameRenditionTests.swift new file mode 100644 index 000000000..127218d11 --- /dev/null +++ b/Tests/AetherEngineTests/IFrameRenditionTests.swift @@ -0,0 +1,214 @@ +// Tests/AetherEngineTests/IFrameRenditionTests.swift +import Foundation +import Testing +@testable import AetherEngine + +private final class Recorder: @unchecked Sendable { + private let lock = NSLock() + private var _reads: [Int64] = [] + private var _closed = 0, _interrupted = 0, _waits = 0 + var failing: Set = [] + var readsAllowed = true + /// Runs inside a read, before it answers. Lets a test park a read or stop the rendition under it. + var duringRead: (@Sendable (IFrameRendition.Entry) -> Void)? + func read(_ e: IFrameRendition.Entry) -> Data? { + lock.lock(); _reads.append(e.startPts); lock.unlock() + duringRead?(e) + return failing.contains(e.startPts) ? nil : Data("key\(e.startPts)".utf8) + } + func noteClose() { lock.lock(); _closed += 1; lock.unlock() } + /// Signalled when shutdown has set its flag and is asking the reader to abort. + let interruptSeen = DispatchSemaphore(value: 0) + func noteInterrupt() { lock.lock(); _interrupted += 1; lock.unlock(); interruptSeen.signal() } + func noteWait() { lock.lock(); _waits += 1; lock.unlock() } + var reads: [Int64] { lock.lock(); defer { lock.unlock() }; return _reads } + var closed: Int { lock.lock(); defer { lock.unlock() }; return _closed } + var interrupted: Int { lock.lock(); defer { lock.unlock() }; return _interrupted } + var waits: Int { lock.lock(); defer { lock.unlock() }; return _waits } +} + +struct IFrameRenditionTests { + private func make(_ rec: Recorder, count: Int = 5, cacheLimit: Int = 1 << 20) -> (IFrameRendition, URL) { + let dir = FileManager.default.temporaryDirectory + .appendingPathComponent("iframe-rendition-\(UUID().uuidString)", isDirectory: true) + let entries = (0.. URL { + URL(fileURLWithPath: #filePath) + .deletingLastPathComponent().deletingLastPathComponent().deletingLastPathComponent() + .appendingPathComponent("Fixtures/\(name)") +} +private func fixtureExists(_ name: String) -> Bool { + FileManager.default.fileExists(atPath: fixtureURL(name).path) +} + +/// A file-backed custom reader that counts its closes: the stand-in for a host's clone. +private final class CountingFileReader: IOReader, @unchecked Sendable { + let discImageProbeEnabled = false + private let lock = NSLock() + private let handle: FileHandle + private let size: Int64 + private var offset: Int64 = 0 + private var _closes = 0 + init(url: URL) throws { + handle = try FileHandle(forReadingFrom: url) + size = Int64((try FileManager.default.attributesOfItem(atPath: url.path)[.size] as? NSNumber)?.int64Value ?? 0) + } + var closes: Int { lock.withLock { _closes } } + func read(_ buffer: UnsafeMutablePointer?, size count: Int32) -> Int32 { + lock.withLock { + guard let buffer, count > 0, offset < size else { return 0 } + try? handle.seek(toOffset: UInt64(offset)) + let data = handle.readData(ofLength: Int(min(Int64(count), size - offset))) + data.copyBytes(to: buffer, count: data.count) + offset += Int64(data.count) + return Int32(data.count) + } + } + func seek(offset requested: Int64, whence: Int32) -> Int64 { + lock.withLock { + if whence & 0x10000 != 0 { return size } + switch whence & ~0x20000 { + case SEEK_SET: offset = requested + case SEEK_CUR: offset += requested + case SEEK_END: offset = size + requested + default: return -1 + } + return offset + } + } + func close() { lock.withLock { _closes += 1 } } +} + +private func waitUntil(_ timeout: TimeInterval = 5, _ condition: () -> Bool) -> Bool { + let deadline = Date().addingTimeInterval(timeout) + while Date() < deadline { if condition() { return true }; Thread.sleep(forTimeInterval: 0.02) } + return condition() +} + +@Suite("I-frame rendition on a real session", .serialized) +struct IFrameSessionTests { + private static let fixture = "restart-witness-av.mp4" + + @Test("a flagged VOD session lists the rendition and serves keyframes on the plan's timeline", + .enabled(if: fixtureExists(fixture)), .timeLimit(.minutes(2))) + func servesRendition() throws { + let engine = HLSVideoEngine(url: fixtureURL(Self.fixture), dvModeAvailable: false) + engine.requestIFramePlaylist() + let playbackURL = try engine.start() + defer { engine.stop() } + try #require(engine.planBoundariesClaimRandomAccess, "fixture must plan on its keyframe index") + #expect(engine.iFrameRenditionVerdict == .served) + #expect(playbackURL.lastPathComponent == "master.m3u8") + let prov = try #require(engine.provider) + #expect(prov.iFrameRenditionServed) + try #require(prov.segmentCount >= 2) + + let initSegment = try #require(prov.iFrameInitSegment()) + let timescale = Double(try #require(IFrameTestBoxes.timescale(initSegment: initSegment))) + // Deliberately out of order: the last entry first. + for index in [prov.segmentCount - 1, 0, 1] { + let fragment = try #require(prov.iFrameSegment(at: index)) + #expect(IFrameTestBoxes.sampleCount(fragment: fragment) == 1) + let tfdt = Double(try #require(IFrameTestBoxes.baseDecodeTime(fragment: fragment))) + #expect(abs(tfdt / timescale - engine.segmentPlan[index].startSeconds) < 0.002, + "iframe\(index) sits at \(tfdt / timescale), plan says \(engine.segmentPlan[index].startSeconds)") + } + #expect(prov.iFrameSegment(at: prov.segmentCount) == nil) + } + + @Test("the loopback server answers the three new paths", + .enabled(if: fixtureExists(fixture)), .timeLimit(.minutes(2))) + func routes() throws { + let engine = HLSVideoEngine(url: fixtureURL(Self.fixture), dvModeAvailable: false) + engine.requestIFramePlaylist() + let playbackURL = try engine.start() + defer { engine.stop() } + let base = playbackURL.deletingLastPathComponent() + let master = try String(contentsOf: playbackURL, encoding: .utf8) + #expect(master.contains("#EXT-X-I-FRAME-STREAM-INF:")) + #expect(master.contains("URI=\"iframe.m3u8\"")) + let playlist = try String(contentsOf: base.appendingPathComponent("iframe.m3u8"), encoding: .utf8) + #expect(playlist.contains("#EXT-X-I-FRAMES-ONLY")) + #expect(playlist.contains("iframe1.mp4")) + let initSegment = try Data(contentsOf: base.appendingPathComponent("iframe_init.mp4")) + #expect(IFrameTestBoxes.topLevelTypes(initSegment) == ["ftyp", "moov"]) + let fragment = try Data(contentsOf: base.appendingPathComponent("iframe1.mp4")) + #expect(IFrameTestBoxes.sampleCount(fragment: fragment) == 1) + #expect((try? Data(contentsOf: base.appendingPathComponent("iframe99999.mp4"))) == nil) + } + + @Test("without the request the session is what it was: no tag, no route, no reader", + .enabled(if: fixtureExists(fixture)), .timeLimit(.minutes(2))) + func flagOffLeavesMasterUntouched() throws { + let engine = HLSVideoEngine(url: fixtureURL(Self.fixture), dvModeAvailable: false) + _ = try engine.start() + defer { engine.stop() } + #expect(engine.iFrameRenditionVerdict == .absent(.notRequested)) + let prov = try #require(engine.provider) + #expect(!prov.iFrameRenditionServed) + #expect(prov.iFrameInitSegment() == nil) + #expect(prov.iFrameSegment(at: 0) == nil) + #expect(!HLSLocalServer.buildMasterPlaylistText(provider: prov).contains("I-FRAME")) + } + + @Test("the fallback to the media playlist takes the rendition down with it", + .enabled(if: fixtureExists(fixture)), .timeLimit(.minutes(2))) + func fallbackDropsRendition() throws { + let engine = HLSVideoEngine(url: fixtureURL(Self.fixture), dvModeAvailable: false) + engine.requestIFramePlaylist() + _ = try engine.start() + defer { engine.stop() } + let prov = try #require(engine.provider) + #expect(prov.iFrameSegment(at: 0) != nil) + engine.markServingMediaAfterFallback() + #expect(!prov.iFrameRenditionServed) + #expect(prov.iFrameSegment(at: 0) == nil) + } + + @Test("stopping while a keyframe request is in flight neither crashes nor hangs", + .enabled(if: fixtureExists(fixture)), .timeLimit(.minutes(2))) + func stopWithRenditionInFlightDoesNotCrash() async throws { + let engine = HLSVideoEngine(url: fixtureURL(Self.fixture), dvModeAvailable: false) + engine.requestIFramePlaylist() + _ = try engine.start() + let prov = try #require(engine.provider) + let count = prov.segmentCount + await withTaskGroup(of: Void.self) { group in + for round in 0..<20 { + group.addTask { _ = prov.iFrameSegment(at: round % count) } + } + group.addTask { engine.stop() } + } + #expect(prov.iFrameSegment(at: 0) == nil) + } + + @Test("a live session is never a candidate, whatever else is true") + func liveSessionServesNoRendition() { + let verdict = IFrameRenditionEligibility.candidate(.init( + requested: true, isLive: true, planBoundariesClaimRandomAccess: true, + sequentialOrigin: false, heldSourceConnection: false, originIsSerial: false, + isDiscSource: false, secondReaderAvailable: true)) + #expect(verdict == .absent(.live)) + } + + @Test("the load option defaults to off and is a tuning field, not a session identity") + func loadOptionDefault() { + #expect(LoadOptions().serveIFramePlaylist == false) + var on = LoadOptions() + on.serveIFramePlaylist = true + #expect(LoadOptions(serveIFramePlaylist: true) == on) + #expect(SessionOptionCorrection.refusedFields(from: LoadOptions(), to: on).isEmpty) + #expect(SessionOptionCorrection.knownFields.contains("serveIFramePlaylist")) + } + + @Test("a custom source's clone feeds the side reader and is closed exactly once at stop", + .enabled(if: fixtureExists(fixture)), .timeLimit(.minutes(2))) + func customCloneIsClosedAtStop() throws { + let clone = try CountingFileReader(url: fixtureURL(Self.fixture)) + let engine = HLSVideoEngine(url: fixtureURL(Self.fixture), dvModeAvailable: false) + engine.requestIFramePlaylist() + engine.customIFrameReader = (reader: clone, formatHint: "mp4") + _ = try engine.start() + let prov = try #require(engine.provider) + let fragment = try #require(prov.iFrameSegment(at: 1)) + #expect(IFrameTestBoxes.sampleCount(fragment: fragment) == 1) + #expect(clone.closes == 0) + engine.stop() + #expect(waitUntil { clone.closes == 1 }) + Thread.sleep(forTimeInterval: 0.2) + #expect(clone.closes == 1) + } + + @Test("a clone nobody ever read from is still closed at stop", + .enabled(if: fixtureExists(fixture)), .timeLimit(.minutes(2))) + func untouchedCloneIsClosedAtStop() throws { + let clone = try CountingFileReader(url: fixtureURL(Self.fixture)) + let engine = HLSVideoEngine(url: fixtureURL(Self.fixture), dvModeAvailable: false) + engine.requestIFramePlaylist() + engine.customIFrameReader = (reader: clone, formatHint: "mp4") + _ = try engine.start() + engine.stop() + #expect(waitUntil { clone.closes == 1 }) + } + + @Test("a clone handed to a session that cannot serve the rendition is closed at once", + .enabled(if: fixtureExists("sdr-h264.mp4")), .timeLimit(.minutes(2))) + func cloneOfAnAbsentRenditionIsClosed() throws { + // One IRAP in the whole file: no keyframe-aligned plan, so the rendition stays absent. + let clone = try CountingFileReader(url: fixtureURL("sdr-h264.mp4")) + let engine = HLSVideoEngine(url: fixtureURL("sdr-h264.mp4"), dvModeAvailable: false) + engine.requestIFramePlaylist() + engine.customIFrameReader = (reader: clone, formatHint: "mp4") + _ = try engine.start() + defer { engine.stop() } + #expect(engine.iFrameRenditionVerdict == .absent(.planNotKeyframeAligned)) + #expect(clone.closes == 1) + } +} diff --git a/Tests/AetherEngineTests/IFrameSideReaderTests.swift b/Tests/AetherEngineTests/IFrameSideReaderTests.swift new file mode 100644 index 000000000..5df349ad0 --- /dev/null +++ b/Tests/AetherEngineTests/IFrameSideReaderTests.swift @@ -0,0 +1,155 @@ +// Tests/AetherEngineTests/IFrameSideReaderTests.swift +import Foundation +import Testing +import AetherLibavcodec +@testable import AetherEngine + +private func fixtureURL(_ name: String) -> URL { + URL(fileURLWithPath: #filePath) + .deletingLastPathComponent().deletingLastPathComponent().deletingLastPathComponent() + .appendingPathComponent("Fixtures/\(name)") +} +private func fixtureExists(_ name: String) -> Bool { + FileManager.default.fileExists(atPath: fixtureURL(name).path) +} + +/// Every keyframe of the fixture's video stream as (pts-or-dts index timestamp, bytes), read +/// linearly on an independent demuxer: the reference the side reader is checked against. +private func referenceKeyframes(_ name: String) throws -> (index: [Int64], packets: [Int64: Data]) { + let dem = Demuxer() + try dem.open(url: fixtureURL(name)) + defer { dem.close() } + let video = dem.videoStreamIndex + var packets: [Int64: Data] = [:] + while let pkt = try dem.readPacket() { + var owned: UnsafeMutablePointer? = pkt + defer { av_packet_free(&owned) } + guard pkt.pointee.stream_index == video, (pkt.pointee.flags & AV_PKT_FLAG_KEY) != 0, + let bytes = pkt.pointee.data else { continue } + let data = Data(bytes: bytes, count: Int(pkt.pointee.size)) + packets[pkt.pointee.dts] = data + packets[pkt.pointee.pts] = data + } + return (dem.indexedKeyframes(streamIndex: video).sorted(), packets) +} + +@Suite("I-frame side reader", .serialized) +struct IFrameSideReaderTests { + private func reader(_ name: String) -> IFrameSideReader { + IFrameSideReader(open: { dem in + try dem.open(url: fixtureURL(name), profile: .iFrameSideDemuxer) + }) + } + + /// mp4 indexes its keyframes by DTS and Matroska by PTS (`PlanBoundaryAxis`), so both containers + /// are read: the side reader must return the segment's own keyframe on either ladder. + @Test("the payload at an indexed keyframe is that keyframe, in any request order", + arguments: ["restart-witness-av.mp4", "restart-witness-subs.mkv"]) + func readsTheIndexedKeyframe(fixture: String) throws { + guard fixtureExists(fixture) else { return } + let ref = try referenceKeyframes(fixture) + try #require(ref.index.count >= 2, "\(fixture) needs at least two indexed keyframes") + let r = reader(fixture) + defer { r.interrupt(); r.close() } + for ts in ref.index.prefix(3).reversed() { + let payload = try #require(r.payload(startPts: ts)) + #expect(payload == ref.packets[ts], "\(fixture): keyframe at \(ts) differs") + } + } + + @Test("a failed open is retried only after its cool-down, which grows with the second failure") + func failedOpenIsRetriedAfterItsCoolDown() { + final class Clock: @unchecked Sendable { var now = Date(timeIntervalSince1970: 1_000); var opens = 0 } + let clock = Clock() + let r = IFrameSideReader(open: { _ in + clock.opens += 1 + throw NSError(domain: "test", code: 1) + }, now: { clock.now }) + defer { r.interrupt(); r.close() } + #expect(r.payload(startPts: 0) == nil) + #expect(r.payload(startPts: 100) == nil) + #expect(clock.opens == 1, "five parallel requests must not each pay for an open") + clock.now += 6 + #expect(r.payload(startPts: 0) == nil) + #expect(clock.opens == 2) + clock.now += 6 + #expect(r.payload(startPts: 0) == nil) + #expect(clock.opens == 2, "the second failure in a row holds for 30 s") + clock.now += 30 + #expect(r.payload(startPts: 0) == nil) + #expect(clock.opens == 3) + } + + @Test("a failed read cools down, then reopens a fresh demuxer and recovers", + .enabled(if: fixtureExists("restart-witness-av.mp4"))) + func failedReadReopens() throws { + final class Clock: @unchecked Sendable { var now = Date(timeIntervalSince1970: 1_000); var opens = 0 } + let clock = Clock() + let ref = try referenceKeyframes("restart-witness-av.mp4") + let r = IFrameSideReader(open: { dem in + clock.opens += 1 + try dem.open(url: fixtureURL("restart-witness-av.mp4"), profile: .iFrameSideDemuxer) + }, now: { clock.now }) + defer { r.interrupt(); r.close() } + #expect(r.payload(startPts: ref.index[0]) != nil) + // Far past the end: no keyframe to land on, which is what a dead connection looks like here. + #expect(r.payload(startPts: Int64.max / 4) == nil) + #expect(r.payload(startPts: ref.index[0]) == nil, "inside the cool-down the source is left alone") + #expect(clock.opens == 1) + clock.now += 6 + #expect(r.payload(startPts: ref.index[0]) == ref.packets[ref.index[0]]) + #expect(clock.opens == 2) + } + + @Test("cleanup runs exactly once on close, opened or not") + func cleanupRunsOnce() { + final class Count: @unchecked Sendable { var n = 0 } + let untouched = Count() + let never = IFrameSideReader(open: { _ in }, cleanup: { untouched.n += 1 }) + never.interrupt(); never.close(); never.close() + #expect(untouched.n == 1) + } + + @Test("after interrupt every call answers nil", + .enabled(if: fixtureExists("restart-witness-av.mp4"))) + func interruptEndsIt() throws { + let ref = try referenceKeyframes("restart-witness-av.mp4") + let r = reader("restart-witness-av.mp4") + #expect(r.payload(startPts: ref.index[0]) != nil) + r.interrupt() + #expect(r.payload(startPts: ref.index[0]) == nil) + r.close() + } + + @Test("the I-frame side profile reads like the still extractor under its own log name") + func profile() { + let p = DemuxerOpenProfile.iFrameSideDemuxer + #expect(p.readerLabel == "iframe") + #expect(p.avioPrefetch == DemuxerOpenProfile.stillExtraction.avioPrefetch) + #expect(p.probesize == DemuxerOpenProfile.stillExtraction.probesize) + #expect(DemuxerOpenProfile.labelsWithoutCompositionRepair.contains("iframe")) + #expect(DemuxerOpenProfile.labelsWithoutCompositionRepair.contains("extract")) + } + + @Test("interrupt aborts an open that is still in flight", .timeLimit(.minutes(1))) + func interruptAbortsAnOpenInFlight() { + let parked = ProbeParkedReader(operation: .read) + parked.mayReturn.open() + let entered = DispatchSemaphore(value: 0) + let r = IFrameSideReader(open: { dem in + entered.signal() + try dem.open(reader: parked, formatHint: "mp4", profile: .iFrameSideDemuxer) + }) + let done = DispatchSemaphore(value: 0) + // A thread of its own: a loaded runner starves the global pool for seconds. + Thread.detachNewThread { _ = r.payload(startPts: 0); done.signal() } + entered.wait() + Thread.sleep(forTimeInterval: 0.2) + r.interrupt() + let returned = done.wait(timeout: .now() + 10) == .success + #expect(returned, "payload() was still inside the open ten seconds after interrupt()") + parked.release() + if !returned { done.wait() } + r.close() + } +} diff --git a/Tests/AetherEngineTests/ISO9660ReaderTests.swift b/Tests/AetherEngineTests/ISO9660ReaderTests.swift index 1ad61d479..1aa417284 100644 --- a/Tests/AetherEngineTests/ISO9660ReaderTests.swift +++ b/Tests/AetherEngineTests/ISO9660ReaderTests.swift @@ -32,6 +32,26 @@ final class ISO9660ReaderTests: XCTestCase { } } + // audit NET-5: an unchecked 32-bit directory length let a crafted root record zero-fill up to + // 4 GB (jetsam on tvOS/iOS) and then trap converting `count - got` to Int32. Must throw a + // normal error instead, and never attempt the allocation. + func test_hostileRootLengthIsRejectedNotAllocated() throws { + var pvd = [UInt8](repeating: 0, count: ISO9660Fixture.sectorSize) + pvd[0] = 1 + for (i, b) in Array("CD001".utf8).enumerated() { pvd[1 + i] = b } + pvd[6] = 1 + for (i, b) in ISO9660Fixture.both16(ISO9660Fixture.sectorSize).enumerated() { pvd[128 + i] = b } + let rootRec = ISO9660Fixture.dirRecord(name: [0x00], isDir: true, lba: 17, length: 0xFFFF_FFFF) + for (i, b) in rootRec.enumerated() { pvd[156 + i] = b } + var image = [UInt8](repeating: 0, count: 18 * ISO9660Fixture.sectorSize) + for (i, b) in pvd.enumerated() { image[16 * ISO9660Fixture.sectorSize + i] = b } + + let iso = try ISO9660Reader(reader: DataIOReader(data: Data(image))) + XCTAssertThrowsError(try iso.list(directory: "VIDEO_TS")) { err in + guard case DiscError.malformed = err else { return XCTFail("wrong error: \(err)") } + } + } + func test_truncatedImageThrowsMalformedOrNotISO() throws { // PVD present but root extent cut off: must throw, not trap. let full = ISO9660Fixture.make(files: [.init(name: "VTS_01_1.VOB", length: 100)]) diff --git a/Tests/AetherEngineTests/InPlaceSwapRollTests.swift b/Tests/AetherEngineTests/InPlaceSwapRollTests.swift new file mode 100644 index 000000000..b639b46f2 --- /dev/null +++ b/Tests/AetherEngineTests/InPlaceSwapRollTests.swift @@ -0,0 +1,37 @@ +import Foundation +import Testing +@testable import AetherEngine + +/// Audit NAT-103: an in-place item swap (AE#629) reuses a player that is still `.playing`, and that +/// status reaches the fresh item about 1 ms after the load, long before it is ready. The swap's clock +/// hold and #50's `hasEverPlayed` both took it for the item rolling: the hold let AVPlayer's +/// pre-landing readings through to the published clock (15.97 s, then 12.00 s, then up again on the +/// AE#629 harness), and a swapped item that failed at startup took the post-playback branch. +@Suite("An in-place swap waits for the fresh item's own roll") +struct InPlaceSwapRollTests { + + @Test("A playing status carried onto an item that is not ready is not its roll") + func carriedPlayingIsNotARoll() { + #expect(!NativeAVPlayerHost.playingIsThisItemsRoll(itemIsReadyToPlay: false)) + #expect(NativeAVPlayerHost.playingIsThisItemsRoll(itemIsReadyToPlay: true)) + } + + /// A host-level test cannot put AVFoundation's carried edge on a fresh item, so this one reads the + /// sink, as the #98 placement test does. + @Test("The status sink releases the hold and latches has-played only on the item's roll") + func statusSinkGatesOnTheRoll() throws { + let source = URL(fileURLWithPath: #filePath) + .deletingLastPathComponent() + .deletingLastPathComponent() + .deletingLastPathComponent() + .appendingPathComponent("Sources/AetherEngine/Native/NativeAVPlayerHost.swift") + let text = try #require(try? String(contentsOf: source, encoding: .utf8)) + let sink = try #require(text.range(of: "timeControlObservation = avPlayer.observe(\\.timeControlStatus")) + let body = String(text[sink.upperBound...].prefix(3000)) + let gate = try #require(body.range(of: "Self.playingIsThisItemsRoll(itemIsReadyToPlay:")) + let latch = try #require(body.range(of: "self.hasEverPlayed = true")) + let release = try #require(body.range(of: "self.mountSeekPending = false")) + #expect(gate.lowerBound < latch.lowerBound) + #expect(gate.lowerBound < release.lowerBound) + } +} diff --git a/Tests/AetherEngineTests/InjectedASSRenditionSuppressionTests.swift b/Tests/AetherEngineTests/InjectedASSRenditionSuppressionTests.swift new file mode 100644 index 000000000..33be97ad4 --- /dev/null +++ b/Tests/AetherEngineTests/InjectedASSRenditionSuppressionTests.swift @@ -0,0 +1,196 @@ +// `PythonOrigin` needs `Process`, which exists only on macOS. +#if os(macOS) + +import Testing +import Foundation +import AVFoundation +@testable import AetherEngine + +/// AE#616 measures the bypass clock off the injected rendition AVPlayer presents, and only while one +/// is selected. A styled ASS track the host draws itself used to deselect its plain rendition, which +/// left `sourceTime` on item time for exactly the host that draws from it. The rendition now stays +/// selected behind a suppressing legible output. These run against a real `AVPlayerItem`, because +/// whether AVPlayer draws a rendition is decided by the item's outputs and media selection, not by +/// anything the engine records about them. +@Suite("Overlay-drawn injected ASS keeps its rendition selected (AE#616)", .timeLimit(.minutes(2))) +@MainActor +struct InjectedASSRenditionSuppressionTests { + + private static let script = #""" + import http.server, os, time + class Handler(http.server.BaseHTTPRequestHandler): + protocol_version = "HTTP/1.1" + def log_message(self, *args): pass + def do_GET(self): + if self.path == "/held.ass": + deadline = time.time() + 60 + while not os.path.exists("release") and time.time() < deadline: + time.sleep(0.05) + body = open("sub.ass", "rb").read() + self.send_response(200) + self.send_header("Content-Type", "text/x-ssa") + self.send_header("Content-Length", str(len(body))) + self.end_headers() + self.wfile.write(body) + return + if self.path == "/master.m3u8": + body = b"#EXTM3U\n#EXT-X-STREAM-INF:BANDWIDTH=800000\nmedia.m3u8\n" + elif self.path == "/media.m3u8": + body = b"#EXTM3U\n#EXT-X-TARGETDURATION:6\n#EXT-X-PLAYLIST-TYPE:VOD\n#EXTINF:6,\nsegment.ts\n#EXT-X-ENDLIST\n" + else: + self.send_response(404) + self.send_header("Content-Length", "0") + self.end_headers() + return + self.send_response(200) + self.send_header("Content-Type", "application/vnd.apple.mpegurl") + self.send_header("Content-Length", str(len(body))) + self.end_headers() + self.wfile.write(body) + server = http.server.ThreadingHTTPServer(("127.0.0.1", 0), Handler) + print("READY %d" % server.server_address[1], flush=True) + server.serve_forever() + """# + + private static let assScript = """ + [Script Info] + ScriptType: v4.00+ + PlayResX: 320 + PlayResY: 180 + [V4+ Styles] + Format: Name, Fontname, Fontsize, PrimaryColour, SecondaryColour, OutlineColour, BackColour, Bold, Italic, Underline, StrikeOut, ScaleX, ScaleY, Spacing, Angle, BorderStyle, Outline, Shadow, Alignment, MarginL, MarginR, MarginV, Encoding + Style: Default,Arial,24,&H00FFFFFF,&H00FFFFFF,&H00000000,&H00000000,0,0,0,0,100,100,0,0,1,1,0,2,10,10,10,1 + [Events] + Format: Layer, Start, End, Style, Name, MarginL, MarginR, MarginV, Effect, Text + Dialogue: 0,0:00:01.00,0:00:04.00,Default,,0,0,0,,{\\b1}Styled caption + """ + + private static func suppressed(_ item: AVPlayerItem) -> Bool { + item.outputs.contains { ($0 as? AVPlayerItemLegibleOutput)?.suppressesPlayerRendering == true } + } + + /// The served master names each injected rendition plainly, so its display name is that NAME. + private static func selectedName(_ item: AVPlayerItem, in group: AVMediaSelectionGroup) -> String? { + item.currentMediaSelection.selectedMediaOption(in: group)?.displayName + } + + @Test("The overlay hides the selected rendition, a native surface shows it, and Off deselects it") + func overlayASSKeepsTheRenditionSelectedButHidden() async throws { + let origin = try #require(await PythonOrigin.launch(prefix: "aether-ass-suppress", script: Self.script)) + defer { origin.stop() } + let ass = FileManager.default.temporaryDirectory + .appendingPathComponent("suppressed-\(UUID().uuidString).ass") + try Self.assScript.write(to: ass, atomically: true, encoding: .utf8) + defer { try? FileManager.default.removeItem(at: ass) } + + let engine = try AetherEngine() + defer { engine.stop(finalTeardown: true) } + engine.hiddenRenditionIgnoresPlaybackForTesting = true + let master = try #require(URL(string: "http://127.0.0.1:\(origin.port)/master.m3u8")) + _ = try await engine.load(url: master, options: LoadOptions( + nativeRemoteHLS: true, preserveASSMarkup: true, + externalSubtitles: [ExternalSubtitleTrack(url: ass, name: "Styled ASS")], autoplay: false)) + + let id = AetherEngine.externalSubtitleTrackIDBase + let name = try #require(engine.injectedSubtitleRenditionNames[id]) + let item = try #require(engine.currentAVPlayer?.currentItem) + let group = try #require(try await item.asset.loadMediaSelectionGroup(for: .legible)) + + engine.selectSubtitleTrack(index: id) + try await waitFor { Self.selectedName(item, in: group) == name && Self.suppressed(item) } + // The host's overlay gets the raw events whatever AVPlayer is doing with the rendition. + try await waitFor { !engine.subtitleCues.isEmpty } + guard case .text(let raw) = try #require(engine.subtitleCues.first).body else { + Issue.record("Expected raw ASS events for the host's styled renderer") + return + } + #expect(raw.contains("{\\b1}Styled caption")) + + engine.setNativeSubtitleRendering(true) + try await waitFor { !Self.suppressed(item) } + #expect(Self.selectedName(item, in: group) == name) + + engine.setNativeSubtitleRendering(false) + try await waitFor { Self.selectedName(item, in: group) == name && Self.suppressed(item) } + + // Off takes the rendition down at once and leaves the output: nothing selected draws + // nothing, and the next styled pick must not uncover a line while it resolves. + engine.clearSubtitle() + #expect(Self.selectedName(item, in: group) == nil) + #expect(Self.suppressed(item)) + + engine.selectSubtitleTrack(index: id) + try await waitFor { Self.selectedName(item, in: group) == name } + #expect(Self.suppressed(item)) + } + + /// AE#616 measures presented lines, so a paused item has nothing to gain from the hidden rendition, + /// and selected before playback starts it would join the startup buffer. Silo's authorization + /// playback tests start with stall waiting off and stopped at rate 0 when it did. + @Test("A paused item gets no hidden rendition, and the overlay still gets its cues") + func pausedItemGetsNoHiddenRendition() async throws { + let origin = try #require(await PythonOrigin.launch(prefix: "aether-ass-paused", script: Self.script)) + defer { origin.stop() } + let ass = FileManager.default.temporaryDirectory + .appendingPathComponent("paused-\(UUID().uuidString).ass") + try Self.assScript.write(to: ass, atomically: true, encoding: .utf8) + defer { try? FileManager.default.removeItem(at: ass) } + let engine = try AetherEngine() + defer { engine.stop(finalTeardown: true) } + let master = try #require(URL(string: "http://127.0.0.1:\(origin.port)/master.m3u8")) + _ = try await engine.load(url: master, options: LoadOptions( + nativeRemoteHLS: true, preserveASSMarkup: true, + externalSubtitles: [ExternalSubtitleTrack(url: ass, name: "Styled ASS")], autoplay: false)) + let id = AetherEngine.externalSubtitleTrackIDBase + let item = try #require(engine.currentAVPlayer?.currentItem) + let group = try #require(try await item.asset.loadMediaSelectionGroup(for: .legible)) + let provider = try #require(engine.remoteHLSSubtitleProxy?.provider) + + engine.selectSubtitleTrack(index: id) + try await waitFor { provider.isFillFinished && !engine.subtitleCues.isEmpty } + try await Task.sleep(for: .milliseconds(500)) + #expect(engine.currentAVPlayer?.timeControlStatus != .playing) + #expect(Self.selectedName(item, in: group) == nil) + #expect(!Self.suppressed(item)) + } + + /// AVPlayer fetches and buffers a selected rendition, and the whole-program .vtt is served only + /// once extraction finishes. A hidden rendition selected before that holds playback up for a + /// subtitle nobody sees, and one still selected when the host turns subtitles off and plays is + /// fetched on into playback. Seen as a stalled start in Silo's authorization playback tests. + @Test("The hidden rendition waits for its extraction, and Off takes it down at once") + func hiddenRenditionWaitsForExtraction() async throws { + let origin = try #require(await PythonOrigin.launch( + prefix: "aether-ass-held", script: Self.script, files: ["sub.ass": Self.assScript])) + defer { origin.stop() } + let engine = try AetherEngine() + defer { engine.stop(finalTeardown: true) } + engine.hiddenRenditionIgnoresPlaybackForTesting = true + let master = try #require(URL(string: "http://127.0.0.1:\(origin.port)/master.m3u8")) + let sidecar = try #require(URL(string: "http://127.0.0.1:\(origin.port)/held.ass")) + _ = try await engine.load(url: master, options: LoadOptions( + nativeRemoteHLS: true, preserveASSMarkup: true, + externalSubtitles: [ExternalSubtitleTrack(url: sidecar, name: "Styled ASS")], autoplay: false)) + let id = AetherEngine.externalSubtitleTrackIDBase + let name = try #require(engine.injectedSubtitleRenditionNames[id]) + let item = try #require(engine.currentAVPlayer?.currentItem) + let group = try #require(try await item.asset.loadMediaSelectionGroup(for: .legible)) + let provider = try #require(engine.remoteHLSSubtitleProxy?.provider) + + // The origin holds the sidecar until the test lets it go, so the extraction is still running. + engine.selectSubtitleTrack(index: id) + try await Task.sleep(for: .milliseconds(800)) + try #require(!provider.isFillFinished) + #expect(Self.selectedName(item, in: group) == nil) + + FileManager.default.createFile( + atPath: origin.workDir.appendingPathComponent("release").path, contents: Data()) + try await waitFor { provider.isFillFinished } + try await waitFor { Self.selectedName(item, in: group) == name && Self.suppressed(item) } + + engine.clearSubtitle() + #expect(Self.selectedName(item, in: group) == nil) + } +} + +#endif diff --git a/Tests/AetherEngineTests/InjectedRenditionSelectionTests.swift b/Tests/AetherEngineTests/InjectedRenditionSelectionTests.swift new file mode 100644 index 000000000..a3806fe0b --- /dev/null +++ b/Tests/AetherEngineTests/InjectedRenditionSelectionTests.swift @@ -0,0 +1,44 @@ +import Testing +@testable import AetherEngine + +/// Audit NAT-104: `selectInjectedSubtitleRendition` took the first option whose playlist NAME or +/// localized display name equalled the sidecar's served NAME. AVFoundation derives the display name from +/// LANGUAGE, so the origin's own "English CC" rendition reads back as "English" and, sitting ahead of the +/// injected ones in the served master, won over the sidecar actually asked for. +@Suite("Injected rendition selection (NAT-104)") +struct InjectedRenditionSelectionTests { + + private func option(display: String, playlistName: String?) -> RemoteHLSMediaSelection.LegibleOption { + RemoteHLSMediaSelection.LegibleOption( + displayName: display, extendedLanguageTag: "en", + isDefault: false, isForced: false, isSDH: false, playlistName: playlistName) + } + + @Test("an origin rendition whose localized display name equals the sidecar's NAME does not win") + func originDisplayNameDoesNotShadowTheInjectedRendition() { + let group = [ + option(display: "English", playlistName: "English CC"), // origin, NAME="English CC" + option(display: "English", playlistName: "English"), // injected sidecar + ] + #expect(RemoteHLSMediaSelection.injectedRenditionIndex(named: "English", in: group) == 1) + } + + @Test("a rendition is found by its playlist NAME when the display name differs") + func matchesByPlaylistName() { + let group = [option(display: "Chinese", playlistName: "简体中文")] + #expect(RemoteHLSMediaSelection.injectedRenditionIndex(named: "简体中文", in: group) == 0) + } + + @Test("an OS that exposes no m3u8/NAME falls back to the display name") + func fallsBackToDisplayName() { + let group = [option(display: "German", playlistName: nil)] + #expect(RemoteHLSMediaSelection.injectedRenditionIndex(named: "German", in: group) == 0) + } + + @Test("a name that no option carries is a miss") + func missIsNil() { + let group = [option(display: "English", playlistName: "English CC")] + #expect(RemoteHLSMediaSelection.injectedRenditionIndex(named: "English", in: group) == nil) + #expect(RemoteHLSMediaSelection.injectedRenditionIndex(named: "English", in: []) == nil) + } +} diff --git a/Tests/AetherEngineTests/Issue119RemoteHLSHeaderTests.swift b/Tests/AetherEngineTests/Issue119RemoteHLSHeaderTests.swift index 495fe015a..d9daaa9ec 100644 --- a/Tests/AetherEngineTests/Issue119RemoteHLSHeaderTests.swift +++ b/Tests/AetherEngineTests/Issue119RemoteHLSHeaderTests.swift @@ -37,7 +37,8 @@ struct Issue119RemoteHLSHeaderTests { let cfg = URLSessionConfiguration.ephemeral cfg.protocolClasses = [HeaderCaptureURLProtocol.self] return AudioTapHLSFetcher(session: URLSession(configuration: cfg), - httpHeaders: httpHeaders) + httpHeaders: httpHeaders, + credentialOrigin: URL(string: "https://origin.test/live/channel.m3u8")) } @Test("fetchPlaylist sends the configured headers") diff --git a/Tests/AetherEngineTests/Issue177IngestPrefetchTests.swift b/Tests/AetherEngineTests/Issue177IngestPrefetchTests.swift index ee0512791..aac0ef0b6 100644 --- a/Tests/AetherEngineTests/Issue177IngestPrefetchTests.swift +++ b/Tests/AetherEngineTests/Issue177IngestPrefetchTests.swift @@ -285,6 +285,38 @@ struct Issue177IngestPrefetchTests { #expect(origin.concurrencyHighWater <= HLSLiveIngestReader.maxConcurrentSegmentFetches) } + @Test("a declared single-request origin gets its segments one at a time (AE#678)") + func declaredRequestCeilingSerializesTheBacklog() throws { + let segmentCount = 16 + let segments = (0.. TD=1 -> 3s holdback. + // 0.96s keyframe cadence, cut here: ceil(1.5*0.96)=2 (AE#670 GOP headroom) -> TD=2 -> 6s holdback. let cut = HLSVideoEngine.liveCutTargetSeconds(for: .fastZap) - XCTAssertEqual(LiveEdgePolicy.targetDurationSeconds(maxSegmentDuration: 0.96, cutTargetSeconds: cut, cadenceFloorSeconds: nil), 1) - XCTAssertEqual(LiveEdgePolicy.holdBackSeconds(targetDuration: 1), 3.0, accuracy: 0.0001) - // 2s GOP: ceil(1.92)=2 -> TD=2 -> 6s holdback. Still 3x faster than the standard 18s. - XCTAssertEqual(LiveEdgePolicy.targetDurationSeconds(maxSegmentDuration: 1.92, cutTargetSeconds: cut, cadenceFloorSeconds: nil), 2) + XCTAssertEqual(LiveEdgePolicy.targetDurationSeconds(maxSegmentDuration: 0.96, cutTargetSeconds: cut, cadenceFloorSeconds: nil, segmentsAreCutHere: true), 2) + XCTAssertEqual(LiveEdgePolicy.holdBackSeconds(targetDuration: 2), 6.0, accuracy: 0.0001) + // 2s GOP: ceil(1.5*1.92)=3 -> TD=3 -> 9s holdback. Still 2x faster than the standard 18s. + XCTAssertEqual(LiveEdgePolicy.targetDurationSeconds(maxSegmentDuration: 1.92, cutTargetSeconds: cut, cadenceFloorSeconds: nil, segmentsAreCutHere: true), 3) } func testFastZapLongGOPDegradesToStandardBehavior() { diff --git a/Tests/AetherEngineTests/Issue220BoundedRangeTests.swift b/Tests/AetherEngineTests/Issue220BoundedRangeTests.swift index 1f9735397..6614b5c28 100644 --- a/Tests/AetherEngineTests/Issue220BoundedRangeTests.swift +++ b/Tests/AetherEngineTests/Issue220BoundedRangeTests.swift @@ -75,7 +75,7 @@ struct Issue220BoundedRangeTests { /// the window legitimately opens the next range AT the frontier. A count cannot tell that /// apart from a re-fetch: this asserted two requests and saw three on a loaded CI runner, /// where the third was `bytes=524288-34078719`, the refill doing its job. - @Test("a completed range is read out of the window, not fetched again") + @Test("a completed range is read out of the window, not fetched again", .timeLimit(.minutes(1))) func completedRangeIsNotRefetched() async throws { let server = try #require(ThrottledOriginServer(totalSize: 512 * 1024 * 1024)) defer { server.stop() } @@ -91,9 +91,7 @@ struct Issue220BoundedRangeTests { defer { buf.deallocate() } _ = reader.read(into: buf, size: 64 * 1024) // Let the bounded range finish and its completion callback clear activeTask. - for _ in 0..<100 where reader.hasLiveConnectionForTesting { - try? await Task.sleep(nanoseconds: 10_000_000) - } + try await waitFor { !reader.hasLiveConnectionForTesting } // Still inside the delivered range, so the bytes come out of the window. #expect(reader.read(into: buf, size: 64 * 1024) == 64 * 1024) diff --git a/Tests/AetherEngineTests/Issue220SoftwareDecoderDrainTests.swift b/Tests/AetherEngineTests/Issue220SoftwareDecoderDrainTests.swift index 0fbb1d279..d86573c25 100644 --- a/Tests/AetherEngineTests/Issue220SoftwareDecoderDrainTests.swift +++ b/Tests/AetherEngineTests/Issue220SoftwareDecoderDrainTests.swift @@ -96,7 +96,7 @@ struct Issue220SoftwareDecoderDrainTests { /// ffmpeg -f lavfi -i "color=c=red:s=128x96:r=10:d=4" -c:v libx264 -preset ultrafast \ /// -tune zerolatency -bf 0 -pix_fmt yuv420p -x264-params keyint=10 -movflags +faststart swdec.mp4 /// base64 -i swdec.mp4 - private static let fixtureBase64 = """ + static let fixtureBase64 = """ AAAAIGZ0eXBpc29tAAACAGlzb21pc28yYXZjMW1wNDEAAAPPbW9vdgAAAGxtdmhkAAAAAAAAAAAAAAAAAAAD6AAAD6AAAQAA AQAAAAAAAAAAAAAAAAEAAAAAAAAAAAAAAAAAAAABAAAAAAAAAAAAAAAAAABAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAgAAAvl0cmFrAAAAXHRraGQAAAADAAAAAAAAAAAAAAABAAAAAAAAD6AAAAAAAAAAAAAAAAAAAAAAAAEAAAAAAAAAAAAA diff --git a/Tests/AetherEngineTests/Issue233ASSOverrideStyleTests.swift b/Tests/AetherEngineTests/Issue233ASSOverrideStyleTests.swift index 3e97e4443..1d1638d8f 100644 --- a/Tests/AetherEngineTests/Issue233ASSOverrideStyleTests.swift +++ b/Tests/AetherEngineTests/Issue233ASSOverrideStyleTests.swift @@ -149,6 +149,21 @@ struct Issue233ASSOverrideStyleTests { #expect(SubtitleRectText.playRes(fromASSHeader: "[Script Info]\nScriptType: v4.00+") == nil) } + // Audit SUB-112: `Double(_:)` accepts `nan`, `inf` and `1e999`, and a NaN position breaks the + // published [0, 1] placement contract (a UIKit host raises `CALayer position contains NaN`). + @Test("a non-finite position tag yields no position", arguments: [ + #"{\pos(nan,nan)}x"#, #"{\pos(1e999,5)}x"#, #"{\pos(5,-inf)}x"#, + ]) + func nonFinitePositionIgnored(body: String) { + #expect(placement("0,0,Default,,0,0,0,,\(body)")?.position == nil) + } + + @Test("a non-finite play resolution is no play resolution") + func nonFinitePlayResIgnored() { + #expect(SubtitleRectText.playRes(fromASSHeader: "[Script Info]\nPlayResX: inf\nPlayResY: 1080") == nil) + #expect(SubtitleRectText.playRes(fromASSHeader: "[Script Info]\nPlayResX: 1920\nPlayResY: nan") == nil) + } + // MARK: - The formats this actually serves /// What `ff_htmlmarkup_to_ass` emits for `ciao rosso`. diff --git a/Tests/AetherEngineTests/Issue243ReaderRetentionTests.swift b/Tests/AetherEngineTests/Issue243ReaderRetentionTests.swift index bc08553e0..82a2c0c3f 100644 --- a/Tests/AetherEngineTests/Issue243ReaderRetentionTests.swift +++ b/Tests/AetherEngineTests/Issue243ReaderRetentionTests.swift @@ -152,7 +152,9 @@ struct Issue243ReaderRetentionTests { config.protocolClasses = [RetentionRangeURLProtocol.self] HTTPDiscIOReader.resetLifetimeFetchedBytes() // Deltas, not absolutes: the tally is process-wide, and a reader in another suite may be - // fetching in parallel. Concurrent fetches can only add to it. + // fetching in parallel. A parallel engine session start also zeroes it (`startMemoryProbe`), + // which only the reset count can tell apart from a missed range. + let resetsAtStart = HTTPDiscIOReader.lifetimeFetchedResetCount let atStart = HTTPDiscIOReader.lifetimeFetchedBytes let reader = try #require(HTTPDiscIOReader( url: url, baseChunkSize: 256 * 1024, maxChunkSize: 256 * 1024, @@ -170,7 +172,10 @@ struct Issue243ReaderRetentionTests { } #expect(read == 1024 * 1024) // Four 256 KB refills plus the one-byte size probe from init. - #expect(HTTPDiscIOReader.lifetimeFetchedBytes - atStart >= Int64(1024 * 1024 + 1)) + let fetched = HTTPDiscIOReader.lifetimeFetchedBytes - atStart + if HTTPDiscIOReader.lifetimeFetchedResetCount == resetsAtStart { + #expect(fetched >= Int64(1024 * 1024 + 1)) + } HTTPDiscIOReader.resetLifetimeFetchedBytes() #expect(HTTPDiscIOReader.lifetimeFetchedBytes < Int64(1024 * 1024)) diff --git a/Tests/AetherEngineTests/Issue255BodyReserveTests.swift b/Tests/AetherEngineTests/Issue255BodyReserveTests.swift index bf140bd71..568fc659f 100644 --- a/Tests/AetherEngineTests/Issue255BodyReserveTests.swift +++ b/Tests/AetherEngineTests/Issue255BodyReserveTests.swift @@ -38,6 +38,8 @@ final class ScriptedOriginServer: @unchecked Sendable { /// Filler bytes actually written. May be far below or above `declaredLength`. var bodyBytes: Int = 0 var close: Bool = false + /// Real bytes to serve instead of the filler; `bodyBytes` is ignored when set. + var body: Data? = nil } private let listenFD: Int32 @@ -179,6 +181,15 @@ final class ScriptedOriginServer: @unchecked Sendable { guard method != "HEAD" else { return keepAlive } let sliceBytes = 256 * 1024 + if let body = r.body { + var sent = 0 + while sent < body.count && !stopped { + let n = min(sliceBytes, body.count - sent) + guard writeCounting(fd, Array(body[sent..<(sent + n)])) else { return false } + sent += n + } + return keepAlive + } let slice = [UInt8](repeating: 0x5A, count: sliceBytes) var sent = 0 while sent < r.bodyBytes && !stopped { diff --git a/Tests/AetherEngineTests/Issue281ColdStartRoundTripTests.swift b/Tests/AetherEngineTests/Issue281ColdStartRoundTripTests.swift index 429df2046..76f1b29ad 100644 --- a/Tests/AetherEngineTests/Issue281ColdStartRoundTripTests.swift +++ b/Tests/AetherEngineTests/Issue281ColdStartRoundTripTests.swift @@ -300,7 +300,7 @@ struct Issue281ColdStartRoundTripTests { /// itself is still a second connection opened at the same instant as the data connection whose /// first byte IS the cold start, and paid again on every open, against a server that has already /// shown it cannot serve it. - @Test("an origin that declined a suffix range is not asked again") + @Test("an origin that declined a suffix range is not asked again", .timeLimit(.minutes(1))) func declinedSuffixRangesAreNotRetried() async throws { let declared: Int64 = 4 * 1024 * 1024 * 1024 let server = try #require(ScriptedOriginServer { recorded in @@ -317,9 +317,7 @@ struct Issue281ColdStartRoundTripTests { let first = AVIOReader(url: url) try first.open() - for _ in 0..<100 where SuffixRangeSupport.shared.denialReason(for: url) == nil { - try? await Task.sleep(nanoseconds: 20_000_000) - } + try await waitFor { SuffixRangeSupport.shared.denialReason(for: url) != nil } first.markClosed(); first.close() let reason = try #require(SuffixRangeSupport.shared.denialReason(for: url), @@ -395,6 +393,34 @@ struct Issue281ColdStartRoundTripTests { #expect(AVIOReader.suffixRangeStart(response(nil, length: 100), expectedLength: 100) == nil) #expect(AVIOReader.suffixRangeStart(response("bytes */1000", length: 100), expectedLength: 100) == nil) + // Audit DMX-8: a suffix ends on the last byte of a numeric total, or it is not a suffix. + #expect(AVIOReader.suffixRangeStart(response("bytes 900-999/*", length: 100), + expectedLength: 100) == nil) + #expect(AVIOReader.suffixRangeStart(response("bytes 900-999/2000", length: 100), + expectedLength: 100) == nil) + #expect(AVIOReader.suffixRangeStart( + response("bytes 9223372036854775708-9223372036854775807/*", length: 100), + expectedLength: 100) == nil) + } + + /// `Int64` parses a sign, so a total of `Int64.min` reached `total - 1` and trapped the process + /// from a single response header. + @Test("a suffix 206 with a total that is not positive is refused, not trapped on") + func suffixStartRefusesANonPositiveTotal() { + let url = URL(string: "http://example.invalid/movie.bin")! + func response(_ contentRange: String) -> HTTPURLResponse { + HTTPURLResponse(url: url, statusCode: 206, httpVersion: "HTTP/1.1", + headerFields: ["Content-Range": contentRange])! + } + + #expect(AVIOReader.suffixRangeStart(response("bytes 0-0/-9223372036854775808"), + expectedLength: 1) == nil) + #expect(AVIOReader.suffixRangeStart(response("bytes 0-0/-1"), expectedLength: 1) == nil) + #expect(AVIOReader.suffixRangeStart(response("bytes 0-0/0"), expectedLength: 1) == nil) + #expect(AVIOReader.suffixRangeStart(response("bytes 0-0/1"), expectedLength: 1) == 0) + #expect(AVIOReader.suffixRangeStart( + response("bytes 9223372036854775707-9223372036854775806/9223372036854775807"), + expectedLength: 100) == 9223372036854775707) } /// The calibration the in-flight test above no longer carries, checked where it costs no socket diff --git a/Tests/AetherEngineTests/Issue296CarriageProbeConnectionBudgetTests.swift b/Tests/AetherEngineTests/Issue296CarriageProbeConnectionBudgetTests.swift index e5ebf7276..b60d544f7 100644 --- a/Tests/AetherEngineTests/Issue296CarriageProbeConnectionBudgetTests.swift +++ b/Tests/AetherEngineTests/Issue296CarriageProbeConnectionBudgetTests.swift @@ -266,6 +266,7 @@ final class Issue296CarriageProbeConnectionBudgetTests: XCTestCase { let verdict = await HLSCarriageProbe.classifyDeferredSegmentHead( url: try XCTUnwrap(URL(string: "https://cap.test/direct0.ts")), httpHeaders: ["X-Fixture": "allowed"], + credentialOrigin: try XCTUnwrap(URL(string: "https://cap.test/live.m3u8")), session: session ) @@ -284,6 +285,7 @@ final class Issue296CarriageProbeConnectionBudgetTests: XCTestCase { let verdict = await HLSCarriageProbe.classifyDeferredSegmentHead( url: try XCTUnwrap(URL(string: "https://cap.test/gone.ts")), httpHeaders: [:], + credentialOrigin: try XCTUnwrap(URL(string: "https://cap.test/live.m3u8")), session: session ) diff --git a/Tests/AetherEngineTests/Issue357BackgroundTeardownSelectionTests.swift b/Tests/AetherEngineTests/Issue357BackgroundTeardownSelectionTests.swift index 888915379..535a754c5 100644 --- a/Tests/AetherEngineTests/Issue357BackgroundTeardownSelectionTests.swift +++ b/Tests/AetherEngineTests/Issue357BackgroundTeardownSelectionTests.swift @@ -27,6 +27,16 @@ struct Issue357BackgroundTeardownSelectionTests { engine.stopInternal(resetDisplayCriteria: false, keepNativeHost: true, keepCustomReader: true) } + @Test("foreground restore is owed only after video teardown") + func restoreRequiresTeardown() throws { + let engine = try AetherEngine() + #expect(!engine.needsForegroundVideoRestore) + backgroundTeardown(engine) + #expect(engine.needsForegroundVideoRestore) + _ = engine.consumeReloadSelection() + #expect(!engine.needsForegroundVideoRestore) + } + @Test("the selection stopInternal wipes is still there for the reload that follows minutes later") func teardownHandsSelectionToReload() throws { let engine = try AetherEngine() @@ -86,6 +96,27 @@ struct Issue357BackgroundTeardownSelectionTests { #expect(selection.subtitles.reapplyOrdinalMatchesActiveTrack) } + /// Audit CORE-3: on iOS a `pause()` while backgrounded re-arms the grace window after the first + /// teardown, and its expiry tears down again. That second capture reads what `stopInternal` + /// already wiped and used to replace the good snapshot with it. + @Test("a second teardown before the reload keeps the selection the first one parked") + func secondTeardownKeepsParkedSelection() throws { + let engine = try AetherEngine() + let track = engine.addExternalSubtitleTrack(makeTrack("picked")) + engine.selectSubtitleTrack(index: track.id) + engine.activeAudioTrackIndex = 2 + engine.activeDiscTitleID = 7 + + backgroundTeardown(engine) + backgroundTeardown(engine) + + let selection = engine.consumeReloadSelection() + #expect(selection.subtitles.activeSubtitleTrackIndex == track.id) + #expect(selection.subtitles.hostExplicitSubtitleAction) + #expect(selection.audioTrackIndex == 2) + #expect(selection.discTitleID == 7) + } + @Test("a pick made after the teardown is newer intent and wins over the snapshot") func newerIntentWins() throws { let engine = try AetherEngine() @@ -147,12 +178,20 @@ struct Issue357BackgroundTeardownSelectionTests { let engine = try AetherEngine() engine.nativeHost = NativeAVPlayerHost() engine.playbackBackend = .native + let surface = AetherPlayerView(frame: .zero) + engine.bind(view: surface) + let layer = engine.nativePlayerLayer + #expect(layer?.superlayer != nil) backgroundTeardown(engine) // The teardown preserves the host on purpose: AVKit registers its Now-Playing client once // per AVPlayer instance (issue #15), so the instance has to outlive the suspension. #expect(engine.nativeHost != nil) + #expect(engine.nativePlayerLayer === layer) + #expect(layer?.superlayer != nil) + #expect(!engine.isSessionReady) + #expect(!engine.hasFirstFrameReadyForDisplay) // And it resets the backend, which is the state the foreground reload's load() reads. #expect(engine.playbackBackend == .none) // Reading the backend alone answered "nothing native here" and threw the kept host away. diff --git a/Tests/AetherEngineTests/Issue374FirstServeAccountTests.swift b/Tests/AetherEngineTests/Issue374FirstServeAccountTests.swift index ce1e9b4e8..456bbcf31 100644 --- a/Tests/AetherEngineTests/Issue374FirstServeAccountTests.swift +++ b/Tests/AetherEngineTests/Issue374FirstServeAccountTests.swift @@ -25,32 +25,6 @@ private final class Issue374WaitResult: @unchecked Sendable { } } -/// Captures `EngineLog` lines for the duration of one test. The handler is global, so it is restored in -/// every path rather than at the end of the happy one. -private final class Issue374LogTap: @unchecked Sendable { - private let lock = NSLock() - private var lines: [String] = [] - private let previous: ((String) -> Void)? - - init() { - previous = EngineLog.handler - let sink = { [self] (line: String) in - lock.lock() - lines.append(line) - lock.unlock() - } - EngineLog.handler = sink - } - - func restore() { EngineLog.handler = previous } - - func matching(_ needle: String) -> [String] { - lock.lock() - defer { lock.unlock() } - return lines.filter { $0.contains(needle) } - } -} - final class Issue374FirstServeAccountTests: XCTestCase { // MARK: The account line @@ -148,8 +122,8 @@ final class Issue374FirstServeAccountTests: XCTestCase { func testSatisfiedGateAccountsForItsWaitExactlyOnceAcrossRepeatedRequests() { let (provider, cache) = makeProvider(allowsBoundedDegradedStart: false) defer { cache.close() } - let tap = Issue374LogTap() - defer { tap.restore() } + let tap = EngineLogCapture() + defer { tap.end() } for index in 0..<15 { append(provider, index: index) @@ -168,8 +142,8 @@ final class Issue374FirstServeAccountTests: XCTestCase { func testBoundedFastZapStartNamesTheWholeWaitAndNotOnlyItsGrace() { let (provider, cache) = makeProvider(allowsBoundedDegradedStart: true) defer { cache.close() } - let tap = Issue374LogTap() - defer { tap.restore() } + let tap = EngineLogCapture() + defer { tap.end() } let result = Issue374WaitResult() let finished = expectation(description: "startup waiter finished") @@ -200,8 +174,8 @@ final class Issue374FirstServeAccountTests: XCTestCase { func testAGateThatNeverCutASegmentSaysSoRatherThanReturningInSilence() { let (provider, cache) = makeProvider(allowsBoundedDegradedStart: false) defer { cache.close() } - let tap = Issue374LogTap() - defer { tap.restore() } + let tap = EngineLogCapture() + defer { tap.end() } XCTAssertFalse(provider.waitForFirstLiveSegment(timeout: 0.2)) diff --git a/Tests/AetherEngineTests/Issue377FoldWitnessTests.swift b/Tests/AetherEngineTests/Issue377FoldWitnessTests.swift index c1d70a162..629fb0571 100644 --- a/Tests/AetherEngineTests/Issue377FoldWitnessTests.swift +++ b/Tests/AetherEngineTests/Issue377FoldWitnessTests.swift @@ -10,45 +10,21 @@ import XCTest @testable import AetherEngine -private final class Issue377FoldLogTap: @unchecked Sendable { - private let lock = NSLock() - private var lines: [String] = [] - private let previous: ((String) -> Void)? - - init() { - previous = EngineLog.handler - let sink = { [self] (line: String) in - lock.lock() - lines.append(line) - lock.unlock() - } - EngineLog.handler = sink - } - - func restore() { EngineLog.handler = previous } - - func matching(_ needle: String) -> [String] { - lock.lock() - defer { lock.unlock() } - return lines.filter { $0.contains(needle) } - } -} - final class Issue377FoldWitnessTests: XCTestCase { private let source = URL(string: "https://proxy.example.dev/stream/abc?token=one")! private let target = URL(string: "https://nexus-097.cdn.example.st/file.mkv?sig=aaa")! - private var tap: Issue377FoldLogTap! + private var tap: EngineLogCapture! override func setUp() { super.setUp() OriginRequestBudget.shared.resetForTesting() - tap = Issue377FoldLogTap() + tap = EngineLogCapture() } override func tearDown() { - tap.restore() + tap.end() tap = nil OriginRequestBudget.shared.resetForTesting() super.tearDown() diff --git a/Tests/AetherEngineTests/Issue377HeldConnectionTests.swift b/Tests/AetherEngineTests/Issue377HeldConnectionTests.swift index 4a9c63db9..5405c4b8c 100644 --- a/Tests/AetherEngineTests/Issue377HeldConnectionTests.swift +++ b/Tests/AetherEngineTests/Issue377HeldConnectionTests.swift @@ -81,7 +81,7 @@ struct Issue377HeldConnectionTests { @Test("the request carries the range, the host and the path with its query") func requestFraming() throws { let url = try #require(URL(string: "https://cdn.example.com/media/file.mkv?token=abc&x=1")) - let bytes = HeldSourceConnection.requestBytes( + let bytes = try HeldSourceConnection.requestBytes( target: url, host: "cdn.example.com", port: 443, secure: true, offset: 14_652_209_616, extraHeaders: [:], userAgent: "AetherEngine/test") let text = String(decoding: bytes, as: UTF8.self) @@ -96,7 +96,7 @@ struct Issue377HeldConnectionTests { @Test("a non-default port rides in Host, because an origin may route on it") func hostCarriesNonDefaultPort() throws { let url = try #require(URL(string: "http://192.168.1.10:8096/Videos/1/stream")) - let text = String(decoding: HeldSourceConnection.requestBytes( + let text = String(decoding: try HeldSourceConnection.requestBytes( target: url, host: "192.168.1.10", port: 8096, secure: false, offset: 0, extraHeaders: [:], userAgent: nil), as: UTF8.self) #expect(text.contains("\r\nHost: 192.168.1.10:8096\r\n")) @@ -106,7 +106,7 @@ struct Issue377HeldConnectionTests { @Test("a source header set wins without duplicating a header the request already sends") func extraHeadersDoNotDuplicate() throws { let url = try #require(URL(string: "https://jellyfin.example.com/Items/1/Download")) - let text = String(decoding: HeldSourceConnection.requestBytes( + let text = String(decoding: try HeldSourceConnection.requestBytes( target: url, host: "jellyfin.example.com", port: 443, secure: true, offset: 0, extraHeaders: ["X-Emby-Token": "secret", "User-Agent": "Sodalite/1.0", "Range": "bytes=99-"], userAgent: "AetherEngine/test"), as: UTF8.self) @@ -120,6 +120,38 @@ struct Issue377HeldConnectionTests { #expect(text.contains("\r\nRange: bytes=0-\r\n")) } + @Test("the path and query go on the wire percent-encoded, as the URL spells them") + func requestKeepsThePercentEncoding() throws { + let url = try #require(URL(string: "http://h.example/My%20Movie%3F.mkv/?a=%20b&c=%0D%0A")) + let text = String(decoding: try HeldSourceConnection.requestBytes( + target: url, host: "h.example", port: 80, secure: false, + offset: 0, extraHeaders: [:], userAgent: nil), as: UTF8.self) + #expect(text.hasPrefix("GET /My%20Movie%3F.mkv/?a=%20b&c=%0D%0A HTTP/1.1\r\n")) + #expect(text.components(separatedBy: "\r\n").allSatisfy { !$0.contains("\r") && !$0.contains("\n") }) + } + + @Test("an IPv6 literal is bracketed in Host") + func ipv6HostIsBracketed() throws { + let url = try #require(URL(string: "http://[::1]:8096/Videos/1/stream")) + let host = try #require(url.host) + let text = String(decoding: try HeldSourceConnection.requestBytes( + target: url, host: host, port: 8096, secure: false, + offset: 0, extraHeaders: [:], userAgent: nil), as: UTF8.self) + #expect(text.contains("\r\nHost: [::1]:8096\r\n")) + } + + @Test("a header value carrying a line break is refused rather than written") + func lineBreakInAHeaderIsRefused() throws { + let url = try #require(URL(string: "http://h.example/a.mkv")) + for value in ["a\r\nX-Injected: 1", "a\nX-Injected: 1", "a\rb"] { + #expect(throws: (any Error).self) { + try HeldSourceConnection.requestBytes( + target: url, host: "h.example", port: 80, secure: false, offset: 0, + extraHeaders: ["X-Custom": value], userAgent: nil) + } + } + } + // MARK: - Chunked framing @Test("a chunked body decodes across arbitrary feed boundaries") @@ -264,6 +296,37 @@ struct Issue377HeldConnectionTests { #expect(origin.requestLog.map(\.path) == ["/source.mkv", "/pinned.mkv"]) } + @Test("a cross-origin redirect drops the credential headers and keeps the rest") + func crossOriginRedirectDropsCredentials() async throws { + let total: Int64 = 64 * 1024 + final class PortBox: @unchecked Sendable { var port: UInt16 = 0 } + let box = PortBox() + let respond: @Sendable (Int, Int64, String) -> ThrottledOriginServer.Directive = { _, _, path in + path == "/pinned.mkv" ? .serve206 : .redirect(to: "http://localhost:\(box.port)/pinned.mkv") + } + let redirecting = ThrottledOriginServer(totalSize: total, throttleUs: 0, respond: respond) + let origin = try #require(redirecting) + defer { origin.stop() } + box.port = origin.port + let url = try #require(URL(string: "http://127.0.0.1:\(origin.port)/source.mkv")) + + let delegate = RecordingHeldDelegate(defaultBudget: 64 * 1024) + let connection = HeldSourceConnection( + url: url, offset: 0, + extraHeaders: ["X-Emby-Token": "secret", "Authorization": "MediaBrowser Token=\"secret\"", + "X-Custom": "kept"], + userAgent: nil, label: "test", delegate: delegate) + connection.start() + #expect(delegate.waitForEnd()) + + let headers = origin.requestHeaders + try #require(headers.count == 2) + #expect(headers[0]["x-emby-token"] == "secret") + #expect(headers[1]["x-emby-token"] == nil) + #expect(headers[1]["authorization"] == nil) + #expect(headers[1]["x-custom"] == "kept") + } + @Test("a mid-body offset asks for exactly that offset") func offsetIsHonoured() async throws { let total: Int64 = 4 * 1024 * 1024 diff --git a/Tests/AetherEngineTests/Issue377ReaderTransportLogTests.swift b/Tests/AetherEngineTests/Issue377ReaderTransportLogTests.swift index b37d7a76f..01e612797 100644 --- a/Tests/AetherEngineTests/Issue377ReaderTransportLogTests.swift +++ b/Tests/AetherEngineTests/Issue377ReaderTransportLogTests.swift @@ -8,43 +8,18 @@ import XCTest @testable import AetherEngine -/// Captures `EngineLog` lines for one test. The handler is global, so it is restored on every path. -private final class Issue377LogTap: @unchecked Sendable { - private let lock = NSLock() - private var lines: [String] = [] - private let previous: ((String) -> Void)? - - init() { - previous = EngineLog.handler - let sink = { [self] (line: String) in - lock.lock() - lines.append(line) - lock.unlock() - } - EngineLog.handler = sink - } - - func restore() { EngineLog.handler = previous } - - func matching(_ needle: String) -> [String] { - lock.lock() - defer { lock.unlock() } - return lines.filter { $0.contains(needle) } - } -} - final class Issue377ReaderTransportLogTests: XCTestCase { - private var tap: Issue377LogTap! + private var tap: EngineLogCapture! override func setUp() { super.setUp() ReaderTransportLog.resetForTesting() - tap = Issue377LogTap() + tap = EngineLogCapture() } override func tearDown() { - tap.restore() + tap.end() tap = nil ReaderTransportLog.resetForTesting() super.tearDown() diff --git a/Tests/AetherEngineTests/Issue378SourceRefusedTests.swift b/Tests/AetherEngineTests/Issue378SourceRefusedTests.swift index da879bc26..8f0557208 100644 --- a/Tests/AetherEngineTests/Issue378SourceRefusedTests.swift +++ b/Tests/AetherEngineTests/Issue378SourceRefusedTests.swift @@ -96,7 +96,7 @@ struct Issue378SourceRefusedTests { /// The regression guard for #281: an origin that answers the suffix form itself — here with a /// 416 — is still remembered after one occurrence. - @Test("a 416 to the suffix form still latches") + @Test("a 416 to the suffix form still latches", .timeLimit(.minutes(1))) func rangeNotSatisfiableStillLatches() async throws { let declared: Int64 = 4 * 1024 * 1024 * 1024 let server = try #require(ScriptedOriginServer { recorded in @@ -114,9 +114,7 @@ struct Issue378SourceRefusedTests { let reader = AVIOReader(url: url) defer { reader.markClosed(); reader.close() } try reader.open() - for _ in 0..<100 where SuffixRangeSupport.shared.denialReason(for: url) == nil { - try? await Task.sleep(nanoseconds: 20_000_000) - } + try await waitFor { SuffixRangeSupport.shared.denialReason(for: url) != nil } let reason = try #require(SuffixRangeSupport.shared.denialReason(for: url)) #expect(reason.contains("416")) } diff --git a/Tests/AetherEngineTests/Issue406NoCutWatchdogTests.swift b/Tests/AetherEngineTests/Issue406NoCutWatchdogTests.swift index 66083f210..09719fa04 100644 --- a/Tests/AetherEngineTests/Issue406NoCutWatchdogTests.swift +++ b/Tests/AetherEngineTests/Issue406NoCutWatchdogTests.swift @@ -200,6 +200,22 @@ struct Issue406NoCutWatchdogTests { #expect(abs(window.videoPtsAdvanceSeconds - 0.36) < 0.001) // 9 * 3600 ticks at 90 kHz } + @Test("a PTS advance across opposite ends of Int64 saturates instead of trapping (audit SEG-101)") + func ptsAdvanceSaturates() { + let w = makeWatchdog() + w.noteFinalize(at: t0) + w.notePacketRead() + w.noteVideoPacket(pts: Int64.min + 1, isKeyframe: true) + w.notePacketRead() + w.noteVideoPacket(pts: Int64.max, isKeyframe: false) + guard case .exitForRetune(let window)? = w.evaluate(now: t0.addingTimeInterval(36)) else { + Issue.record("expected an exit") + return + } + #expect(window.videoPtsAdvanceSeconds > 0) + #expect(window.videoPtsAdvanceSeconds.isFinite) + } + @Test("a video packet without a PTS does not open the advance window") func noptsVideoIsNotAnAdvance() { let w = makeWatchdog() diff --git a/Tests/AetherEngineTests/Issue432SynthesizedTimestampStrideTests.swift b/Tests/AetherEngineTests/Issue432SynthesizedTimestampStrideTests.swift index 8757a90a8..f085244ef 100644 --- a/Tests/AetherEngineTests/Issue432SynthesizedTimestampStrideTests.swift +++ b/Tests/AetherEngineTests/Issue432SynthesizedTimestampStrideTests.swift @@ -51,6 +51,41 @@ struct Issue432SynthesizedTimestampStrideTests { packetDuration: -1, observedStride: -5, fallbackDuration: -40) == 1) } + /// Audit SEG-101: the packet-duration arm was the one left uncapped, so a crafted BlockDuration + /// just under the source bound walked nine packets to Int64.max. + @Test("every arm of the cascade is capped at one second of ticks") + func strideIsCapped() { + let cap: Int64 = 90_000 + #expect(HLSSegmentProducer.repairStrideTicks( + packetDuration: (1 << 60) - 1, observedStride: 0, fallbackDuration: 0, maxStrideTicks: cap) == cap) + #expect(HLSSegmentProducer.repairStrideTicks( + packetDuration: 0, observedStride: 1 << 40, fallbackDuration: 0, maxStrideTicks: cap) == cap) + #expect(HLSSegmentProducer.repairStrideTicks( + packetDuration: 0, observedStride: 0, fallbackDuration: 1 << 40, maxStrideTicks: cap) == cap) + #expect(HLSSegmentProducer.repairStrideTicks( + packetDuration: 1800, observedStride: 0, fallbackDuration: 0, maxStrideTicks: cap) == 1800) + #expect(HLSSegmentProducer.repairStrideTicks( + packetDuration: 1800, observedStride: 0, fallbackDuration: 0, maxStrideTicks: 0) == 1) + } + + @Test("a synthesized timestamp that would leave the source range drops the packet instead") + func synthesizedWalkStaysInRange() { + #expect(HLSSegmentProducer.synthesizedDts(anchor: 1000, stride: 1800) == 2800) + #expect(HLSSegmentProducer.synthesizedDts(anchor: (1 << 60) - 10, stride: 90_000) == nil) + #expect(HLSSegmentProducer.synthesizedDts(anchor: Int64.max - 5, stride: 10) == nil) + var anchor: Int64 = 0 + for _ in 0..<9 { + let stride = HLSSegmentProducer.repairStrideTicks( + packetDuration: (1 << 60) - 1, observedStride: 0, fallbackDuration: 0, maxStrideTicks: 90_000) + guard let next = HLSSegmentProducer.synthesizedDts(anchor: anchor, stride: stride) else { + Issue.record("a one-second stride from 0 cannot leave the range") + return + } + anchor = next + } + #expect(anchor == 9 * 90_000) + } + // MARK: - Learning the stride from the source @Test("a forward delta inside the cap becomes the stride") diff --git a/Tests/AetherEngineTests/Issue440LiveJoinRollTests.swift b/Tests/AetherEngineTests/Issue440LiveJoinRollTests.swift index e67c13e88..2a3f88d41 100644 --- a/Tests/AetherEngineTests/Issue440LiveJoinRollTests.swift +++ b/Tests/AetherEngineTests/Issue440LiveJoinRollTests.swift @@ -79,14 +79,16 @@ struct Issue440LiveJoinRollTests { func firesOnTheHold() { #expect(NativeAVPlayerHost.shouldStartLiveJoinImmediately( armed: true, alreadySpent: false, hostWantsToPlay: true, - isWaitingToMinimizeStalls: true, playbackBufferEmpty: false, bufferedAheadSeconds: 4.0)) + isWaitingToMinimizeStalls: true, playbackBufferEmpty: false, bufferedAheadSeconds: 4.0, + itemIsReadyToPlay: true)) } @Test("never without the opt-in") func offByDefault() { #expect(!NativeAVPlayerHost.shouldStartLiveJoinImmediately( armed: false, alreadySpent: false, hostWantsToPlay: true, - isWaitingToMinimizeStalls: true, playbackBufferEmpty: false, bufferedAheadSeconds: 4.0)) + isWaitingToMinimizeStalls: true, playbackBufferEmpty: false, bufferedAheadSeconds: 4.0, + itemIsReadyToPlay: true)) } /// One shot, and it belongs to the join. Past it AVPlayer's stall policy is the right one: a live @@ -95,7 +97,8 @@ struct Issue440LiveJoinRollTests { func onceOnly() { #expect(!NativeAVPlayerHost.shouldStartLiveJoinImmediately( armed: true, alreadySpent: true, hostWantsToPlay: true, - isWaitingToMinimizeStalls: true, playbackBufferEmpty: false, bufferedAheadSeconds: 4.0)) + isWaitingToMinimizeStalls: true, playbackBufferEmpty: false, bufferedAheadSeconds: 4.0, + itemIsReadyToPlay: true)) } /// An empty buffer is the documented failure shape: `AVPlayer.h` says the call then behaves as if @@ -104,7 +107,8 @@ struct Issue440LiveJoinRollTests { func refusesEmptyBuffer() { #expect(!NativeAVPlayerHost.shouldStartLiveJoinImmediately( armed: true, alreadySpent: false, hostWantsToPlay: true, - isWaitingToMinimizeStalls: true, playbackBufferEmpty: true, bufferedAheadSeconds: 0.0)) + isWaitingToMinimizeStalls: true, playbackBufferEmpty: true, bufferedAheadSeconds: 0.0, + itemIsReadyToPlay: true)) } /// `EvaluatingBufferingRate` is the brief monitoring period Apple documents as not worth showing a @@ -113,7 +117,8 @@ struct Issue440LiveJoinRollTests { func refusesEvaluatingReason() { #expect(!NativeAVPlayerHost.shouldStartLiveJoinImmediately( armed: true, alreadySpent: false, hostWantsToPlay: true, - isWaitingToMinimizeStalls: false, playbackBufferEmpty: false, bufferedAheadSeconds: 4.0)) + isWaitingToMinimizeStalls: false, playbackBufferEmpty: false, bufferedAheadSeconds: 4.0, + itemIsReadyToPlay: true)) } /// A host that paused during the join asked for a still picture; overriding into motion would @@ -122,7 +127,8 @@ struct Issue440LiveJoinRollTests { func refusesWithoutPlayIntent() { #expect(!NativeAVPlayerHost.shouldStartLiveJoinImmediately( armed: true, alreadySpent: false, hostWantsToPlay: false, - isWaitingToMinimizeStalls: true, playbackBufferEmpty: false, bufferedAheadSeconds: 4.0)) + isWaitingToMinimizeStalls: true, playbackBufferEmpty: false, bufferedAheadSeconds: 4.0, + itemIsReadyToPlay: true)) } /// The reason string the host compares against is AVFoundation's own, not a literal of ours. @@ -142,7 +148,8 @@ struct Issue440LiveJoinRollTests { #expect(NativeAVPlayerHost.shouldStartLiveJoinImmediately( armed: true, alreadySpent: false, hostWantsToPlay: true, isWaitingToMinimizeStalls: true, playbackBufferEmpty: false, - bufferedAheadSeconds: ahead)) + bufferedAheadSeconds: ahead, + itemIsReadyToPlay: true)) } } @@ -157,7 +164,8 @@ struct Issue440LiveJoinRollTests { #expect(!NativeAVPlayerHost.shouldStartLiveJoinImmediately( armed: true, alreadySpent: false, hostWantsToPlay: true, isWaitingToMinimizeStalls: true, playbackBufferEmpty: false, - bufferedAheadSeconds: ahead)) + bufferedAheadSeconds: ahead, + itemIsReadyToPlay: true)) } } @@ -166,7 +174,8 @@ struct Issue440LiveJoinRollTests { #expect(NativeAVPlayerHost.shouldStartLiveJoinImmediately( armed: true, alreadySpent: false, hostWantsToPlay: true, isWaitingToMinimizeStalls: true, playbackBufferEmpty: false, - bufferedAheadSeconds: NativeAVPlayerHost.minimumLiveJoinBufferAhead)) + bufferedAheadSeconds: NativeAVPlayerHost.minimumLiveJoinBufferAhead, + itemIsReadyToPlay: true)) } /// A non-finite reading is an absent one. `currentTime()` on an item that has not resolved yields @@ -177,11 +186,13 @@ struct Issue440LiveJoinRollTests { #expect(!NativeAVPlayerHost.shouldStartLiveJoinImmediately( armed: true, alreadySpent: false, hostWantsToPlay: true, isWaitingToMinimizeStalls: true, playbackBufferEmpty: false, - bufferedAheadSeconds: .nan)) + bufferedAheadSeconds: .nan, + itemIsReadyToPlay: true)) #expect(!NativeAVPlayerHost.shouldStartLiveJoinImmediately( armed: true, alreadySpent: false, hostWantsToPlay: true, isWaitingToMinimizeStalls: true, playbackBufferEmpty: false, - bufferedAheadSeconds: .infinity)) + bufferedAheadSeconds: .infinity, + itemIsReadyToPlay: true)) } /// Both readings have to agree. AVPlayer calling the buffer empty outranks a span that looks deep, @@ -191,7 +202,8 @@ struct Issue440LiveJoinRollTests { #expect(!NativeAVPlayerHost.shouldStartLiveJoinImmediately( armed: true, alreadySpent: false, hostWantsToPlay: true, isWaitingToMinimizeStalls: true, playbackBufferEmpty: true, - bufferedAheadSeconds: 4.0)) + bufferedAheadSeconds: 4.0, + itemIsReadyToPlay: true)) } /// The depth is the CONTIGUOUS span ahead of the playhead, not the sum of every loaded range: an @@ -205,7 +217,8 @@ struct Issue440LiveJoinRollTests { #expect(!NativeAVPlayerHost.shouldStartLiveJoinImmediately( armed: true, alreadySpent: false, hostWantsToPlay: true, isWaitingToMinimizeStalls: true, playbackBufferEmpty: false, - bufferedAheadSeconds: ahead)) + bufferedAheadSeconds: ahead, + itemIsReadyToPlay: true)) } /// The default is what a host gets without asking, and since 6.55.0 that is on: the device A/B @@ -272,7 +285,8 @@ struct Issue440LiveJoinRollTests { #expect(NativeAVPlayerHost.shouldStartLiveJoinImmediately( armed: true, alreadySpent: false, hostWantsToPlay: true, isWaitingToMinimizeStalls: true, playbackBufferEmpty: empty, - bufferedAheadSeconds: ahead) == expected) + bufferedAheadSeconds: ahead, + itemIsReadyToPlay: true) == expected) } } @@ -459,14 +473,14 @@ struct Issue440LiveJoinRollTests { /// that followed 15 ms later reached no decision and no line. @Test("a playing status carried onto an item that is not ready yet does not spend the one-shot") func carriedPlayingDoesNotSpend() { - #expect(!NativeAVPlayerHost.playingSpendsLiveJoinOneShot(itemIsReadyToPlay: false)) + #expect(!NativeAVPlayerHost.playingIsThisItemsRoll(itemIsReadyToPlay: false)) } /// The reason the spend exists stays intact: once the item's own rate has rolled, every later hold /// is a mid-stream rebuffer and keeps AVPlayer's stall policy. @Test("a playing status on a ready item is the roll, and spends it") func realRollSpends() { - #expect(NativeAVPlayerHost.playingSpendsLiveJoinOneShot(itemIsReadyToPlay: true)) + #expect(NativeAVPlayerHost.playingIsThisItemsRoll(itemIsReadyToPlay: true)) } // MARK: - What `ahead 0.00s` was hiding (AE#447 follow-up) diff --git a/Tests/AetherEngineTests/Issue450ConcurrentReaderCeilingTests.swift b/Tests/AetherEngineTests/Issue450ConcurrentReaderCeilingTests.swift index ceca5f736..69b9967f3 100644 --- a/Tests/AetherEngineTests/Issue450ConcurrentReaderCeilingTests.swift +++ b/Tests/AetherEngineTests/Issue450ConcurrentReaderCeilingTests.swift @@ -69,29 +69,6 @@ struct Issue450ConcurrentReaderCeilingTests { } } - /// `EngineLog.handler` is process-global and swift-testing runs suites in parallel, so this - /// collects whatever else is logging at the same time. It only ever filters, never counts a - /// total, so a foreign line cannot change a verdict here. - private final class LogSink: @unchecked Sendable { - private let lock = NSLock() - private var lines: [String] = [] - private let previous: ((String) -> Void)? - init() { - previous = EngineLog.handler - let sink = { [self] (line: String) in - lock.lock() - lines.append(line) - lock.unlock() - } - EngineLog.handler = sink - } - func restore() { EngineLog.handler = previous } - func matching(_ needle: String) -> [String] { - lock.lock(); defer { lock.unlock() } - return lines.filter { $0.contains(needle) } - } - } - // MARK: - The transport contract the reader pools are built on /// The negative control. Without it the test below could pass for a reason that has nothing to @@ -190,10 +167,12 @@ struct Issue450ConcurrentReaderCeilingTests { let server = try #require(serverMaybe) defer { server.stop() } - let sink = LogSink() - defer { sink.restore() } + let sink = EngineLogCapture() + defer { sink.end() } + let label = "silent-\(UUID().uuidString)" let reader = AVIOReader(url: URL(string: "http://127.0.0.1:\(server.port)/movie.bin")!, + label: label, boundedInitialFetch: firstRange, connStallTimeout: stallTimeout) defer { reader.markClosed(); reader.close() } @@ -203,9 +182,11 @@ struct Issue450ConcurrentReaderCeilingTests { // refill is the generation that receives headers and no body. try await waitFor(upTo: .seconds(10)) { !reader.hasLiveConnectionForTesting } #expect(Self.read(reader, bytes: 512 * 1024, deadline: 10) == 512 * 1024) - try await waitFor(upTo: .seconds(10)) { !sink.matching("no first byte after").isEmpty } + // The capture sees every reader in the process, so the reader's own label picks its line. + func reported() -> [String] { sink.matching(label).filter { $0.contains("no first byte after") } } + try await waitFor { !reported().isEmpty } - let lines = sink.matching("no first byte after") + let lines = reported() #expect(!lines.isEmpty, "a generation that received headers and no body was never reported") // The two facts that separate a parked request from an origin sitting on one. Without them // the line names the silence but not the side it is on. diff --git a/Tests/AetherEngineTests/Issue451SessionSweepLivenessTests.swift b/Tests/AetherEngineTests/Issue451SessionSweepLivenessTests.swift index ed8e3cd1d..d46838302 100644 --- a/Tests/AetherEngineTests/Issue451SessionSweepLivenessTests.swift +++ b/Tests/AetherEngineTests/Issue451SessionSweepLivenessTests.swift @@ -111,7 +111,7 @@ struct Issue451SessionSweepLivenessTests { #expect(c.totalBytes == 512) // Whatever removed it (a sibling's sweep, the OS reclaiming tmp) is outside this class. - try? FileManager.default.removeItem(at: c.sessionDir.appendingPathComponent("seg-4.m4s")) + if let url = c.peekURL(index: 4) { try? FileManager.default.removeItem(at: url) } #expect(c.peekURL(index: 4) == nil, "a URL handed to a response must name a file that exists") @@ -127,7 +127,7 @@ struct Issue451SessionSweepLivenessTests { let c = SegmentCache(baseDirectory: base) defer { c.close() } c.store(index: 2, data: Data(repeating: 0xDD, count: 64)) - try? FileManager.default.removeItem(at: c.sessionDir.appendingPathComponent("seg-2.m4s")) + if let url = c.peekURL(index: 2) { try? FileManager.default.removeItem(at: url) } // The serve that finds nothing answers a retriable 503; without dropping the entry, the // next fetch takes the same branch and the producer is never asked to make it again. diff --git a/Tests/AetherEngineTests/Issue458AudibleReadbackTests.swift b/Tests/AetherEngineTests/Issue458AudibleReadbackTests.swift index 375c5c0ad..7151397f6 100644 --- a/Tests/AetherEngineTests/Issue458AudibleReadbackTests.swift +++ b/Tests/AetherEngineTests/Issue458AudibleReadbackTests.swift @@ -122,7 +122,7 @@ struct Issue458AudibleReadbackTests { .deletingLastPathComponent() .deletingLastPathComponent() .appendingPathComponent("Sources/AetherEngine/AetherEngine+Loading.swift") - guard let text = try? String(contentsOf: source, encoding: .utf8) else { return } + let text = try String(contentsOf: source, encoding: .utf8) #expect(text.contains("logAudibleReadback(host: host)")) } } diff --git a/Tests/AetherEngineTests/Issue460ReloadWithOptionsTests.swift b/Tests/AetherEngineTests/Issue460ReloadWithOptionsTests.swift index 81403ecf4..cf0746688 100644 --- a/Tests/AetherEngineTests/Issue460ReloadWithOptionsTests.swift +++ b/Tests/AetherEngineTests/Issue460ReloadWithOptionsTests.swift @@ -17,6 +17,27 @@ struct Issue460SessionOptionCorrectionTests { } } + /// Listing a field as identity refuses nothing on its own: `refusedFields` compares each one by + /// hand, and `sharedOutputRole` was listed without a comparison. + @Test("every identity field, flipped alone, is refused by name") + func everyIdentityFieldIsRefused() { + let base = LoadOptions() + let flips: [String: (inout LoadOptions) -> Void] = [ + "isLive": { $0.isLive.toggle() }, + "audioOnly": { $0.audioOnly.toggle() }, + "nativeRemoteHLS": { $0.nativeRemoteHLS.toggle() }, + "sequentialOrigin": { $0.sequentialOrigin.toggle() }, + "heldSourceConnection": { $0.heldSourceConnection.toggle() }, + "sharedOutputRole": { $0.sharedOutputRole = $0.sharedOutputRole == .primary ? .secondary : .primary }, + ] + #expect(Set(flips.keys) == Set(SessionOptionCorrection.loadIdentityFields)) + for (field, flip) in flips { + var proposed = base + flip(&proposed) + #expect(SessionOptionCorrection.refusedFields(from: base, to: proposed) == [field]) + } + } + /// The inventory guard. A field added to `LoadOptions` is correctable by default (it falls /// through `refusedFields`), which is right for a tuning lever and wrong for one that names the /// session, so adding one has to be a decision. This fails until someone makes it. diff --git a/Tests/AetherEngineTests/Issue462AudioDeliveryTests.swift b/Tests/AetherEngineTests/Issue462AudioDeliveryTests.swift index 2ba734c30..ff04d5616 100644 --- a/Tests/AetherEngineTests/Issue462AudioDeliveryTests.swift +++ b/Tests/AetherEngineTests/Issue462AudioDeliveryTests.swift @@ -158,9 +158,15 @@ struct AudioDeliveryClassificationTests { /// video-only, exactly like the loopback tail and just as silently. @Test("the software host separates an absent track from a decoder that would not open") func softwareHostSeparatesAbsentFromDropped() { - #expect(SoftwarePlaybackHost.audioDelivery(resolvedAudioIndex: 1, decoderOpened: true) == .decoded) - #expect(SoftwarePlaybackHost.audioDelivery(resolvedAudioIndex: 1, decoderOpened: false) == .droppedNoPipeline) - #expect(SoftwarePlaybackHost.audioDelivery(resolvedAudioIndex: -1, decoderOpened: false) == .noAudioInSource) + #expect(SoftwarePlaybackHost.audioDelivery(resolvedAudioIndex: 1, decoderOpened: true, + sourceCarriesAudio: true) == .decoded) + #expect(SoftwarePlaybackHost.audioDelivery(resolvedAudioIndex: 1, decoderOpened: false, + sourceCarriesAudio: true) == .droppedNoPipeline) + #expect(SoftwarePlaybackHost.audioDelivery(resolvedAudioIndex: -1, decoderOpened: false, + sourceCarriesAudio: false) == .noAudioInSource) + // AE#641: a stream the pick passed over (empty parameters) is still the source's audio. + #expect(SoftwarePlaybackHost.audioDelivery(resolvedAudioIndex: -1, decoderOpened: false, + sourceCarriesAudio: true) == .droppedNoPipeline) } /// The label is the second half of the same lie: it was built from the probe's track list, so a diff --git a/Tests/AetherEngineTests/Issue464AudioDelayTests.swift b/Tests/AetherEngineTests/Issue464AudioDelayTests.swift index 281ef0ae4..cdb432bb8 100644 --- a/Tests/AetherEngineTests/Issue464AudioDelayTests.swift +++ b/Tests/AetherEngineTests/Issue464AudioDelayTests.swift @@ -111,7 +111,7 @@ struct Issue464PlacementRationaleTests { } @Test("a backwards audio step is clamped away, so a smaller offset cannot be spliced mid-track") - func sanitizerEatsABackwardsStep() { + func sanitizerEatsABackwardsStep() throws { // The naive loopback implementation lowers the muxer's offset and keeps cutting into the same // output track. OutputTimestampSanitizer enforces strictly increasing DTS per stream, so the // overlap the change creates is not delivered: it is clamped to last+1 and the audio stays @@ -119,7 +119,8 @@ struct Issue464PlacementRationaleTests { var sanitizer = OutputTimestampSanitizer() let audio: Int32 = 1 _ = sanitizer.sanitize(streamIndex: audio, pts: 48000, dts: 48000) - let stepBack = sanitizer.sanitize(streamIndex: audio, pts: 43200, dts: 43200) // -100 ms @48k + let clamped = sanitizer.sanitize(streamIndex: audio, pts: 43200, dts: 43200) // -100 ms @48k + let stepBack = try #require(clamped) #expect(stepBack.dts == 48001, "the backwards step was clamped, not honoured") #expect(stepBack.pts >= stepBack.dts) } diff --git a/Tests/AetherEngineTests/Issue464ReloadPreservesSessionTests.swift b/Tests/AetherEngineTests/Issue464ReloadPreservesSessionTests.swift index 887ff29cb..62c312727 100644 --- a/Tests/AetherEngineTests/Issue464ReloadPreservesSessionTests.swift +++ b/Tests/AetherEngineTests/Issue464ReloadPreservesSessionTests.swift @@ -1,3 +1,4 @@ +import Combine import Foundation import Testing @testable import AetherEngine @@ -74,6 +75,24 @@ struct Issue464RebuildPositionTests { // what a reload stacked onto it must read: the previous session's playhead is gone. #expect(AetherEngine.rebuildPosition(state: .loading, clock: 0, underReconstruction: 0) == 0) } + + @Test("a load that has returned but not yet published a playhead still answers with its parked position") + func returnedLoadBeforeFirstPublishKeepsThePosition() { + // Round 5 (cmcpherson274, E8-F4): the autostart at the tail of `load()` writes `.playing` + // before the new host has published a position, so a correction raised the moment a rebuild + // returned read the zeroed clock and rebuilt at the head. Measured on the CLI with two + // `setAudioDelay` presses 50-90 ms apart on `.loopback`: `#3 mount seek: item axis 0.00s`. + #expect(AetherEngine.rebuildPosition(state: .playing, clock: 0, underReconstruction: 312.8) == 312.8) + #expect(AetherEngine.rebuildPosition(state: .paused, clock: 0, underReconstruction: 312.8) == 312.8) + } + + @Test("once the session has published a playhead, or has none, the clock answers again") + func publishedOrTerminalReadsTheClock() { + #expect(AetherEngine.rebuildPosition(state: .playing, clock: 312.9, underReconstruction: 312.8) == 312.9) + #expect(AetherEngine.rebuildPosition(state: .seeking, clock: 0, underReconstruction: 312.8) == 0) + #expect(AetherEngine.rebuildPosition(state: .idle, clock: 0, underReconstruction: 312.8) == 0) + #expect(AetherEngine.rebuildPosition(state: .ended, clock: 0, underReconstruction: 312.8) == 0) + } } /// AE#464 round 3 (measured on the CLI while building the in-flight latch): the window round 2 @@ -119,3 +138,177 @@ struct Issue464StackedRebuildTransportTests { state: .playing, nativeTransportIntent: nil, underReconstruction: false)) } } + +/// Audit LIF-102: `reloadWithAudioOverride` is the rebuild behind every audio-track pick, the custom +/// disc-title pick and `reloadAtCurrentPosition` on a custom source, and it ended in an unconditional +/// `play()`. The pure answer above was right and the call site ignored it: pause, pick another +/// language, and the film started behind the menu. +@Suite("Audit LIF-102: the audio-switch and custom-source rebuild comes back in the session's transport", + .timeLimit(.minutes(2))) +@MainActor +struct Issue464RebuildCallSiteTransportTests { + + private static func customSource() throws -> MediaSource { + .custom(DataIOReader(data: try ProbeTestFixtures.hdr10Plus()), formatHint: "mp4") + } + + @Test("an audio-track rebuild of a paused session leaves it paused") + func audioSwitchKeepsAPausedSessionPaused() async throws { + let engine = try AetherEngine() + defer { engine.stop() } + _ = try await engine.load(source: Self.customSource()) + engine.pause() + #expect(engine.state == .paused) + let url = try #require(engine.loadedURL) + + let failure = await engine.reloadWithAudioOverride( + url: url, audioStreamIndex: nil, expectedGeneration: engine.loadGeneration) + + #expect(failure == nil) + #expect(engine.state != .playing) + #expect(engine.nativeHost?.transportIntentIsPlaying == false) + try await waitFor { engine.state == .paused } + } + + @Test("a rebuild stacked behind that audio switch reads the paused transport, not the mount flag") + func stackedRebuildReadsThePausedTransport() async throws { + let engine = try AetherEngine() + defer { engine.stop() } + _ = try await engine.load(source: Self.customSource()) + engine.pause() + let url = try #require(engine.loadedURL) + + // Stack the read behind the rebuild instead of polling for `.loading`: a custom-source + // rebuild can enter and leave `.loading` between two polls, and the poll then waits for a + // state that is already gone until `.timeLimit` ends the run (red CI on 2026-10-05 and + // 2026-10-06). A main-actor job enqueued as `.loading` is published runs at the rebuild's + // first suspension, which is exactly where a second rebuild raised behind it would read. + var stacked: Task<(PlaybackState, Bool), Never>? + let observer = engine.$state.sink { next in + MainActor.assumeIsolated { + guard stacked == nil, next == .loading else { return } + stacked = Task { @MainActor in (engine.state, engine.sessionRebuildResumesPlaying) } + } + } + defer { observer.cancel() } + + _ = await engine.reloadWithAudioOverride( + url: url, audioStreamIndex: nil, expectedGeneration: engine.loadGeneration) + let (stateThen, resumes) = try #require(await stacked?.value) + #expect(stateThen == .loading) + #expect(!resumes) + } + + @Test("a mount with autoplay off stays paused across a custom-source reload") + func pausedMountStaysPausedAcrossACustomReload() async throws { + let engine = try AetherEngine() + defer { engine.stop() } + _ = try await engine.load(source: Self.customSource(), options: LoadOptions(autoplay: false)) + try await waitFor { engine.state == .paused } + + try await engine.reloadAtCurrentPosition() + + #expect(engine.state != .playing) + #expect(engine.nativeHost?.transportIntentIsPlaying == false) + try await waitFor { engine.state == .paused } + } + + /// AE#711 follow-up: a press that lands while the rebuild runs is the newer word. `body` runs at + /// the rebuild's first suspension after `.loading` is published, where a host's button would. + private static func duringRebuild( + of engine: AetherEngine, _ body: @escaping @MainActor () -> Void, + rebuild: () async throws -> Void + ) async throws { + var fired: Task? + let observer = engine.$state.sink { next in + MainActor.assumeIsolated { + guard fired == nil, next == .loading else { return } + fired = Task { @MainActor in body() } + } + } + defer { observer.cancel() } + try await rebuild() + await fired?.value + #expect(fired != nil) + } + + @Test("a pause pressed during an audio rebuild of a playing session holds") + func pauseDuringAudioRebuildHolds() async throws { + let engine = try AetherEngine() + defer { engine.stop() } + _ = try await engine.load(source: Self.customSource()) + #expect(engine.state == .playing) + let url = try #require(engine.loadedURL) + + try await Self.duringRebuild(of: engine, { engine.pause() }) { + _ = await engine.reloadWithAudioOverride( + url: url, audioStreamIndex: nil, expectedGeneration: engine.loadGeneration) + } + + #expect(engine.state != .playing) + #expect(engine.nativeHost?.transportIntentIsPlaying == false) + } + + @Test("a play pressed during a reload of a paused session holds") + func playDuringReloadHolds() async throws { + let engine = try AetherEngine() + defer { engine.stop() } + _ = try await engine.load(source: Self.customSource()) + engine.pause() + try await waitFor { engine.state == .paused } + + try await Self.duringRebuild(of: engine, { engine.play() }) { + try await engine.reloadAtCurrentPosition() + } + + #expect(engine.state == .playing) + #expect(engine.nativeHost?.transportIntentIsPlaying == true) + } + + @Test("a pause pressed during a URL reload of a playing session holds") + func pauseDuringURLReloadHolds() async throws { + let file = FileManager.default.temporaryDirectory + .appendingPathComponent("ae711-intent-\(UUID().uuidString).mp4") + try ProbeTestFixtures.hdr10Plus().write(to: file) + defer { try? FileManager.default.removeItem(at: file) } + let engine = try AetherEngine() + defer { engine.stop() } + _ = try await engine.load(url: file) + #expect(engine.state == .playing) + #expect(!engine.isCustomSource) + + try await Self.duringRebuild(of: engine, { engine.pause() }) { + try await engine.reloadAtCurrentPosition() + } + + #expect(engine.state != .playing) + #expect(engine.nativeHost?.transportIntentIsPlaying == false) + } + + @Test("outside a rebuild a press parks nothing for the next one") + func pressOutsideARebuildParksNothing() throws { + let engine = try AetherEngine() + defer { engine.stop() } + engine.transportIntentUnderReconstruction = nil + engine.pause() + engine.play() + #expect(engine.transportIntentUnderReconstruction == nil) + engine.sessionPreservingReloadInFlight = true + engine.pause() + #expect(engine.transportIntentUnderReconstruction == false) + engine.sessionPreservingReloadInFlight = false + } + + @Test("a playing session still comes back playing") + func playingSessionComesBackPlaying() async throws { + let engine = try AetherEngine() + defer { engine.stop() } + _ = try await engine.load(source: Self.customSource()) + #expect(engine.state == .playing) + + try await engine.reloadAtCurrentPosition() + + #expect(engine.state == .playing) + #expect(engine.nativeHost?.transportIntentIsPlaying == true) + } +} diff --git a/Tests/AetherEngineTests/Issue511MatroskaCodingOrderTests.swift b/Tests/AetherEngineTests/Issue511MatroskaCodingOrderTests.swift index 7957e3b9e..f63122715 100644 --- a/Tests/AetherEngineTests/Issue511MatroskaCodingOrderTests.swift +++ b/Tests/AetherEngineTests/Issue511MatroskaCodingOrderTests.swift @@ -134,6 +134,22 @@ struct Issue511MatroskaCodingOrderTests { #expect(result.sequence.slots == [Self.reportedSlots[60]]) } + /// Audit BIT-3: `I0 P(poc6) B(poc2)` then an IDR. P waits for rank 3, which this sequence never + /// states; the slot it can own is the one rank 2 would have taken, not rank 3 of the next GOP. + @Test("a picture left waiting when an IDR closes its sequence takes that sequence's free slot") + func waitingPictureKeepsItsOwnSequence() throws { + var sequence = Permutation.Sequence(pocStep: 2, videoDelay: 1) + #expect(sequence.admit(slot: 0, pictureOrderCount: 0, isKeyframe: true) == 0) + let admitted = sequence.admit(slot: 33, pictureOrderCount: 6, isKeyframe: false) + let waiting = try #require(admitted) + #expect(sequence.admit(slot: 67, pictureOrderCount: 2, isKeyframe: false) == 1) + #expect(sequence.slot(forRank: waiting) == nil) + #expect(sequence.unclaimedSlots() == [67]) + + _ = sequence.admit(slot: 100, pictureOrderCount: 0, isKeyframe: true) + #expect(sequence.unclaimedSlots().isEmpty) + } + @Test("a healthy reordered MKV is out on the packet that steps back, not after the window") func healthyLadderExits() { // The same pictures with their slots where the format says they belong: in display order, diff --git a/Tests/AetherEngineTests/Issue514AverageBitrateTests.swift b/Tests/AetherEngineTests/Issue514AverageBitrateTests.swift index 340f2293b..8d2ceb2b0 100644 --- a/Tests/AetherEngineTests/Issue514AverageBitrateTests.swift +++ b/Tests/AetherEngineTests/Issue514AverageBitrateTests.swift @@ -1,140 +1,210 @@ import Foundation import Testing -import AVFoundation @testable import AetherEngine -/// AE#514: `LiveTelemetry.averageBitrateMbps` divided the session's lifetime bytes by pure wall-clock -/// time. Bytes stop arriving while the transport is paused (the forward buffer is already full), the -/// divisor does not, so the reported average decayed toward zero for as long as the pause lasted and -/// never came back: the paused seconds stayed in the divisor for the rest of the session. The fix -/// charges a tick's second only when the session is in a phase that consumes media. -@MainActor +/// AE#514: both bitrate fields in `LiveTelemetry` were metered from the reader's transfer counter. +/// Round one fixed the divisor (a pause dragged the average toward zero); round two, from the +/// reporter's retest, fixes the numerator: transfer and playback part ways on every route that reads +/// ahead. VOD prefetch put a 20 Mbps stream at ~35 Mbps, and a paused live session kept draining the +/// origin into its DVR window while the divisor stood still, so its average climbed for as long as the +/// pause ran. Both fields now meter the bytes of the packets the playhead crossed over the media +/// seconds it crossed. struct Issue514AverageBitrateTests { - // MARK: - Which seconds belong in the divisor - - /// A pause and the three terminal phases are the session standing still. Everything else is the - /// session working, and the bytes it did or did not get in those seconds are part of its average. - @Test("only the phases that consume media charge a second to the divisor") - func pausedAndTerminalPhasesDoNotCharge() { - #expect(LiveTelemetrySampler.chargesActiveTime(.paused) == false) - #expect(LiveTelemetrySampler.chargesActiveTime(.idle) == false) - #expect(LiveTelemetrySampler.chargesActiveTime(.ended) == false) - #expect(LiveTelemetrySampler.chargesActiveTime(.error("boom")) == false) - - #expect(LiveTelemetrySampler.chargesActiveTime(.playing)) - #expect(LiveTelemetrySampler.chargesActiveTime(.loading)) - #expect(LiveTelemetrySampler.chargesActiveTime(.rebuffering)) - #expect(LiveTelemetrySampler.chargesActiveTime(.stalled(reconnecting: true))) - #expect(LiveTelemetrySampler.chargesActiveTime(.stalled(reconnecting: false))) + /// 20 Mbps split over a 25 fps video track and a ~47 Hz audio track, recorded as a pump would. + private static let videoBytesPerSecond = 2_400_000 + private static let audioBytesPerSecond = 100_000 + private static let mbps = Double(videoBytesPerSecond + audioBytesPerSecond) * 8 / 1_000_000 + + private static func feed(_ ledger: PlayedMediaLedger, from start: Double, to end: Double) { + var pts = start + while pts < end - 1e-9 { + ledger.record(.video, pts: pts, bytes: videoBytesPerSecond / 25) + pts += 1.0 / 25 + } + pts = start + while pts < end - 1e-9 { + ledger.record(.audio, pts: pts, bytes: audioBytesPerSecond * 1024 / 48_000) + pts += 1024.0 / 48_000 + } } - /// Deliberately NOT excluded, against the report's own suggestion: a seek is where the session - /// fetches hardest (the buffer is discarded and a new range pulled). Dropping those seconds while - /// keeping their bytes would push the average above the media's real rate on every scrub. - @Test("a seek charges its seconds, because it is also where the bytes arrive") - func seekingCharges() { - #expect(LiveTelemetrySampler.chargesActiveTime(.seeking)) + private static func play(_ meter: inout PlayedBitrateMeter, _ ledger: PlayedMediaLedger, + from start: Double, seconds: Int) -> Double { + var playhead = start + for _ in 0.. Bool { + guard let value else { return false } + return abs(value - expected) <= tolerance * expected + } - @Test("the average is the lifetime bytes over the active seconds") - func averageIsBytesOverActiveSeconds() { - // 2 Mbps for ten active seconds: 2 500 000 bytes. - let rate = LiveTelemetrySampler.averageBitrateMbps(lifetimeBytes: 2_500_000, activeSeconds: 10) - #expect(abs((rate ?? 0) - 2.0) < 0.001) + // MARK: - The reporter's table + + /// VOD, playing: minutes of read-ahead sit in the ledger the moment they are fetched, and count + /// only once the playhead crosses them. + @Test("VOD read-ahead does not inflate either field") + func vodReadAheadIsNotCounted() { + let ledger = PlayedMediaLedger() + Self.feed(ledger, from: 0, to: 240) + var meter = PlayedBitrateMeter() + meter.advance(to: 0, wallSeconds: 0, ledger: ledger) + _ = Self.play(&meter, ledger, from: 0, seconds: 30) + #expect(Self.near(meter.averageMbps, Self.mbps)) + #expect(Self.near(meter.instantMbps, Self.mbps)) } - /// Same rule as `observedTransferMbps` one field over: "not measurable yet" is a gap, never a - /// confident zero. Before the fix the very first tick published 0.00 Mbps, because it seeds - /// `sessionStartBytes` from the same counter it then subtracts. - @Test("a session with no active time and no bytes publishes nil, not zero") - func unmeasurableSessionPublishesNil() { - #expect(LiveTelemetrySampler.averageBitrateMbps(lifetimeBytes: 0, activeSeconds: 0) == nil) - #expect(LiveTelemetrySampler.averageBitrateMbps(lifetimeBytes: 350_000, activeSeconds: 0) == nil) - #expect(LiveTelemetrySampler.averageBitrateMbps(lifetimeBytes: 0, activeSeconds: 30) == nil) + /// VOD, paused: the reader keeps topping up the buffer, the playhead does not move. + @Test("a VOD pause leaves both fields standing") + func vodPauseFreezes() { + let ledger = PlayedMediaLedger() + Self.feed(ledger, from: 0, to: 60) + var meter = PlayedBitrateMeter() + meter.advance(to: 0, wallSeconds: 0, ledger: ledger) + let playhead = Self.play(&meter, ledger, from: 0, seconds: 30) + let average = meter.averageMbps, instant = meter.instantMbps + Self.feed(ledger, from: 60, to: 120) + for _ in 0..<180 { meter.advance(to: playhead, wallSeconds: 1, ledger: ledger) } + #expect(meter.averageMbps == average) + #expect(meter.instantMbps == instant) } - // MARK: - The reported session shape - - /// The reporter's steps, folded tick by tick: 30 s of a 2.8 Mbps file, a three-minute pause during - /// which the demuxer counter stands still, then playback again. - @Test("a three-minute pause leaves the average flat instead of collapsing it") - func pauseDoesNotDragTheAverageDown() { - let bytesPerSecond: Int64 = 350_000 // 2.8 Mbps - var lifetimeBytes: Int64 = 0 - var activeSeconds: Double = 0 - var wallClockSeconds: Double = 0 - - func tick(_ phase: PlaybackPhase, bytes: Int64) { - lifetimeBytes += bytes - wallClockSeconds += 1 - if LiveTelemetrySampler.chargesActiveTime(phase) { activeSeconds += 1 } + /// Live, paused: the pump keeps draining the origin at the broadcast rate for the whole pause + /// (AE#443). Nothing is played, so nothing may move; on resume the backlog counts as it plays. + @Test("a live pause does not climb, and the backlog counts at its real rate once played") + func livePauseDoesNotClimb() { + let ledger = PlayedMediaLedger() + var meter = PlayedBitrateMeter() + var edge = 1_000.0 + Self.feed(ledger, from: edge, to: edge + 2) + meter.advance(to: edge, wallSeconds: 0, ledger: ledger) + var playhead = edge + for _ in 0..<30 { + edge += 1 + Self.feed(ledger, from: edge + 1, to: edge + 2) + playhead += 1 + meter.advance(to: playhead, wallSeconds: 1, ledger: ledger) } + let average = meter.averageMbps + #expect(Self.near(average, Self.mbps)) - for _ in 0..<30 { tick(.playing, bytes: bytesPerSecond) } - let beforePause = LiveTelemetrySampler.averageBitrateMbps( - lifetimeBytes: lifetimeBytes, activeSeconds: activeSeconds) - #expect(abs((beforePause ?? 0) - 2.8) < 0.001) - - // Paused: the forward buffer is full, so the demuxer counter does not move. - for _ in 0..<180 { tick(.paused, bytes: 0) } - let duringPause = LiveTelemetrySampler.averageBitrateMbps( - lifetimeBytes: lifetimeBytes, activeSeconds: activeSeconds) - #expect(duringPause == beforePause, "the pause must not move the average at all") - - // What used to ship, for the record: the same bytes over wall-clock time. - let wallClockAverage = Double(lifetimeBytes) * 8.0 / wallClockSeconds / 1_000_000.0 - #expect(wallClockAverage < 0.5, "the old divisor collapsed a 2.8 Mbps session to \(wallClockAverage)") - - // And on resume it is still the media's rate, not a value climbing back out of a hole. - for _ in 0..<30 { tick(.playing, bytes: bytesPerSecond) } - let afterResume = LiveTelemetrySampler.averageBitrateMbps( - lifetimeBytes: lifetimeBytes, activeSeconds: activeSeconds) - #expect(abs((afterResume ?? 0) - 2.8) < 0.001) - } + for _ in 0..<300 { + edge += 1 + Self.feed(ledger, from: edge + 1, to: edge + 2) + meter.advance(to: playhead, wallSeconds: 1, ledger: ledger) + } + #expect(meter.averageMbps == average, "five minutes of paused live must not move the average") - /// End of media is the other unbounded divisor: the sampler runs until the host tears the session - /// down, so a snapshot left on screen after the last frame used to decay exactly like a pause. - @Test("the average stops moving once the source has ended") - func endedSessionFreezesTheAverage() { - var activeSeconds: Double = 0 - for _ in 0..<30 where LiveTelemetrySampler.chargesActiveTime(.playing) { activeSeconds += 1 } - let atEnd = activeSeconds - for _ in 0..<120 where LiveTelemetrySampler.chargesActiveTime(.ended) { activeSeconds += 1 } - #expect(activeSeconds == atEnd) + _ = Self.play(&meter, ledger, from: playhead, seconds: 60) + #expect(Self.near(meter.averageMbps, Self.mbps)) + #expect(Self.near(meter.instantMbps, Self.mbps)) } - // MARK: - Wiring - - /// The two functions above are only right if the tick actually asks them. A paused session accrues - /// no active time however long its sampler runs; the same sampler starts accruing on play. - @Test("the running sampler charges no active time while the transport is paused") - func samplerAccruesNoActiveTimeWhilePaused() async throws { - let engine = try AetherEngine() - engine.playbackBackend = .native - let item = AVPlayerItem(url: URL(fileURLWithPath: "/nonexistent-514.mp4")) - engine.currentAVPlayer = AVPlayer(playerItem: item) - engine.state = .paused - #expect(engine.playbackPhase == .paused) - - let sampler = LiveTelemetrySampler(engine: engine, nativeRead: { _, _ in - NativeAVFReadings(forwardBufferSeconds: 12.0) - }) - sampler.start() - defer { sampler.stop() } - - // Long enough for several 1 Hz ticks to have run and charged nothing. - try await Task.sleep(for: .milliseconds(2_500)) - #expect(sampler.activeSeconds == 0) - - engine.state = .playing - let started = ContinuousClock().now - while sampler.activeSeconds == 0 { - if ContinuousClock().now - started > .seconds(30) { break } - try await Task.sleep(for: .milliseconds(50)) + /// Live, playing: this was right before by accident of the clock, and must stay right. + @Test("live at the edge reads the broadcast rate") + func livePlayingReadsTheRate() { + let ledger = PlayedMediaLedger() + var meter = PlayedBitrateMeter() + Self.feed(ledger, from: 50, to: 53) + meter.advance(to: 50, wallSeconds: 0, ledger: ledger) + var playhead = 50.0 + for _ in 0..<60 { + Self.feed(ledger, from: playhead + 3, to: playhead + 4) + playhead += 1 + meter.advance(to: playhead, wallSeconds: 1, ledger: ledger) } - #expect(sampler.activeSeconds > 0) + #expect(Self.near(meter.averageMbps, Self.mbps)) + #expect(Self.near(meter.instantMbps, Self.mbps)) + } + + // MARK: - Seeks + + /// The span a seek jumps over was never played, and the buffer the seek discarded is fetched again: + /// neither may land in the fields. + @Test("a seek charges nothing for the jump, and a re-fetched range counts once") + func seekAndRefetch() { + let ledger = PlayedMediaLedger() + Self.feed(ledger, from: 0, to: 120) + var meter = PlayedBitrateMeter() + meter.advance(to: 0, wallSeconds: 0, ledger: ledger) + _ = Self.play(&meter, ledger, from: 0, seconds: 20) + let secondsBefore = meter.lifetimeSeconds + + // Forward seek to 600: a new range is fetched from there. + Self.feed(ledger, from: 600, to: 700) + meter.advance(to: 600, wallSeconds: 1, ledger: ledger) + #expect(meter.lifetimeSeconds == secondsBefore) + _ = Self.play(&meter, ledger, from: 600, seconds: 10) + + // Back to 100: the reader delivers 100..200 again, over entries it already held. + Self.feed(ledger, from: 100, to: 200) + meter.advance(to: 100, wallSeconds: 1, ledger: ledger) + _ = Self.play(&meter, ledger, from: 100, seconds: 30) + + #expect(abs(meter.lifetimeSeconds - 60) < 1e-6) + #expect(Self.near(meter.averageMbps, Self.mbps)) + } + + /// A playhead the ledger holds nothing for (not fed there, or on another axis) is unmeasured. + @Test("a playhead off the ledger's span publishes nil, never zero") + func unmeasuredSpanIsNil() { + let ledger = PlayedMediaLedger() + Self.feed(ledger, from: 5_000, to: 5_100) + var meter = PlayedBitrateMeter() + meter.advance(to: 0, wallSeconds: 0, ledger: ledger) + _ = Self.play(&meter, ledger, from: 0, seconds: 30) + #expect(meter.averageMbps == nil) + #expect(meter.instantMbps == nil) + } + + // MARK: - The ledger + + @Test("the same packet delivered twice is held once") + func duplicatePacketOverwrites() { + let ledger = PlayedMediaLedger() + ledger.record(.video, pts: 1.0, bytes: 100) + ledger.record(.video, pts: 1.04, bytes: 100) + ledger.record(.video, pts: 1.0, bytes: 100) + #expect(ledger.entryCount == 2) + #expect(ledger.consume(from: 0, to: 2) == 200) + } + + @Test("B-frame reorder inserts in place, a re-read from far behind drops what lies ahead") + func reorderAndReread() { + let ledger = PlayedMediaLedger() + for pts in [0.0, 0.12, 0.04, 0.08, 0.24, 0.16, 0.20] { ledger.record(.video, pts: pts, bytes: 10) } + #expect(ledger.consume(from: 0.04, to: 0.20) == 40) + ledger.record(.video, pts: 10, bytes: 10) + ledger.record(.video, pts: 11, bytes: 10) + ledger.record(.video, pts: 5, bytes: 10) + #expect(ledger.consume(from: 5, to: 12) == 10, "10 and 11 lie ahead of the re-read and are gone") + } + + @Test("consuming a span forgets everything before it") + func consumePrunes() { + let ledger = PlayedMediaLedger() + Self.feed(ledger, from: 0, to: 10) + _ = ledger.consume(from: 4, to: 5) + #expect(ledger.consume(from: 0, to: 5) == 0) + #expect(ledger.consume(from: 5, to: 10) > 0) + } + + // MARK: - The playhead + + @Test("native folds AVPlayer's clock back with the producer's shift; routes without a pump read nil") + func ledgerPlayheadPerBackend() { + #expect(LiveTelemetrySampler.ledgerPlayhead( + backend: .native, nativeClock: 12, playlistShift: 3_600, softwareSourceClock: nil) == 3_612) + #expect(LiveTelemetrySampler.ledgerPlayhead( + backend: .native, nativeClock: nil, playlistShift: 3_600, softwareSourceClock: nil) == nil) + #expect(LiveTelemetrySampler.ledgerPlayhead( + backend: .software, nativeClock: 12, playlistShift: 3_600, softwareSourceClock: 90) == 90) + #expect(LiveTelemetrySampler.ledgerPlayhead( + backend: .audio, nativeClock: 12, playlistShift: 0, softwareSourceClock: 12) == nil) } } diff --git a/Tests/AetherEngineTests/Issue535RefusalNeedsEligibilityTests.swift b/Tests/AetherEngineTests/Issue535RefusalNeedsEligibilityTests.swift new file mode 100644 index 000000000..200bb906e --- /dev/null +++ b/Tests/AetherEngineTests/Issue535RefusalNeedsEligibilityTests.swift @@ -0,0 +1,56 @@ +import Foundation +import Testing +@testable import AetherEngine + +/// AE#535: a master refusal taken while the display is not eligible for HDR is not a fact about the panel. +/// +/// Measured by cmcpherson274 on an Apple TV 4K (tvOS 26.6): an audio route death right after a display +/// mode switch, a session-preserving reload landing 45 ms later with the readout at +/// `matching=off hdrEligible=no`, and AVPlayer failing the master with -11868. Three of four such reloads +/// latched `panelRefusedHDRMaster`, two of them in processes that never backgrounded, so the AE#588 clear +/// never came and every later HDR title in the session played media-direct. +struct Issue535RefusalNeedsEligibilityTests { + + @Test("A display rejection while the display is ineligible for HDR does not latch") + func ineligibleWindowDoesNotLatch() { + #expect(!MasterFallbackDecision.shouldLatchPanelRefusal(code: -11868, displayEligibleForHDRNow: false, displaySwitchInProgress: false)) + #expect(!MasterFallbackDecision.shouldLatchPanelRefusal(code: -11848, displayEligibleForHDRNow: false, displaySwitchInProgress: false)) + } + + @Test("A display rejection on an eligible display still latches") + func eligibleRefusalStillLatches() { + #expect(MasterFallbackDecision.shouldLatchPanelRefusal(code: -11868, displayEligibleForHDRNow: true, displaySwitchInProgress: false)) + #expect(MasterFallbackDecision.shouldLatchPanelRefusal(code: -11848, displayEligibleForHDRNow: true, displaySwitchInProgress: false)) + } + + /// -1002 is a manifest filtered at parse time (#130), a statement about the playlist, never the display. + @Test("The parse-time rejection never latches, eligible or not") + func parseRejectionNeverLatches() { + #expect(!MasterFallbackDecision.shouldLatchPanelRefusal(code: -1002, displayEligibleForHDRNow: true, displaySwitchInProgress: false)) + #expect(!MasterFallbackDecision.shouldLatchPanelRefusal(code: -1002, displayEligibleForHDRNow: false, displaySwitchInProgress: false)) + } + + /// The fallback itself is unaffected: the item still reloads the media playlist whatever the latch does. + @Test("An ineligible-window refusal still earns the item its media fallback") + func fallbackIsIndependentOfTheLatch() { + #expect(MasterFallbackDecision.shouldFallBackToMediaPlaylist( + errorCode: -11868, servingMasterPlaylist: true, alreadyFellBack: false)) + } + + /// A policy test cannot see the call site, so this one reads it: the latch is set only behind the gate. + @Test("The latch site asks the eligibility gate before setting the latch") + func latchSiteUsesTheGate() throws { + let source = URL(fileURLWithPath: #filePath) + .deletingLastPathComponent() + .deletingLastPathComponent() + .deletingLastPathComponent() + .appendingPathComponent("Sources/AetherEngine/AetherEngine.swift") + let text = try #require(try? String(contentsOf: source, encoding: .utf8)) + let fn = try #require(text.range(of: "func fallBackToMediaPlaylist(")) + let body = String(text[fn.lowerBound...].prefix(2500)) + let gate = try #require(body.range(of: "shouldLatchPanelRefusal(")) + let set = try #require(body.range(of: "Self.panelRefusedHDRMaster = true")) + #expect(gate.lowerBound < set.lowerBound) + #expect(body.contains("AVPlayer.eligibleForHDRPlayback")) + } +} diff --git a/Tests/AetherEngineTests/Issue541ReloadCarriesHDRRouteTests.swift b/Tests/AetherEngineTests/Issue541ReloadCarriesHDRRouteTests.swift index 8ac481a24..91478d7db 100644 --- a/Tests/AetherEngineTests/Issue541ReloadCarriesHDRRouteTests.swift +++ b/Tests/AetherEngineTests/Issue541ReloadCarriesHDRRouteTests.swift @@ -110,7 +110,7 @@ struct Issue541ReloadCarriesHDRRouteTests { .deletingLastPathComponent() .deletingLastPathComponent() .appendingPathComponent("Sources/AetherEngine/AetherEngine+Loading.swift") - guard let text = try? String(contentsOf: source, encoding: .utf8) else { return } + let text = try String(contentsOf: source, encoding: .utf8) #expect(!text.contains("panelIsInHDRMode: loadedOptions.panelIsInHDRMode")) #expect(text.contains("panelIsInHDRMode: reloadRoutingPanelHDR")) } diff --git a/Tests/AetherEngineTests/Issue587PreserveASSMarkupCodecGateTests.swift b/Tests/AetherEngineTests/Issue587PreserveASSMarkupCodecGateTests.swift index f58803995..c97ecf552 100644 --- a/Tests/AetherEngineTests/Issue587PreserveASSMarkupCodecGateTests.swift +++ b/Tests/AetherEngineTests/Issue587PreserveASSMarkupCodecGateTests.swift @@ -94,7 +94,7 @@ struct Issue587PreserveASSMarkupCodecGateTests { #expect(bodies == ["styled ass line", "second ass line"]) } - private static let base64 = [ + static let base64 = [ "GkXfo6NChoEBQveBAULygQRC84EIQoKIbWF0cm9za2FCh4EEQoWBAhhTgGcBAAAAAAAIqBFNm3TAv4QrfW8sTbuLU6uEFUmpZlOs", "gaFNu4tTq4QWVK5rU6yB8U27jFOrhBJUw2dTrIID9k27jFOrhBxTu2tTrIIIPOwBAAAAAAAAUwAAAAAAAAAAAAAAAAAAAAAAAAAA", "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAFUmpZsu/hGgSNtMq", diff --git a/Tests/AetherEngineTests/Issue592PacketEnvelopeTests.swift b/Tests/AetherEngineTests/Issue592PacketEnvelopeTests.swift index 839fb861c..441533680 100644 --- a/Tests/AetherEngineTests/Issue592PacketEnvelopeTests.swift +++ b/Tests/AetherEngineTests/Issue592PacketEnvelopeTests.swift @@ -48,6 +48,43 @@ struct Issue592PacketEnvelopeTests { #expect(try SoftwareStoredPacket.decode(p.encoded()) == p) } + /// The binary property list encoder uniqued every object through a `Set`, which hashed the + /// whole payload on every packet: 92 % of the envelope's encode time in a release profile of a + /// 91 Mbit/s HEVC session. The envelope is now the payload plus a fixed header, so its size + /// says whether anything else crept back in. + @Test("the envelope is a fixed header, the payload and the side data, nothing else") + func envelopeSizeIsExact() throws { + let p = packet(payload: 425_000, sideData: 3) + let side = p.sideData.reduce(0) { $0 + SoftwareStoredPacket.sideDataHeaderBytes + $1.bytes.count } + #expect(try p.encoded().count == SoftwareStoredPacket.headerBytes + 425_000 + side) + } + + @Test("a truncated envelope throws instead of reading past its end") + func truncatedEnvelopeThrows() throws { + let data = try packet(payload: 4_096, sideData: 2).encoded() + for cut in [0, 1, SoftwareStoredPacket.headerBytes - 1, SoftwareStoredPacket.headerBytes + 10, data.count - 1] { + #expect(throws: (any Error).self) { try SoftwareStoredPacket.decode(data.prefix(cut)) } + } + } + + @Test("an envelope of another version or with trailing bytes throws") + func foreignEnvelopeThrows() throws { + var data = try packet(payload: 16).encoded() + var trailing = data + trailing.append(0) + #expect(throws: (any Error).self) { try SoftwareStoredPacket.decode(trailing) } + data[data.startIndex] = 0xFF + #expect(throws: (any Error).self) { try SoftwareStoredPacket.decode(data) } + } + + @Test("a slice decodes like the whole buffer") + func sliceDecodes() throws { + let p = packet(payload: 777, sideData: 1) + var framed = Data([1, 2, 3]) + framed.append(try p.encoded()) + #expect(try SoftwareStoredPacket.decode(framed.dropFirst(3)) == p) + } + /// Reported, not asserted: a throughput floor pinned here would be a CI coin toss. The number is /// the deliverable, read off the run. Each round gets its own payload, so nothing is reused /// across iterations and a copy the encoder makes is a copy this measures. diff --git a/Tests/AetherEngineTests/Issue597MediaServicesResetTests.swift b/Tests/AetherEngineTests/Issue597MediaServicesResetTests.swift index 8ecb0da31..78fe43ed6 100644 --- a/Tests/AetherEngineTests/Issue597MediaServicesResetTests.swift +++ b/Tests/AetherEngineTests/Issue597MediaServicesResetTests.swift @@ -1,3 +1,4 @@ +import Foundation import Testing @testable import AetherEngine @@ -29,4 +30,72 @@ struct Issue597MediaServicesResetTests { #expect(!AetherEngine.shouldPreserveNativeHostAcrossLoad( backend: .software, nativeHostSurvives: true, mediaServicesWereReset: true)) } + + private static let missingSource = URL(fileURLWithPath: "/nonexistent/aether-597-reset.m4a") + + /// Audit CORE-4: the audio-only host is kept across every teardown so its Now Playing session + /// persists between tracks, and the reset only ever dropped the video host. Music after a reset + /// went to the dead AVPlayer, and the first load of any kind cleared the flag. + @Test("the load that consumes a reset drops the audio-only player too") + @MainActor + func resetDropsTheAudioOnlyPlayer() async throws { + let engine = try AetherEngine() + engine.audioAVPlayerHost = AudioAVPlayerHost() + engine.noteMediaServicesReset(lost: false) + + _ = try? await engine.load(url: Self.missingSource) + + #expect(engine.audioAVPlayerHost == nil) + #expect(!engine.consumeMediaServicesReset()) + } + + /// Audit LIF-104: `reloadWithAudioOverride` (the audio pick, the disc-title pick, the custom-source + /// reload) keeps the native host on the native route, and it never read the flag, so the first + /// rebuild after a reset mounted the item on the invalidated AVPlayer while the flag stayed up. + @Test("a custom-source rebuild after a reset builds a fresh native player and consumes the reset", + .timeLimit(.minutes(2))) + @MainActor + func customRebuildConsumesTheReset() async throws { + let engine = try AetherEngine() + defer { engine.stop() } + _ = try await engine.load( + source: .custom(DataIOReader(data: try ProbeTestFixtures.hdr10Plus()), formatHint: "mp4")) + let invalidated = try #require(engine.nativeHost) + engine.audioAVPlayerHost = AudioAVPlayerHost() + engine.noteMediaServicesReset(lost: false) + + try await engine.reloadAtCurrentPosition() + + #expect(engine.nativeHost != nil) + #expect(engine.nativeHost !== invalidated) + #expect(engine.audioAVPlayerHost == nil) + #expect(!engine.consumeMediaServicesReset()) + } + + @Test("without a reset a custom-source rebuild keeps its native player (issue #15)", + .timeLimit(.minutes(2))) + @MainActor + func customRebuildKeepsTheHostWithoutAReset() async throws { + let engine = try AetherEngine() + defer { engine.stop() } + _ = try await engine.load( + source: .custom(DataIOReader(data: try ProbeTestFixtures.hdr10Plus()), formatHint: "mp4")) + let host = try #require(engine.nativeHost) + + try await engine.reloadAtCurrentPosition() + + #expect(engine.nativeHost === host) + } + + @Test("without a reset the audio-only player survives a load, as its Now Playing session must") + @MainActor + func audioOnlyPlayerSurvivesWithoutAReset() async throws { + let engine = try AetherEngine() + let host = AudioAVPlayerHost() + engine.audioAVPlayerHost = host + + _ = try? await engine.load(url: Self.missingSource) + + #expect(engine.audioAVPlayerHost === host) + } } diff --git a/Tests/AetherEngineTests/Issue605SoftwareVODStillTests.swift b/Tests/AetherEngineTests/Issue605SoftwareVODStillTests.swift new file mode 100644 index 000000000..dbc01f84d --- /dev/null +++ b/Tests/AetherEngineTests/Issue605SoftwareVODStillTests.swift @@ -0,0 +1,218 @@ +import Foundation +import Testing +@testable import AetherEngine + +/// AE#605: a software VOD session decodes scrub stills out of the packet cache its seeks already +/// land in, instead of opening a second connection a single-slot origin refuses. +/// +/// Two properties carry the feature. The run a still needs has to be read from HISTORY without +/// moving the consumer, or every preview would rewind playback. And the run is offered exactly where +/// a commit would be a cache hit: a target past the retained frontier has a real frame the store +/// does not hold yet, so the frame before it is the wrong answer rather than a close one. +@Suite("Software VOD scrub stills from the packet cache (AE#605)") +struct Issue605SoftwareVODStillTests { + + // MARK: - FIFO history walk + + private static func makeRoot() throws -> URL { + let root = FileManager.default.temporaryDirectory + .appendingPathComponent("issue605-\(UUID().uuidString)", isDirectory: true) + try FileManager.default.createDirectory(at: root, withIntermediateDirectories: true) + return root + } + + private static func record(_ value: UInt8) -> Data { Data(repeating: value, count: 20) } + + @Test("A history walk crosses chunks and leaves the consumer where it stood") + func historyWalkDoesNotMoveConsumer() throws { + let root = try Self.makeRoot() + defer { try? FileManager.default.removeItem(at: root) } + // 64-byte chunks hold two 28-byte records, so ten records span five chunks. + let fifo = try SoftwarePacketDiskFIFO(chunkTargetBytes: 64, retainConsumed: true, + parentDirectory: root) + var cursors: [SoftwarePacketDiskFIFO.Cursor] = [] + for value in UInt8(0)..<10 { cursors.append(try fifo.append(Self.record(value))) } + + var seen: [UInt8] = [] + try fifo.readHistory(from: cursors[3]) { data in + seen.append(data.first ?? 255) + return true + } + #expect(seen == Array(3..<10)) + #expect(try fifo.pop() == Self.record(0)) + #expect(fifo.snapshot.count == 9) + } + + @Test("A history walk stops when the visitor says so") + func historyWalkStopsEarly() throws { + let root = try Self.makeRoot() + defer { try? FileManager.default.removeItem(at: root) } + let fifo = try SoftwarePacketDiskFIFO(chunkTargetBytes: 64, retainConsumed: true, + parentDirectory: root) + var cursors: [SoftwarePacketDiskFIFO.Cursor] = [] + for value in UInt8(0)..<10 { cursors.append(try fifo.append(Self.record(value))) } + + var seen: [UInt8] = [] + try fifo.readHistory(from: cursors[1]) { data in + seen.append(data.first ?? 255) + return seen.count < 3 + } + #expect(seen == [1, 2, 3]) + } + + @Test("A cursor from before a reset is refused, and the store stays usable") + func historyWalkRefusesStaleCursor() throws { + let root = try Self.makeRoot() + defer { try? FileManager.default.removeItem(at: root) } + let fifo = try SoftwarePacketDiskFIFO(chunkTargetBytes: 64, retainConsumed: true, + parentDirectory: root) + let stale = try fifo.append(Self.record(1)) + try fifo.reset() + try fifo.append(Self.record(2)) + + #expect(throws: SoftwarePacketDiskFIFO.Failure.invalidCursor) { + try fifo.readHistory(from: stale) { _ in true } + } + #expect(!fifo.snapshot.hasFailure) + #expect(try fifo.pop() == Self.record(2)) + } + + @Test("A cursor whose chunk was evicted is refused") + func historyWalkRefusesEvictedCursor() throws { + let root = try Self.makeRoot() + defer { try? FileManager.default.removeItem(at: root) } + let fifo = try SoftwarePacketDiskFIFO(chunkTargetBytes: 64, retainConsumed: true, + parentDirectory: root) + var cursors: [SoftwarePacketDiskFIFO.Cursor] = [] + for value in UInt8(0)..<10 { cursors.append(try fifo.append(Self.record(value))) } + for _ in 0..<6 { _ = try fifo.pop() } + try fifo.trimConsumed(toByteBudget: 0) + + #expect(throws: SoftwarePacketDiskFIFO.Failure.invalidCursor) { + try fifo.readHistory(from: cursors[0]) { _ in true } + } + var seen: [UInt8] = [] + try fifo.readHistory(from: cursors[6]) { seen.append($0.first ?? 255); return true } + #expect(seen == Array(6..<10)) + } + + // MARK: - Still run planning + + /// 25 pictures a second on a 1/1000 time base, a keyframe every second, and a non-video packet + /// after every picture so the run has something to filter out. + private static let tickRate: Int32 = 1000 + private static let holdTicks: Int64 = 40 + private static let pictureRate = 25 + + private final class Counter: @unchecked Sendable { + private let lock = NSLock() + private var count = 0 + func next() -> Int { lock.lock(); defer { lock.unlock() }; count += 1; return count } + var value: Int { lock.lock(); defer { lock.unlock() }; return count } + } + + private static func sourcePacket(index: Int, keyframeEvery: Int) -> SoftwareStoredPacket { + let picture = (index - 1) / 2 + let pts = Int64(picture) * holdTicks + let isVideo = (index - 1) % 2 == 0 + let isKey = isVideo && picture % keyframeEvery == 0 + return SoftwareStoredPacket(pts: pts, dts: pts, duration: holdTicks, position: 0, + streamIndex: isVideo ? 0 : 5, flags: isKey ? 0x1 : 0, + timeBaseNumerator: 1, timeBaseDenominator: tickRate, + bytes: Data(repeating: 7, count: 100), sideData: []) + } + + /// A producer that has filled its forward window and parked, with no consumer yet. + private static func parkedCache(root: URL, forwardSeconds: Double, + keyframeEvery: Int = pictureRate) throws -> SoftwarePacketReadAhead { + let counter = Counter() + let fifo = try SoftwarePacketDiskFIFO(chunkTargetBytes: 64 * 1024, retainConsumed: true, + parentDirectory: root) + let cache = SoftwarePacketReadAhead( + video: .init(index: 0, numerator: 1, denominator: tickRate), audio: nil, + byteBudget: 50_000_000, forwardSeconds: forwardSeconds, initialSourceClock: 0, + fifo: fifo + ) { _ in Self.sourcePacket(index: counter.next(), keyframeEvery: keyframeEvery) } + cache.start() + let deadline = Date().addingTimeInterval(10) + var previous = -1 + while Date() < deadline { + Thread.sleep(forTimeInterval: 0.15) + let now = counter.value + if now == previous, now > 0 { break } + previous = now + } + return cache + } + + private static func seconds(_ packet: SoftwareStoredPacket) -> Double { + Double(packet.pts) / Double(tickRate) + } + + @Test("A run opens on the keyframe before the target and reaches past it, video only") + func runOpensOnKeyframeAndReachesTarget() throws { + let root = try Self.makeRoot() + defer { try? FileManager.default.removeItem(at: root) } + let cache = try Self.parkedCache(root: root, forwardSeconds: 10) + defer { cache.close() } + + let run = try #require(cache.stillRun(atSeconds: 3.5, maxPackets: 900, + maxSpanSeconds: 12, reorderTail: 4)) + #expect(run.first.map(Self.seconds) == 3.0) + #expect(run.first.map { $0.flags & 1 != 0 } == true) + #expect(run.allSatisfy { $0.streamIndex == 0 }) + // 3.00 through 3.52 is fourteen pictures (the first at or past 3.5), plus the tail of four. + #expect(run.count == 18) + #expect(run.contains { Self.seconds($0) >= 3.5 }) + } + + @Test("A still leaves the consumer at the packet it would have read next") + func runDoesNotMoveConsumer() throws { + let root = try Self.makeRoot() + defer { try? FileManager.default.removeItem(at: root) } + let cache = try Self.parkedCache(root: root, forwardSeconds: 10) + defer { cache.close() } + + #expect(cache.stillRun(atSeconds: 6.2, maxPackets: 900, maxSpanSeconds: 12, + reorderTail: 4) != nil) + let next = try #require(try cache.read()) + #expect(next.pts == 0) + #expect(next.streamIndex == 0) + } + + @Test("A target past the retained frontier gets no still, not the frame before it") + func runRefusesPastFrontier() throws { + let root = try Self.makeRoot() + defer { try? FileManager.default.removeItem(at: root) } + let cache = try Self.parkedCache(root: root, forwardSeconds: 10) + defer { cache.close() } + + #expect(cache.stillRun(atSeconds: 60, maxPackets: 900, maxSpanSeconds: 12, + reorderTail: 4) == nil) + } + + @Test("A keyframe further back than the span bound gets no still") + func runRefusesWideSpan() throws { + let root = try Self.makeRoot() + defer { try? FileManager.default.removeItem(at: root) } + // One keyframe every eight seconds. + let cache = try Self.parkedCache(root: root, forwardSeconds: 10, keyframeEvery: 200) + defer { cache.close() } + + #expect(cache.stillRun(atSeconds: 5, maxPackets: 900, maxSpanSeconds: 2, + reorderTail: 4) == nil) + #expect(cache.stillRun(atSeconds: 5, maxPackets: 900, maxSpanSeconds: 12, + reorderTail: 4) != nil) + } + + @Test("A run longer than the packet bound gets no still") + func runRefusesPacketOverflow() throws { + let root = try Self.makeRoot() + defer { try? FileManager.default.removeItem(at: root) } + let cache = try Self.parkedCache(root: root, forwardSeconds: 10) + defer { cache.close() } + + #expect(cache.stillRun(atSeconds: 0.9, maxPackets: 10, maxSpanSeconds: 12, + reorderTail: 4) == nil) + } +} diff --git a/Tests/AetherEngineTests/Issue613PacketCoverageTests.swift b/Tests/AetherEngineTests/Issue613PacketCoverageTests.swift new file mode 100644 index 000000000..21b440d98 --- /dev/null +++ b/Tests/AetherEngineTests/Issue613PacketCoverageTests.swift @@ -0,0 +1,160 @@ +import Foundation +import AetherLibavcodec +import Testing +@testable import AetherEngine + +/// #613, found while answering #592: the software VOD read-ahead's packet coverage broke in two +/// ways that both leave the frontier describing less than the store holds. +/// +/// HEVC kept the strict `pts + duration` model, and a Matroska file muxed with 41 ms durations +/// against 41/42 ms deltas opens a one-tick hole at every 42 ms step: 0.70 islands a frame, +/// `bufferedPosition` stuck at the end of the current island. And a coverage that reached its range +/// cap stopped describing new packets (plain) or invalidated itself (successor model) instead of +/// forgetting the history behind the playhead. +@Suite("Software VOD packet coverage (#613)") +struct Issue613PacketCoverageTests { + + private static let tickRate: Int32 = 1000 + + /// 23.976 fps on a 1/1000 time base, the fixture's own timestamps: PTS rounded to the + /// millisecond, every duration 41, so each 42 ms delta leaves a one-tick hole. + private static func fragmentedPTS(_ index: Int) -> Int64 { + Int64((Double(index) * 1000 * 1001 / 24000).rounded()) + } + + private static func packet(pts: Int64, duration: Int64 = 41) -> SoftwareStoredPacket { + SoftwareStoredPacket(pts: pts, dts: pts, duration: duration, position: 0, + streamIndex: 0, flags: 0x1, timeBaseNumerator: 1, + timeBaseDenominator: Self.tickRate, + bytes: Data(repeating: 7, count: 100), sideData: []) + } + + private final class Counter: @unchecked Sendable { + private let lock = NSLock() + private var count = 0 + func next() -> Int { lock.lock(); defer { lock.unlock() }; count += 1; return count } + } + + private func makeReadAhead( + root: URL, reorderDepth: Int?, forwardSeconds: Double, rangeCap: Int = 4096, + pts: @escaping @Sendable (Int) -> Int64 + ) throws -> SoftwarePacketReadAhead { + let counter = Counter() + let fifo = try SoftwarePacketDiskFIFO(chunkTargetBytes: 64 * 1024, retainConsumed: true, + parentDirectory: root) + return SoftwarePacketReadAhead( + video: .init(index: 0, numerator: 1, denominator: Self.tickRate), audio: nil, + byteBudget: 50_000_000, forwardSeconds: forwardSeconds, initialSourceClock: 0, + fifo: fifo, videoReorderDepth: reorderDepth, coverageRangeCap: rangeCap + ) { _ in Self.packet(pts: pts(counter.next() - 1)) } + } + + private func withRoot(_ body: (URL) throws -> T) throws -> T { + let root = FileManager.default.temporaryDirectory + .appendingPathComponent("issue613-\(UUID().uuidString)", isDirectory: true) + try FileManager.default.createDirectory(at: root, withIntermediateDirectories: true) + defer { try? FileManager.default.removeItem(at: root) } + return try body(root) + } + + /// Waits for the producer to park, read as a packet count that stops moving. + private func waitForPark(_ source: SoftwarePacketReadAhead) { + let deadline = Date().addingTimeInterval(10) + var previous = -1 + while Date() < deadline { + Thread.sleep(forTimeInterval: 0.15) + let now = source.snapshot.packetCount + if now == previous, now > 0 { return } + previous = now + } + } + + // MARK: - 1. HEVC gets the successor model + + /// The bound that lets HEVC in: FFmpeg rejects more than 15 reorder pictures, inside the + /// 32-timestamp queue. Codecs without an established bound keep the strict model. + @Test("H.264 and HEVC take the successor model, other codecs keep the duration model") + func reorderDepthByCodec() { + #expect(SoftwarePlaybackHost.presentationReorderDepth( + codecID: AV_CODEC_ID_H264.rawValue) == 32) + #expect(SoftwarePlaybackHost.presentationReorderDepth( + codecID: AV_CODEC_ID_HEVC.rawValue) == 32) + for other in [AV_CODEC_ID_AV1, AV_CODEC_ID_VP9, AV_CODEC_ID_MPEG2VIDEO, AV_CODEC_ID_VC1] { + #expect(SoftwarePlaybackHost.presentationReorderDepth(codecID: other.rawValue) == nil) + } + } + + /// The reported shape. On the duration model the frontier ends at the first rounding hole, 41 ms + /// in; on the successor model it reaches what the producer stored before it parked. + @Test("Rounded durations do not fragment the successor model") + func roundedDurationsStayOneIsland() throws { + try withRoot { root in + let strict = try makeReadAhead(root: root, reorderDepth: nil, forwardSeconds: 10, + pts: Self.fragmentedPTS) + strict.start() + waitForPark(strict) + let strictFrontier = strict.snapshot.frontier + strict.close() + + let successor = try makeReadAhead(root: root, reorderDepth: 32, forwardSeconds: 10, + pts: Self.fragmentedPTS) + successor.start() + waitForPark(successor) + let successorFrontier = successor.snapshot.frontier + successor.close() + + #expect((strictFrontier ?? 0) < 0.05, "the premise: the duration model splits at 41 ms") + // Ten seconds stored, less the 32 timestamps the reorder queue holds back (1.33 s). + #expect((successorFrontier ?? 0) > 8, "the frontier must reach the store, not the first hole") + } + } + + // MARK: - 2. A full coverage forgets history instead of freezing + + /// Consumes `count` packets, moving the playhead onto each, and answers the frontier at the end. + /// Every read waits for 40 stored packets first, the way a real consumer trails the producer: + /// the successor model describes a picture only once 32 later timestamps have arrived, so a + /// consumer that drains the store as fast as it fills sees no coverage at all. + private func frontierAfterConsuming(_ count: Int, source: SoftwarePacketReadAhead) throws -> Double? { + source.start() + for _ in 0.. [UInt8] { + let n = count ?? window.count - offset + var out = [UInt8](repeating: 0, count: n) + out.withUnsafeMutableBufferPointer { window.copyBytes(to: $0.baseAddress!, from: offset, count: n) } + return out + } + + private func chunk(_ start: Int, _ count: Int) -> Data { + Data((0.. [UInt8]? { + var out = [UInt8](repeating: 0, count: count) + var got = 0 + let ok = out.withUnsafeMutableBufferPointer { buffer -> Bool in + while got < count { + let n = reader.read(into: buffer.baseAddress! + got, size: Int32(min(64 * 1024, count - got))) + if n <= 0 { return false } + got += Int(n) + } + return true + } + return ok ? out : nil + } + + private func expected(_ count: Int, at offset: Int64) -> [UInt8] { + (0.. (rgba: [UInt8], x: Int, y: Int, w: Int, h: Int)? { + var minX = width, minY = height, maxX = -1, maxY = -1 + for y in 0..= 8 { + minX = min(minX, x); minY = min(minY, y); maxX = max(maxX, x); maxY = max(maxY, y) + } + } + guard maxX >= minX, maxY >= minY else { return nil } + let cropW = maxX - minX + 1, cropH = maxY - minY + 1 + var rgba = [UInt8](repeating: 0, count: cropW * cropH * 4) + for cy in 0.. [UInt8] { + let data = image.dataProvider!.data! as Data + var out: [UInt8] = [] + for row in 0.. width: the padding must never be read + var palette = [UInt8](repeating: 0, count: 256 * 4) + for i in 0..<256 { + for c in 0..<4 { palette[i * 4 + c] = UInt8(truncatingIfNeeded: rng.next()) } + } + // Around the threshold on purpose: 7 is invisible, 8 is not. + palette[0 * 4 + 3] = 0; palette[1 * 4 + 3] = 7; palette[2 * 4 + 3] = 8; palette[3 * 4 + 3] = 255 + var pixels = [UInt8](repeating: 0, count: stride * height) + for y in 8..<30 { + for x in 11..<83 { pixels[y * stride + x] = UInt8(truncatingIfNeeded: rng.next()) } + } + for y in 0.. (AetherEngine, SubtitlePacketStore, Demuxer) { + let data = try #require(Data(base64Encoded: Issue587PreserveASSMarkupCodecGateTests.base64.joined())) + let demuxer = Demuxer() + try demuxer.open(reader: DataIOReader(data: data), formatHint: "matroska") + let store = SubtitlePacketStore() + let stream = try #require(demuxer.stream(at: Self.subripStreamIndex)) + while let pkt = try? demuxer.readPacket() { + if pkt.pointee.stream_index == Self.subripStreamIndex { + store.harvest(streamIndex: Self.subripStreamIndex, packet: pkt, + timeBase: stream.pointee.time_base) + } + var p: UnsafeMutablePointer? = pkt + trackedPacketFree(&p) + } + let engine = try AetherEngine() + engine.loadedURL = URL(string: "https://s/movie.mkv")! + engine.softwareSubtitlePacketStore = store + engine.isSubtitleActive = true + engine.subtitleDrainTargets[.primary] = Self.subripStreamIndex + engine.subtitleDrainDecoderFactoryForTesting = { index in + guard let stream = demuxer.stream(at: index) else { return nil } + return EmbeddedSubtitleDecoder(stream: stream, sourceVideoWidth: 16, sourceVideoHeight: 16) + } + engine.clock.sourceTime = 1.5 + return (engine, store, demuxer) + } + + @MainActor + @Test("a requested drain tick decodes off the MainActor and applies when it lands") + func requestedTickDecodesOffMain() async throws { + let (engine, _, demuxer) = try engineWithHarvestedSubRip() + defer { demuxer.close() } + + engine.requestSubtitleDrainTick() + // Nothing decoded on the caller's stack: the batch is in flight, not applied. + let inFlight = try #require(engine.subtitleDrainTickInFlight) + #expect(engine.subtitleCues.isEmpty) + #expect(engine.subtitleDrainCursors[.primary] == nil) + + await inFlight.value + #expect(engine.subtitleDrainTickInFlight == nil) + #expect(engine.subtitleCues.map(\.startTime) == [1, 3]) + #expect(engine.subtitleDrainCursors[.primary] != nil) + } + + /// Every publish of `subtitleCues`, stamped with the drain tick that was current when it happened. + @MainActor + private final class CuePublications { + var stamps: [(serial: UInt64, starts: [Double])] = [] + } + + @MainActor + @Test("a batch whose channel was re-selected while it decoded is dropped, and the tick runs again") + func reselectionDropsTheStaleBatch() async throws { + let (engine, _, demuxer) = try engineWithHarvestedSubRip() + defer { demuxer.close() } + // Observed per publish rather than sampled after the await: the queued tick starts inside the + // stale batch's landing and its own decode can land before this test resumes, so "still in + // flight" and "nothing published yet" are scheduler outcomes, not the engine's (a CI run took + // 65 s and caught the rerun already finished). + let publications = CuePublications() + let observation = engine.$subtitleCues.dropFirst().sink { cues in + MainActor.assumeIsolated { + publications.stamps.append((engine.subtitleDrainTickSerial, cues.map(\.startTime))) + } + } + defer { observation.cancel() } + + engine.requestSubtitleDrainTick() + let stale = try #require(engine.subtitleDrainTickInFlight) + let staleSerial = engine.subtitleDrainTickSerial + // What a selection does to the channel (selectSubtitleTrack): a fresh decoder and cursor. + engine.subtitleDrainDecoders[.primary] = nil + engine.subtitleDrainCursors[.primary] = nil + engine.subtitleDrainTick() // arrives while the stale batch is in flight: queued, not run + #expect(engine.subtitleDrainTickRequested) + + await stale.value + // The landing dropped its batch and started the queued tick, whether or not that one has + // landed by now. + #expect(engine.subtitleDrainTickSerial == staleSerial &+ 1) + #expect(!engine.subtitleDrainTickRequested) + if let rerun = engine.subtitleDrainTickInFlight { await rerun.value } + #expect(!publications.stamps.contains { $0.serial == staleSerial }, + "the stale batch published nothing") + #expect(engine.subtitleCues.map(\.startTime) == [1, 3]) + } + + @MainActor + @Test("stopping the drainer while a batch decodes discards it") + func stopDiscardsTheInFlightBatch() async throws { + let (engine, _, demuxer) = try engineWithHarvestedSubRip() + defer { demuxer.close() } + + engine.requestSubtitleDrainTick() + let inFlight = try #require(engine.subtitleDrainTickInFlight) + engine.clearSubtitleDrainTarget(channel: .primary, reason: .subtitlesCleared) + #expect(engine.subtitleDrainTickInFlight == nil) + + await inFlight.value + #expect(engine.subtitleCues.isEmpty) + #expect(engine.subtitleDrainCursors.isEmpty) + } +} + +/// Deterministic generator so a failing seed reproduces. +private struct SplitMix { + var state: UInt64 + init(seed: UInt64) { state = seed } + mutating func next() -> UInt64 { + state &+= 0x9E3779B97F4A7C15 + var z = state + z = (z ^ (z >> 30)) &* 0xBF58476D1CE4E5B9 + z = (z ^ (z >> 27)) &* 0x94D049BB133111EB + return z ^ (z >> 31) + } +} diff --git a/Tests/AetherEngineTests/Issue629MountClockHoldTests.swift b/Tests/AetherEngineTests/Issue629MountClockHoldTests.swift new file mode 100644 index 000000000..5dedede41 --- /dev/null +++ b/Tests/AetherEngineTests/Issue629MountClockHoldTests.swift @@ -0,0 +1,55 @@ +import Foundation +import Testing +@testable import AetherEngine + +/// AE#629 round 5 (cmcpherson274, 2026-10-01): the #646 clock hold covered only an in-place swap. A VOD +/// item a host `load()` mounted at 15.90 s published AVPlayer's pre-landing reading, the start of the +/// segment it decodes up from (12.00 s), and once that load had returned nothing parked the 15.90 s any +/// more. AVPlayer refused the item before the mount seek landed, the software rebuild resumed at 12.00 s +/// and the film replayed 3.9 s after the rescue. +@Suite("A mount holds the clock until its mount seek lands") +struct Issue629MountClockHoldTests { + + @Test("Every seeking swap holds, a fresh mount only on VOD past the head") + func whichMountsHold() { + #expect(NativeAVPlayerHost.mountHoldsClock(inPlaceSwap: true, skipInitialSeek: false, + startPosition: 15.9, isLive: false)) + #expect(NativeAVPlayerHost.mountHoldsClock(inPlaceSwap: true, skipInitialSeek: false, + startPosition: nil, isLive: true)) + #expect(NativeAVPlayerHost.mountHoldsClock(inPlaceSwap: false, skipInitialSeek: false, + startPosition: 15.9, isLive: false)) + #expect(!NativeAVPlayerHost.mountHoldsClock(inPlaceSwap: false, skipInitialSeek: false, + startPosition: 0, isLive: false)) + #expect(!NativeAVPlayerHost.mountHoldsClock(inPlaceSwap: false, skipInitialSeek: false, + startPosition: nil, isLive: false)) + #expect(!NativeAVPlayerHost.mountHoldsClock(inPlaceSwap: false, skipInitialSeek: false, + startPosition: 15.9, isLive: true)) + #expect(!NativeAVPlayerHost.mountHoldsClock(inPlaceSwap: true, skipInitialSeek: true, + startPosition: nil, isLive: true)) + } + + @Test("A fresh VOD mount past the head reads its mount position before the item answers") + @MainActor + func freshMountStatesItsPosition() { + let host = NativeAVPlayerHost() + host.load(url: URL(string: "http://127.0.0.1:1/vod/media.m3u8")!, startPosition: 15.9, + contract: .init()) + #expect(host.currentTime == 15.9) + #expect(host.renderedTime == 15.9) + // What the escalation rung resumes at once the host's load has returned. + #expect(AetherEngine.rebuildPosition(state: .playing, clock: host.currentTime, + underReconstruction: 15.9) == 15.9) + } + + @Test("A mount at the head and a live mount publish nothing ahead of the item") + @MainActor + func otherMountsWaitForTheItem() { + let host = NativeAVPlayerHost() + host.load(url: URL(string: "http://127.0.0.1:1/vod/media.m3u8")!, startPosition: 0, + contract: .init()) + #expect(host.currentTime == 0) + host.load(url: URL(string: "http://127.0.0.1:1/live/media.m3u8")!, startPosition: 30, + contract: .init(isLive: true)) + #expect(host.currentTime == 0) + } +} diff --git a/Tests/AetherEngineTests/Issue641LiveUndecodableAudioTests.swift b/Tests/AetherEngineTests/Issue641LiveUndecodableAudioTests.swift new file mode 100644 index 000000000..2cbecadc7 --- /dev/null +++ b/Tests/AetherEngineTests/Issue641LiveUndecodableAudioTests.swift @@ -0,0 +1,185 @@ +import Testing +import Foundation +import AetherLibavcodec +import AetherLibavutil +@testable import AetherEngine + +/// AE#641: a live bridge that decodes nothing has to reach the engine, which rebuilds the session +/// video-only. A FLAC sample entry is built from the encoder's extradata, so on live there is no +/// muxer death to learn it from and AVPlayer would wait on the empty audio track forever. +struct Issue641LiveUndecodableAudioTests { + + private final class Calls: @unchecked Sendable { + private let lock = NSLock() + private var stats: [AudioBridge.FeedStats] = [] + func append(_ s: AudioBridge.FeedStats) { lock.withLock { stats.append(s) } } + var all: [AudioBridge.FeedStats] { lock.withLock { stats } } + } + + private func freeAll(_ packets: inout [UnsafeMutablePointer]) { + for p in packets { + var pp: UnsafeMutablePointer? = p + trackedPacketFree(&pp) + } + packets.removeAll() + } + + private func makeCodecpar(_ id: AVCodecID) -> UnsafeMutablePointer { + let par = avcodec_parameters_alloc()! + par.pointee.codec_type = AVMEDIA_TYPE_AUDIO + par.pointee.codec_id = id + par.pointee.sample_rate = 48_000 + if id == AV_CODEC_ID_PCM_S16LE { + par.pointee.format = AV_SAMPLE_FMT_S16.rawValue + par.pointee.bits_per_coded_sample = 16 + par.pointee.block_align = 4 + } else { + par.pointee.format = AV_SAMPLE_FMT_FLTP.rawValue + } + av_channel_layout_default(&par.pointee.ch_layout, 2) + return par + } + + /// Bytes the mp3 decoder cannot make a frame of, the shape a mislabelled PID hands the bridge. + private func makeUndecodablePackets(count: Int) -> [UnsafeMutablePointer] { + (0..= 0 else { return nil } + memset(pkt.pointee.data, 0xFF, 130) + pkt.pointee.pts = Int64(i) * 26 + pkt.pointee.dts = pkt.pointee.pts + return pkt + } + } + + /// Stereo S16 PCM, 1152 samples a packet: decodes, and fills FLAC's 4608-sample frame every fourth. + private func makePCMPackets(count: Int) -> [UnsafeMutablePointer] { + let samples = 1152 + return (0..= 0 else { return nil } + memset(pkt.pointee.data, 0, samples * 4) + pkt.pointee.pts = Int64(i * samples) + pkt.pointee.dts = pkt.pointee.pts + return pkt + } + } + + private func feed(_ codec: AVCodecID, packets: [UnsafeMutablePointer], + timeBase: AVRational) throws -> (Calls, AudioBridge.FeedStats) { + let codecpar = makeCodecpar(codec) + defer { + var p: UnsafeMutablePointer? = codecpar + avcodec_parameters_free(&p) + } + let bridge = try AudioBridge(srcCodecpar: codecpar, srcTimeBase: timeBase, mode: .surroundCompat) + defer { bridge.close() } + let calls = Calls() + bridge.onDecoderProducedNothing = { calls.append($0) } + var outputs: [UnsafeMutablePointer] = [] + defer { freeAll(&outputs) } + for p in packets { outputs.append(contentsOf: (try? bridge.feed(packet: p)) ?? []) } + return (calls, bridge.feedStats) + } + + @Test("a bridge that decodes nothing tells the session once, with its counters") + func undecodableFeedReportsOnce() throws { + var packets = makeUndecodablePackets(count: 200) + defer { freeAll(&packets) } + let (calls, stats) = try feed(AV_CODEC_ID_MP3, packets: packets, + timeBase: AVRational(num: 1, den: 1000)) + #expect(stats.decodedNothing) + #expect(calls.all.count == 1, "one rebuild per session, not one per packet past the threshold") + #expect(calls.all.first?.framesDecoded == 0) + } + + @Test("a bridge that decodes and emits never asks for the video-only rebuild") + func healthyFeedNeverReports() throws { + var packets = makePCMPackets(count: 200) + defer { freeAll(&packets) } + let (calls, stats) = try feed(AV_CODEC_ID_PCM_S16LE, packets: packets, + timeBase: AVRational(num: 1, den: 48_000)) + #expect(stats.packetsEmitted > 0) + #expect(calls.all.isEmpty) + } + + @Test("the session's own rebuild keeps every marked stream, a host load starts clean") + func markSurvivesOnlyTheSessionsOwnRebuild() { + #expect(AetherEngine.undecodableAudioStreamIndicesAcrossLoad([1, 2], sessionPreservingReload: true) == [1, 2]) + #expect(AetherEngine.undecodableAudioStreamIndicesAcrossLoad([1, 2], sessionPreservingReload: false).isEmpty) + #expect(AetherEngine.undecodableAudioStreamIndicesAcrossLoad([], sessionPreservingReload: true).isEmpty) + } + + /// Audit FEA-103: the mark was a single slot, so a second undecodable stream was refused silently + /// and its pick left the session serving a bridge that never carries a sample. + @Test("a second undecodable stream is marked as well, and a repeat report changes nothing") + @MainActor + func secondUndecodableStreamIsMarked() async throws { + let engine = try AetherEngine() + await engine.dropUndecodableLiveAudio(streamIndex: 1, bridgeSummary: "fed=64 decoded=0") + #expect(engine.undecodableLiveAudioStreamIndices == [1]) + await engine.dropUndecodableLiveAudio(streamIndex: 2, bridgeSummary: "fed=64 decoded=0") + #expect(engine.undecodableLiveAudioStreamIndices == [1, 2]) + await engine.dropUndecodableLiveAudio(streamIndex: 2, bridgeSummary: "fed=128 decoded=0") + #expect(engine.undecodableLiveAudioStreamIndices == [1, 2]) + } + + @Test("the cascade skips every marked stream and only those") + func cascadeSkipsEveryMarkedStream() { + #expect(HLSVideoEngine.isKnownUndecodable([1, 2], sourceAudioStreamIndex: 1)) + #expect(HLSVideoEngine.isKnownUndecodable([1, 2], sourceAudioStreamIndex: 2)) + #expect(!HLSVideoEngine.isKnownUndecodable([1, 2], sourceAudioStreamIndex: 3)) + #expect(!HLSVideoEngine.isKnownUndecodable([], sourceAudioStreamIndex: 1)) + #expect(!HLSVideoEngine.isKnownUndecodable([0], sourceAudioStreamIndex: -1)) + } + + // MARK: - Where the verdict goes + + private final class Surfaced: @unchecked Sendable { + private let lock = NSLock() + private var kinds: [PlaybackErrorKind] = [] + private var live: [Int32] = [] + func surface(_ kind: PlaybackErrorKind) { lock.withLock { kinds.append(kind) } } + func forward(_ index: Int32) { lock.withLock { live.append(index) } } + var surfacedKinds: [PlaybackErrorKind] { lock.withLock { kinds } } + var forwarded: [Int32] { lock.withLock { live } } + } + + private func makeSession(live: Bool) -> (HLSVideoEngine, Surfaced) { + let session = HLSVideoEngine(url: URL(fileURLWithPath: "/nonexistent/ae641.mkv"), + dvModeAvailable: false, isLiveSession: live) + let surfaced = Surfaced() + session.onVODSourceFailed = { _, _, kind in surfaced.surface(kind) } + session.onLiveAudioDecodesNothing = { index, _ in surfaced.forward(index) } + return (session, surfaced) + } + + @Test("a VOD bridge that decodes nothing surfaces the same verdict as the E-AC-3 route") + func vodSurfacesTheTypedFailure() { + let (session, surfaced) = makeSession(live: false) + session.handleBridgeDecodedNothing(streamIndex: 1, summary: "fed=64 decoded=0") + #expect(surfaced.surfacedKinds == [.audioBridgeProducedNoOutput], + "a FLAC bridge has no failed cut to report it, so it played silently as .bridged") + #expect(surfaced.forwarded.isEmpty) + } + + @Test("a live bridge that decodes nothing goes to the engine's video-only rebuild, not to an error") + func liveForwardsInsteadOfFailing() { + let (session, surfaced) = makeSession(live: true) + session.handleBridgeDecodedNothing(streamIndex: 2, summary: "fed=64 decoded=0") + #expect(surfaced.forwarded == [2]) + #expect(surfaced.surfacedKinds.isEmpty) + } + + @Test("the silent-bridge verdict reaches the host once, every other failure as often as it happens") + func silentBridgeVerdictIsSurfacedOnce() { + let (session, surfaced) = makeSession(live: false) + // The detector, then the E-AC-3 route's failed cut, then its exhausted revive. + session.handleBridgeDecodedNothing(streamIndex: 1, summary: "fed=64 decoded=0") + session.surfaceVODSourceFailure(FFmpegErr.einval, "Audio track could not be decoded", + kind: .audioBridgeProducedNoOutput) + session.surfaceVODSourceFailure(FFmpegErr.einval, "Audio could not be transcoded for playback", + kind: .audioBridgeProducedNoOutput) + session.surfaceVODSourceFailure(FFmpegErr.einval, "Source audio cannot be muxed") + session.surfaceVODSourceFailure(FFmpegErr.einval, "Source audio cannot be muxed") + #expect(surfaced.surfacedKinds == [.audioBridgeProducedNoOutput, .vodSourceFailed, .vodSourceFailed]) + } +} diff --git a/Tests/AetherEngineTests/Issue647DiscReaderPrewarmTests.swift b/Tests/AetherEngineTests/Issue647DiscReaderPrewarmTests.swift new file mode 100644 index 000000000..7604bb88b --- /dev/null +++ b/Tests/AetherEngineTests/Issue647DiscReaderPrewarmTests.swift @@ -0,0 +1,129 @@ +import Testing +import Foundation +@testable import AetherEngine + +/// #647: a remote disc image the host warmed is read out of the warm, the way any other source is. +/// +/// Before this the disc reader never looked at the store: it probed the size with `bytes=0-0`, +/// refetched the head, and left the warm in the store for the streaming reader that never came. +/// +/// `.serialized`: the Demuxer case goes through the process-wide store. The reader cases use a +/// private store so nothing here depends on another suite leaving `.shared` alone. +@Suite("Disc reader adopts a prewarmed source (#647)", .serialized) +struct Issue647DiscReaderPrewarmTests { + + private let fileSize: Int64 = 64 * 1024 * 1024 + private let warmBytes = 1024 * 1024 + + private func url(_ server: ThrottledOriginServer, _ name: String = "disc.iso") -> URL { + URL(string: "http://127.0.0.1:\(server.port)/\(name)")! + } + + private func read(_ reader: IOReader, at offset: Int64, _ size: Int) -> [UInt8] { + _ = reader.seek(offset: offset, whence: SEEK_SET) + var out: [UInt8] = [] + let buf = UnsafeMutablePointer.allocate(capacity: size) + defer { buf.deallocate() } + while out.count < size { + let n = reader.read(buf, size: Int32(size - out.count)) + guard n > 0 else { break } + out.append(contentsOf: UnsafeBufferPointer(start: buf, count: Int(n))) + } + return out + } + + private func expected(at offset: Int64, _ size: Int) -> [UInt8] { + (0.. HTTPDiscIOReader { + let report = await SourcePrewarmFetcher.warm(url: url(server), extraHeaders: [:], + byteBudget: warmBytes, into: store) + #expect(report.retainedBytes == warmBytes) + let warm = try #require(HTTPDiscIOReader.takePrewarm(for: url(server), extraHeaders: [:], + store: store)) + #expect(!store.isWarm(for: url(server)), "adoption takes the entry") + return try #require(HTTPDiscIOReader(url: url(server), prewarmed: warm)) + } + + @Test("reads inside the warm cost no request, and the size is not probed") + func warmServesTheHead() async throws { + let server = try #require(ThrottledOriginServer(totalSize: fileSize, patternedBody: true)) + defer { server.stop() } + let reader = try await warmedReader(server, store: SourcePrewarmStore(totalByteCap: 8 << 20)) + defer { reader.close() } + let afterWarm = server.requestedRanges.count + + #expect(reader.seek(offset: 0, whence: 65536) == fileSize) + // The disc layer's opening shape: the ISO 9660 sniff at 0x8001, the UDF anchor at sector + // 256, a directory read back near the start. + #expect(read(reader, at: 0x8001, 5) == expected(at: 0x8001, 5)) + #expect(read(reader, at: 256 * 2048, 2048) == expected(at: 256 * 2048, 2048)) + #expect(read(reader, at: 40_960, 64 * 1024) == expected(at: 40_960, 64 * 1024)) + #expect(server.requestedRanges.count == afterWarm, + "a read inside the warm went to the network: \(server.requestedRanges.dropFirst(afterWarm))") + } + + @Test("a read across the warm frontier is exact and fetches from the frontier on") + func frontierContinues() async throws { + let server = try #require(ThrottledOriginServer(totalSize: fileSize, patternedBody: true)) + defer { server.stop() } + let reader = try await warmedReader(server, store: SourcePrewarmStore(totalByteCap: 8 << 20)) + defer { reader.close() } + let afterWarm = server.requestedRanges.count + + let start = Int64(warmBytes) - 1000 + #expect(read(reader, at: start, 300_000) == expected(at: start, 300_000)) + let opened = Array(server.requestedRanges.dropFirst(afterWarm)) + #expect(opened.first?.start == Int64(warmBytes), "requests: \(opened)") + #expect(!opened.contains(where: { $0.start < Int64(warmBytes) }), "requests: \(opened)") + } + + @Test("a fork shares the warm and the size, so it neither probes nor refetches the head") + func forkSharesTheWarm() async throws { + let server = try #require(ThrottledOriginServer(totalSize: fileSize, patternedBody: true)) + defer { server.stop() } + let reader = try await warmedReader(server, store: SourcePrewarmStore(totalByteCap: 8 << 20)) + defer { reader.close() } + let afterWarm = server.requestedRanges.count + + let fork = try #require(reader.makeIndependentReader()) + defer { fork.close() } + #expect(fork.seek(offset: 0, whence: 65536) == fileSize) + #expect(read(fork, at: 0, 128 * 1024) == expected(at: 0, 128 * 1024)) + #expect(server.requestedRanges.count == afterWarm, + "the fork went to the network: \(server.requestedRanges.dropFirst(afterWarm))") + } + + @Test("a warm fetched with other headers is not adopted") + func headersMustMatch() async throws { + let server = try #require(ThrottledOriginServer(totalSize: fileSize)) + defer { server.stop() } + let store = SourcePrewarmStore(totalByteCap: 8 << 20) + _ = await SourcePrewarmFetcher.warm(url: url(server), extraHeaders: ["Referer": "a"], + byteBudget: warmBytes, into: store) + #expect(HTTPDiscIOReader.takePrewarm(for: url(server), extraHeaders: ["Referer": "b"], + store: store) == nil) + } + + @Test("a disc-image URL that is no disc hands the warm to the streaming reader") + func notADiscKeepsTheWarm() async throws { + let server = try #require(ThrottledOriginServer(totalSize: fileSize, patternedBody: true)) + defer { server.stop() } + let source = url(server, "not-a-disc.iso") + _ = await SourcePrewarmFetcher.warm(url: source, extraHeaders: [:], + byteBudget: warmBytes, into: .shared) + let afterWarm = server.requestedRanges.count + + let demuxer = Demuxer() + defer { demuxer.close() } + // Patterned bytes are no container, so the open fails; what it asked the origin is the point. + try? demuxer.open(url: source) + + let opened = Array(server.requestedRanges.dropFirst(afterWarm)) + #expect(!opened.contains(where: { $0.start == 0 }), + "the open re-fetched bytes it already had: \(opened)") + #expect(!SourcePrewarmStore.shared.isWarm(for: source)) + } +} diff --git a/Tests/AetherEngineTests/Issue651DeclaredSubpictureTests.swift b/Tests/AetherEngineTests/Issue651DeclaredSubpictureTests.swift new file mode 100644 index 000000000..c7d8c4b3a --- /dev/null +++ b/Tests/AetherEngineTests/Issue651DeclaredSubpictureTests.swift @@ -0,0 +1,204 @@ +import Testing +import Foundation +import AetherLibavcodec +@testable import AetherEngine + +/// #651: a DVD title whose IFO declares its subpicture streams opens them before the probe, joins +/// their fragments the way libavformat's `dvdsub` parser would, and needs no long probe to find them. +@Suite("DVD subpicture streams declared from the IFO (#651)") +struct Issue651DeclaredSubpictureTests { + + // MARK: - Assembler + + private func timing(_ pts: Int64) -> DVDSubpictureAssembler.Timing { + .init(pts: pts, dts: pts, pos: 0, duration: 0) + } + + private func ingest(_ a: inout DVDSubpictureAssembler, _ bytes: [UInt8], _ pts: Int64) -> DVDSubpictureAssembler.Unit? { + bytes.withUnsafeBytes { a.ingest($0, timing: timing(pts)) } + } + + /// A unit of `size` bytes whose first two bytes state that size. + private func unit(_ size: Int) -> [UInt8] { + [UInt8(size >> 8), UInt8(size & 0xff)] + (2.. [UInt8] { + var ifo = [UInt8](repeating: 0, count: ISO9660Fixture.sectorSize) + ifo.replaceSubrange(0..<12, with: Array("DVDVIDEO-VTS".utf8)) + ifo[0x255] = UInt8(count) + for n in 0.. [UInt8] { [UInt8((v >> 8) & 0xff), UInt8(v & 0xff)] } + + private let packHeader: [UInt8] = [0, 0, 1, 0xBA, 0x44, 0x00, 0x04, 0x00, 0x04, 0x01, 0x01, 0x89, 0xC3, 0xF8] + + private func ptsBytes(_ pts: Int64) -> [UInt8] { + [UInt8(0x21 | ((pts >> 29) & 0x0E)), UInt8((pts >> 22) & 0xFF), + UInt8(((pts >> 14) & 0xFE) | 1), UInt8((pts >> 7) & 0xFF), UInt8(((pts << 1) & 0xFE) | 1)] + } + + /// One pack holding one private_stream_1 PES for subpicture substream `substream`. + private func subpicturePack(substream: UInt8, payload: [UInt8], pts: Int64?) -> [UInt8] { + let header: [UInt8] = pts.map { [0x81, 0x80, 5] + ptsBytes($0) } ?? [0x81, 0x00, 0] + let body = header + [substream] + payload + return packHeader + [0, 0, 1, 0xBD] + be16(body.count) + body + } + + @Test("a declared stream is a track before its first packet, and its fragments arrive joined") + func demuxerDeclaresAndJoins() throws { + // One sector in all: the fixture writes a single sector per file. + let spu = unit(1200) + var vob: [UInt8] = [] + vob += subpicturePack(substream: 0x20, payload: Array(spu[0..<400]), pts: 90_000) + vob += subpicturePack(substream: 0x20, payload: Array(spu[400..<800]), pts: nil) + vob += subpicturePack(substream: 0x20, payload: Array(spu[800...]), pts: nil) + vob += [0, 0, 1, 0xB9] + vob += [UInt8](repeating: 0, count: ISO9660Fixture.sectorSize - vob.count) + + let image = ISO9660Fixture.make(files: [ + .init(name: "VIDEO_TS.IFO", length: 2048), + .init(name: "VTS_01_0.IFO", length: ISO9660Fixture.sectorSize, + content: vtsIFO(subpictures: 2, withLanguage: 2)), + .init(name: "VTS_01_1.VOB", length: vob.count, content: vob), + ]) + let demuxer = Demuxer() + defer { demuxer.close() } + try demuxer.open(reader: DataIOReader(data: image), formatHint: nil) + + // 0x21 never carries a packet in this title, and is a track anyway. + let ids = (0..<32).compactMap { demuxer.stream(at: Int32($0))?.pointee.id } + #expect(ids.contains(0x20) && ids.contains(0x21), "stream ids: \(ids)") + + var joined: [(data: [UInt8], pts: Int64)] = [] + while let packet = try demuxer.readPacket() { + var p: UnsafeMutablePointer? = packet + if demuxer.stream(at: packet.pointee.stream_index)?.pointee.id == 0x20 { + joined.append((Array(UnsafeBufferPointer(start: packet.pointee.data, + count: Int(packet.pointee.size))), + packet.pointee.pts)) + } + trackedPacketFree(&p) + } + #expect(joined.count == 1) + #expect(joined.first?.data == spu) + #expect(joined.first?.pts == 90_000) + } +} diff --git a/Tests/AetherEngineTests/Issue65LivelockTests.swift b/Tests/AetherEngineTests/Issue65LivelockTests.swift index 1196bade3..3fd01d981 100644 --- a/Tests/AetherEngineTests/Issue65LivelockTests.swift +++ b/Tests/AetherEngineTests/Issue65LivelockTests.swift @@ -285,6 +285,71 @@ struct Issue65LivelockTests { #expect(d.observe(currentTarget: 84, renderedPosition: 391.902) == true) // 3s -> fast wedge } + // MARK: - BackpressureWedgeDetector quiet but rendering consumer (AE#649) + + @Test("A consumer that goes quiet while it keeps rendering never trips the slow path (AE#649)") + func quietRenderingConsumerNeverTrips() { + // Field shape, cellular iPhone: AVPlayer fetched a burst of segments, then nothing for 58 s while + // it played through its forward buffer. Production thresholds: the old slow path tripped at 24 s + // and its nudge seek flushed the buffer on screen. + var d = BackpressureWedgeDetector( + breakThresholdSeconds: 24, fastBreakThresholdSeconds: 5, + initialTarget: 453, initialRenderedPosition: 1697.1) + for i in 1...58 { + #expect(d.observe(currentTarget: 453, renderedPosition: 1697.1 + Double(i)) == false) + } + #expect(d.secondsSinceTargetMoved == 58) + #expect(d.secondsWithoutProgress == 0) + #expect(d.lastPollRendered == true) + } + + @Test("A quiet consumer that plays its buffer dry still breaks on the fast path (AE#649)") + func quietConsumerThatRunsDryFastTrips() { + // A consumer that really stopped fetching looks exactly like the healthy one above until its + // buffer runs out; from then it renders nothing, and that has to break in single digits. + var d = BackpressureWedgeDetector( + breakThresholdSeconds: 24, fastBreakThresholdSeconds: 5, + initialTarget: 453, initialRenderedPosition: 100.0) + for i in 1...30 { + #expect(d.observe(currentTarget: 453, renderedPosition: 100.0 + Double(i)) == false) + } + for _ in 1...4 { + #expect(d.observe(currentTarget: 453, renderedPosition: 130.0) == false) + } + #expect(d.observe(currentTarget: 453, renderedPosition: 130.0) == true) // 5th flat poll + #expect(d.lastTripFast == true) + #expect(d.lastPollRendered == false) + } + + @Test("Rendering seconds hold the slow count rather than reset it (AE#649)") + func renderingHoldsTheSlowCount() { + // A consumer that neither fetches nor plays steadily (a frame now and then) is not saved by + // the odd frame: only the seconds without progress count, and they add up across the frames. + var d = BackpressureWedgeDetector(breakThresholdSeconds: 3, initialTarget: 53, + initialRenderedPosition: 10.0) + #expect(d.observe(currentTarget: 53, renderedPosition: 11.0) == false) // rendered, idle 0 + #expect(d.observe(currentTarget: 53, renderedPosition: 11.0) == false) // idle 1 + #expect(d.observe(currentTarget: 53, renderedPosition: 12.0) == false) // rendered, held at 1 + #expect(d.observe(currentTarget: 53, renderedPosition: 12.0) == false) // idle 2 + #expect(d.observe(currentTarget: 53, renderedPosition: 13.0) == false) // rendered, held at 2 + #expect(d.observe(currentTarget: 53, renderedPosition: 13.0) == true) // idle 3 -> slow wedge + #expect(d.lastTripFast == false) + #expect(d.secondsSinceTargetMoved == 6) + } + + @Test("A fetch still restarts the slow count from zero (AE#649)") + func fetchResetsTheSlowCount() { + var d = BackpressureWedgeDetector(breakThresholdSeconds: 3, initialTarget: 53, + initialRenderedPosition: 10.0) + #expect(d.observe(currentTarget: 53, renderedPosition: 10.0) == false) // idle 1 + #expect(d.observe(currentTarget: 53, renderedPosition: 10.0) == false) // idle 2 + #expect(d.observe(currentTarget: 54, renderedPosition: 10.0) == false) // fetch -> 0 + #expect(d.secondsWithoutProgress == 0) + #expect(d.observe(currentTarget: 54, renderedPosition: 10.0) == false) // idle 1 + #expect(d.observe(currentTarget: 54, renderedPosition: 10.0) == false) // idle 2 + #expect(d.observe(currentTarget: 54, renderedPosition: 10.0) == true) // idle 3 + } + // MARK: - seekIsWedged (Piece B) @Test("Empty forward buffer at the rendered position is a wedge") diff --git a/Tests/AetherEngineTests/Issue660SoftwareVolumeTests.swift b/Tests/AetherEngineTests/Issue660SoftwareVolumeTests.swift new file mode 100644 index 000000000..ab1c80459 --- /dev/null +++ b/Tests/AetherEngineTests/Issue660SoftwareVolumeTests.swift @@ -0,0 +1,53 @@ +import Testing +import Foundation +@testable import AetherEngine + +/// #660: the engine hands its stored volume to a host right after building it, before `load()`. +/// The FFmpeg hosts forwarded that write to an `AudioOutput` that `load()` had not built yet, so it +/// was dropped and every software session started at full volume. The host now keeps the volume +/// itself and hands it to each output it builds. +@Suite("Software hosts keep a volume set before their audio output exists (#660)") +struct Issue660SoftwareVolumeTests { + + @MainActor + @Test("a software host reports the volume it was given before load") + func softwareHostKeepsVolumeBeforeLoad() { + let host = SoftwarePlaybackHost() + host.volume = 0.25 + #expect(host.volume == 0.25) + } + + @MainActor + @Test("an audio host reports the volume it was given before load") + func audioHostKeepsVolumeBeforeLoad() { + let host = AudioPlaybackHost() + host.volume = 0.25 + #expect(host.volume == 0.25) + } + + @MainActor + @Test("an audio host plays at the volume it was given before load") + func audioHostAppliesVolumeOnLoad() async throws { + let host = AudioPlaybackHost() + host.volume = 0.25 + let demuxer = Demuxer() + try demuxer.open(reader: DataIOReader(data: makeWAV(seconds: 1))) + try await host.load(demuxer: demuxer, startPosition: nil, audioSourceStreamIndex: nil) + defer { host.stop() } + #expect(host.outputVolumeForTesting == 0.25) + } + + private func makeWAV(seconds: Double) -> Data { + let sampleRate = 48_000, channels = 2 + let pcm = Data(count: Int(Double(sampleRate) * seconds) * channels * 2) + var d = Data() + func str(_ s: String) { d.append(s.data(using: .ascii)!) } + func u32(_ v: UInt32) { withUnsafeBytes(of: v.littleEndian) { d.append(contentsOf: $0) } } + func u16(_ v: UInt16) { withUnsafeBytes(of: v.littleEndian) { d.append(contentsOf: $0) } } + str("RIFF"); u32(UInt32(36 + pcm.count)); str("WAVE") + str("fmt "); u32(16); u16(1); u16(UInt16(channels)); u32(UInt32(sampleRate)) + u32(UInt32(sampleRate * channels * 2)); u16(UInt16(channels * 2)); u16(16) + str("data"); u32(UInt32(pcm.count)); d.append(pcm) + return d + } +} diff --git a/Tests/AetherEngineTests/Issue667MasterAfterSwitchTests.swift b/Tests/AetherEngineTests/Issue667MasterAfterSwitchTests.swift new file mode 100644 index 000000000..37350a8fa --- /dev/null +++ b/Tests/AetherEngineTests/Issue667MasterAfterSwitchTests.swift @@ -0,0 +1,69 @@ +import Foundation +import Testing +@testable import AetherEngine + +/// AE#667: an unproven HDR master served while the display mode switch is still running is refused for the +/// mode the panel is leaving, and that refusal was latched as a fact about the panel. +/// +/// Measured by the reporter on an Apple TV 4K (3rd gen, tvOS 27.0, HDR10 panel, Match Content on): the +/// pre-flight released at its 2 s cap, the master was served 140 ms later, AVPlayer failed it with `-11868` +/// after 80 ms, and the switch ended 660 ms after the refusal. Every HDR title on that box lost its master. +struct Issue667MasterAfterSwitchTests { + + @Test("An unproven master waits while a switch it saw start is still running") + func unprovenMasterWaitsForRunningSwitch() { + #expect(AetherEngine.unprovenMasterAwaitsSwitchEnd( + routesAsHDR: true, panelPresentsHDR: false, switchRunning: true)) + } + + @Test("No running switch, no wait: the #348 overlap stays where nothing is pending") + func noRunningSwitchNoWait() { + #expect(!AetherEngine.unprovenMasterAwaitsSwitchEnd( + routesAsHDR: true, panelPresentsHDR: false, switchRunning: false)) + } + + @Test("A proven panel already answered, so its master does not wait") + func provenPanelDoesNotWait() { + #expect(!AetherEngine.unprovenMasterAwaitsSwitchEnd( + routesAsHDR: true, panelPresentsHDR: true, switchRunning: true)) + } + + @Test("A media route has no acceptance to wait for") + func mediaRouteDoesNotWait() { + #expect(!AetherEngine.unprovenMasterAwaitsSwitchEnd( + routesAsHDR: false, panelPresentsHDR: false, switchRunning: true)) + } + + @Test("A refusal raised mid-switch does not latch, even on an eligible display") + func midSwitchRefusalDoesNotLatch() { + #expect(!MasterFallbackDecision.shouldLatchPanelRefusal( + code: -11868, displayEligibleForHDRNow: true, displaySwitchInProgress: true)) + #expect(!MasterFallbackDecision.shouldLatchPanelRefusal( + code: -11848, displayEligibleForHDRNow: true, displaySwitchInProgress: true)) + } + + @Test("A refusal after the switch settled still latches") + func settledRefusalStillLatches() { + #expect(MasterFallbackDecision.shouldLatchPanelRefusal( + code: -11868, displayEligibleForHDRNow: true, displaySwitchInProgress: false)) + } + + /// A policy test cannot see the call site, so this one reads it: the wait sits between the route and the + /// serve, and re-reads the panel afterwards so the route is decided on the arrived mode. + @Test("The load waits for the switch before routing, then routes again") + func loadSiteWaitsThenReroutes() throws { + let source = URL(fileURLWithPath: #filePath) + .deletingLastPathComponent() + .deletingLastPathComponent() + .deletingLastPathComponent() + .appendingPathComponent("Sources/AetherEngine/AetherEngine.swift") + let text = try #require(try? String(contentsOf: source, encoding: .utf8)) + let gate = try #require(text.range(of: "Self.unprovenMasterAwaitsSwitchEnd(")) + let after = text[gate.upperBound...] + let wait = try #require(after.range(of: "waitForSwitch(consumesRecord: false, settleCap: .awaitObservedEnd")) + let reroute = try #require(after.range(of: "routingPanelHDR = Self.sessionRoutesAsHDRPanel(")) + let serve = try #require(after.range(of: "panelIsInHDRMode: routingPanelHDR")) + #expect(wait.lowerBound < reroute.lowerBound) + #expect(reroute.lowerBound < serve.lowerBound) + } +} diff --git a/Tests/AetherEngineTests/Issue670FastZapGOPHeadroomTests.swift b/Tests/AetherEngineTests/Issue670FastZapGOPHeadroomTests.swift new file mode 100644 index 000000000..ab48f5db3 --- /dev/null +++ b/Tests/AetherEngineTests/Issue670FastZapGOPHeadroomTests.swift @@ -0,0 +1,145 @@ +// Tests/AetherEngineTests/Issue670FastZapGOPHeadroomTests.swift +// AE#670: live 1080p50 H.264 TS under `.fastZap` froze at irregular intervals, with AVPlayer reporting +// -12888 (playlist unchanged for longer than 1.5 x target duration) and #524 firing on 1.5 to 2 s of +// runway. The fastZap cut target (0.5 s) sits below any real GOP, so every segment the engine cuts is one +// whole source GOP. The TARGETDURATION is sealed from the first window, which on the reported source held +// three 1.000 s GOPs: TD 1, holdback 3 s, and no headroom at all over the GOP it had seen. The source's +// GOPs are not regular (1.0 to 2.4 s), so every longer one broke `EXTINF <= TD` and left the playlist +// unchanged past 1.5 x TD. `.standard` never had the problem because its `1.5 x cut target` floor is that +// headroom; under fastZap it collapses to 1 s. +// +// Measured with `aetherctl live --fast-zap --realtime --preroll 0` on a seed with that GOP pattern, two +// passes per arm: sealed at 1 s, one stall, the playlist refused as a parse error (-12642, macOS AVPlayer +// is stricter than the reporter's tvOS one) and a fall to the software path on both passes; sealed at 2 s, +// none of the three on either. +import XCTest +@testable import AetherEngine + +final class Issue670FastZapGOPHeadroomTests: XCTestCase { + + private let fastZapCut = HLSVideoEngine.liveCutTargetSeconds(for: .fastZap) + private let standardCut = HLSVideoEngine.liveCutTargetSeconds(for: .standard) + + /// The reported seal: three 1.000 s GOPs in the first window. + func testSelfCutFastZapSealsHeadroomOverTheFirstGOPs() { + let td = LiveEdgePolicy.targetDurationSeconds(maxSegmentDuration: 1.0, + cutTargetSeconds: fastZapCut, + cadenceFloorSeconds: nil, + segmentsAreCutHere: true) + XCTAssertEqual(td, 2) + XCTAssertEqual(LiveEdgePolicy.holdBackSeconds(targetDuration: td), 6.0, accuracy: 1e-9) + } + + /// What the seal has to survive afterwards: the 2.4 s GOP the reporter's log shows. It must round + /// under TD (RFC 8216 4.3.3.1), and the playlist must not stay unchanged past AVPlayer's patience + /// while the cutter waits for the keyframe that ends it. + func testLaterLongGOPStaysInsideTheSealedValue() { + let td = LiveEdgePolicy.targetDurationSeconds(maxSegmentDuration: 1.0, + cutTargetSeconds: fastZapCut, + cadenceFloorSeconds: nil, + segmentsAreCutHere: true) + let laterGOP = 2.4 + XCTAssertLessThanOrEqual(Int(laterGOP.rounded()), td) + XCTAssertGreaterThan(Double(td) * LiveEdgePolicy.unchangedPlaylistPatienceMultiplier, laterGOP) + } + + func testHeadroomScalesWithTheObservedGOP() { + XCTAssertEqual(LiveEdgePolicy.targetDurationSeconds(maxSegmentDuration: 0.96, + cutTargetSeconds: fastZapCut, + cadenceFloorSeconds: nil, + segmentsAreCutHere: true), 2) + XCTAssertEqual(LiveEdgePolicy.targetDurationSeconds(maxSegmentDuration: 1.92, + cutTargetSeconds: fastZapCut, + cadenceFloorSeconds: nil, + segmentsAreCutHere: true), 3) + } + + /// AE#447: ingested segments are the upstream's own, bounded by its advertised target duration, so + /// the field-measured TD 2 on 2.000 s segments stays. + func testIngestedSegmentsKeepTheirTargetDuration() { + XCTAssertEqual(LiveEdgePolicy.targetDurationSeconds(maxSegmentDuration: 2.0, + cutTargetSeconds: fastZapCut, + cadenceFloorSeconds: 2.0, + segmentsAreCutHere: false), 2) + } + + /// `.standard`'s cut target bounds its segments, and its `1.5 x cut target` floor is already the + /// headroom: the common 1.92 s GOP shape cuts 5.76 s segments and must keep TD 6, not rise to 9. + func testStandardProfileIsUnchanged() { + XCTAssertEqual(LiveEdgePolicy.targetDurationSeconds(maxSegmentDuration: 5.76, + cutTargetSeconds: standardCut, + cadenceFloorSeconds: nil, + segmentsAreCutHere: true), 6) + XCTAssertEqual(LiveEdgePolicy.targetDurationSeconds(maxSegmentDuration: 4.0, + cutTargetSeconds: standardCut, + cadenceFloorSeconds: nil, + segmentsAreCutHere: true), 6) + } + + /// Round 2, reported on a 59.94 fps source with 1.001 s GOPs: 79 frames sealed 2 and 80 frames + /// sealed 3, one millisecond of `1.5 x` apart (1.977 against 2.002), and the 9 s holdback cost a + /// 6.6 s rebuild backlog its immediate start. + func testOneMoreFrameDoesNotBuyAWholeSecond() { + let frame = 1001.0 / 60000.0 + for frames in [79, 80, 89] { + XCTAssertEqual(fastZapSeal(Double(frames) * frame), 2, "\(frames) frames") + } + } + + /// The reporter's TD 2 sessions later met segments up to 1.485 s and ran without a stall or -12888. + func testFieldMaximumUnderTheSealStaysAtTwo() { + XCTAssertEqual(fastZapSeal(1.485), 2) + } + + /// The boundary is where a GOP 1.5 x the longest seen would no longer list under TD 2: 2.499 s + /// rounds to 2, 2.5 s to 3. + func testHeadroomBoundaryFollowsTheListingRule() { + XCTAssertEqual(fastZapSeal(1.666), 2) + XCTAssertEqual(fastZapSeal(1.667), 3) + } + + /// Every value of the round-2 term satisfies what the first version promised for its 2.4 s GOP: + /// a GOP 1.5 x the longest seen lists under TD and is finalized inside AVPlayer's patience with the + /// delivery margin to spare. And it is never above `ceil(1.5 x max EXTINF)`. + func testHeadroomTermKeepsItsPromiseAndNeverRaisesTheSeal() { + var gop = 0.3 + while gop < 12 { + let td = LiveEdgePolicy.targetDurationForGOPHeadroom(gop) + let longer = LiveEdgePolicy.servedSeconds(gop * 1.5) + XCTAssertLessThanOrEqual(Int(longer.rounded(.toNearestOrAwayFromZero)), td, "\(gop)") + XCTAssertLessThanOrEqual(longer + LiveEdgePolicy.gopHeadroomDeliveryMarginSeconds, + Double(td) * LiveEdgePolicy.unchangedPlaylistPatienceMultiplier, "\(gop)") + XCTAssertLessThanOrEqual(td, LiveEdgePolicy.wholeSecondsCovering(gop * 1.5), "\(gop)") + gop += 0.001 + } + } + + func testHeadroomTermIsTotal() { + XCTAssertEqual(LiveEdgePolicy.targetDurationForGOPHeadroom(.nan), 0) + XCTAssertEqual(LiveEdgePolicy.targetDurationForGOPHeadroom(-1), 0) + XCTAssertEqual(LiveEdgePolicy.targetDurationForGOPHeadroom(.infinity), LiveEdgePolicy.maxCoveredWholeSeconds) + XCTAssertEqual(LiveEdgePolicy.targetDurationForGOPHeadroom(.greatestFiniteMagnitude), + LiveEdgePolicy.maxCoveredWholeSeconds) + } + + func testSealAccountNamesWhatTheHeadroomTermNeeds() { + let derivation = LiveTargetDurationDerivation( + value: 2, maxSegmentDuration: 80 * 1001.0 / 60000.0, cutTargetFloor: fastZapCut, + gopHeadroomApplies: true, cadenceFloor: .unmeasurable, selfReported: nil) + XCTAssertTrue(derivation.account.contains("1.5 x max EXTINF 2.002s needs 2s"), derivation.account) + } + + private func fastZapSeal(_ maxSegment: Double) -> Int { + LiveEdgePolicy.targetDurationSeconds(maxSegmentDuration: maxSegment, + cutTargetSeconds: fastZapCut, + cadenceFloorSeconds: nil, + segmentsAreCutHere: true) + } + + func testSealAccountNamesTheHeadroomTerm() { + let derivation = LiveTargetDurationDerivation( + value: 2, maxSegmentDuration: 1.0, cutTargetFloor: fastZapCut, + gopHeadroomApplies: true, cadenceFloor: .unmeasurable, selfReported: nil) + XCTAssertTrue(derivation.account.contains("1.5 x max EXTINF 1.500s"), derivation.account) + } +} diff --git a/Tests/AetherEngineTests/Issue674HEVCLevelCapTests.swift b/Tests/AetherEngineTests/Issue674HEVCLevelCapTests.swift new file mode 100644 index 000000000..0aafe1dbb --- /dev/null +++ b/Tests/AetherEngineTests/Issue674HEVCLevelCapTests.swift @@ -0,0 +1,96 @@ +import Testing +import Foundation +import AetherLibavcodec +import AetherLibavutil +@testable import AetherEngine + +/// AE#674: a DV Profile 8.1 MKV whose hvcC states level 5.2 (156) on a 1080p24 picture. The level went +/// into the master's CODECS verbatim, and an Apple TV 4K refused the DV master and the reduced HDR master +/// with -11848 / CoreMedia -15517, ending on the media playlist with Dolby Vision dropped. Declared as +/// 5.1 the same master is accepted and the panel engages DV (reporter, Sony X90L). The DV branches also +/// hardcoded `hvc1.2.4.L` and dropped the hvcC's constraint bytes, the disagreement AE#187 fixed +/// for plain HEVC. +@Suite("AE#674 declared HEVC level cap") +struct Issue674HEVCLevelCapTests { + + /// The reporter's hvcC header: profile_idc 2, compat 0x20000000, constraints 90 00.., level_idc 156. + static let duneHvcC: [UInt8] = [0x01, 0x02, 0x20, 0x00, 0x00, 0x00, 0x90, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x9c, + 0xf0, 0x00, 0xfc, 0xfd, 0xfa, 0xfa, 0x00, 0x00, 0x0f, 0x00] + + @Test("a stated level 5.2 is declared as 5.1") + func builderCapsLevel52() { + #expect(HLSVideoEngine.hevcCodecsString(fromConfigRecord: Self.duneHvcC) == "hvc1.2.4.L153.90") + } + + @Test("levels up to 5.1 are declared as stated", arguments: [UInt8(93), 120, 150, 153]) + func builderKeepsLowerLevels(level: UInt8) { + var hvcC = Self.duneHvcC + hvcC[12] = level + #expect(HLSVideoEngine.hevcCodecsString(fromConfigRecord: hvcC) == "hvc1.2.4.L\(level).90") + } + + @Test("the DV Profile 8.1 route declares the capped level with the hvcC's constraint bytes") + func profile81RouteFromHvcC() throws { + let par = try Profile81Codecpar(level: 156, extradata: Self.duneHvcC) + let r = try Self.route(par) + #expect(r.primaryCodecs == "hvc1.2.4.L153.90") + #expect(r.supplementalCodecs == "dvh1.08.03/db1p") + #expect(r.dvVariant == .profile81) + } + + @Test("without a parseable hvcC the fallback declaration is capped too") + func profile81RouteFallback() throws { + let par = try Profile81Codecpar(level: 156, extradata: nil) + #expect(try Self.route(par).primaryCodecs == "hvc1.2.4.L153") + } + + // MARK: - Harness + + private static func route(_ par: Profile81Codecpar) throws -> HLSVideoEngine.CodecRoute { + let engine = HLSVideoEngine(url: URL(fileURLWithPath: "/dev/null"), dvModeAvailable: true) + return try engine.resolveCodecRoute(codecpar: UnsafePointer(par.ptr)) + } + + /// 1920x802 HEVC with a DV Profile 8.1 (compat 1, dv_level 3) record, freed with the test. + private final class Profile81Codecpar { + let ptr: UnsafeMutablePointer + + init(level: Int32, extradata: [UInt8]?) throws { + ptr = try #require(avcodec_parameters_alloc()) + ptr.pointee.codec_type = AVMEDIA_TYPE_VIDEO + ptr.pointee.codec_id = AV_CODEC_ID_HEVC + ptr.pointee.width = 1920 + ptr.pointee.height = 802 + ptr.pointee.level = level + ptr.pointee.color_primaries = AVCOL_PRI_BT2020 + ptr.pointee.color_trc = AVCOL_TRC_SMPTE2084 + ptr.pointee.color_space = AVCOL_SPC_BT2020_NCL + if let bytes = extradata { + let ed = try #require(av_mallocz(bytes.count + Int(AV_INPUT_BUFFER_PADDING_SIZE))) + .assumingMemoryBound(to: UInt8.self) + bytes.withUnsafeBufferPointer { ed.update(from: $0.baseAddress!, count: bytes.count) } + ptr.pointee.extradata = ed + ptr.pointee.extradata_size = Int32(bytes.count) + } + let size = MemoryLayout.size + let sd = try #require(av_packet_side_data_new( + &ptr.pointee.coded_side_data, &ptr.pointee.nb_coded_side_data, + AV_PKT_DATA_DOVI_CONF, size, 0)) + memset(sd.pointee.data, 0, size) + sd.pointee.data.withMemoryRebound(to: AVDOVIDecoderConfigurationRecord.self, capacity: 1) { rec in + rec.pointee.dv_version_major = 1 + rec.pointee.dv_profile = 8 + rec.pointee.dv_level = 3 + rec.pointee.rpu_present_flag = 1 + rec.pointee.bl_present_flag = 1 + rec.pointee.dv_bl_signal_compatibility_id = 1 + } + } + + deinit { + var p: UnsafeMutablePointer? = ptr + avcodec_parameters_free(&p) + } + } +} diff --git a/Tests/AetherEngineTests/Issue684FirstServeLatchTests.swift b/Tests/AetherEngineTests/Issue684FirstServeLatchTests.swift new file mode 100644 index 000000000..b3a529335 --- /dev/null +++ b/Tests/AetherEngineTests/Issue684FirstServeLatchTests.swift @@ -0,0 +1,84 @@ +// Tests/AetherEngineTests/Issue684FirstServeLatchTests.swift +// AE#684 review: on an INGEST the first-serve gate held the second playlist request too. AVPlayer opens +// a session with two `/media.m3u8` requests back to back, each without an `_HLS_msn`, so each re-enters +// the gate. With the cushion satisfied both pass at once. On a bounded start (the window under the +// holdback, served after its grace) the second one waited out a second grace: measured on a +// three-segment 6 s upstream as 2.012 s to the first manifest, then 2.02 s more before `init.mp4`. +// +// Scoped to ingest sessions on purpose. On a source the engine cuts itself (raw MPEG-TS) the second +// grace is part of where the session ends up behind the producing edge: without it the first picture +// comes 2 s sooner and the session sits up to 2 s closer to the edge for its whole life, which nobody +// has measured on a device. That is AE#594's open question, so that path behaves as it did in 7.25.1. +import XCTest +@testable import AetherEngine + +final class Issue684FirstServeLatchTests: XCTestCase { + + private func makeProvider(ingest: Bool) -> (VideoSegmentProvider, SegmentCache) { + let cache = SegmentCache(forwardWindow: 10, backwardWindow: 10) + let policy = ingest ? LiveCadencePolicy(observe: { 0.1 }, cutTargetSeconds: 0.5, clock: { 0 }) : nil + let provider = VideoSegmentProvider( + cache: cache, + segments: [], + codecsString: "avc1.4D001E,mp4a.40.2", + supplementalCodecs: nil, + resolution: (720, 576), + videoRange: .sdr, + frameRate: 25, + hdcpLevel: nil, + sourceBitrate: 1_500_000, + isLive: true, + liveWindowSizing: LiveWindowSizing(targetSegmentDurationSeconds: 0.5, dvrWindowSeconds: nil), + allowsBoundedDegradedStart: true, + liveCadencePolicy: policy + ) + return (provider, cache) + } + + private func seconds(_ body: () -> Void) -> Double { + let start = DispatchTime.now() + body() + return Double(DispatchTime.now().uptimeNanoseconds - start.uptimeNanoseconds) / 1_000_000_000 + } + + /// Two 0.2 s segments against a 3 s holdback: the bounded start, grace 0.5 s. + func testOnAnIngestASecondRequestIsNotHeldForASecondGrace() { + let (provider, cache) = makeProvider(ingest: true) + defer { cache.close() } + provider.appendLiveSegment(index: 0, startSeconds: 0, durationSeconds: 0.2) + provider.appendLiveSegment(index: 1, startSeconds: 0.2, durationSeconds: 0.2) + + var served = false + let first = seconds { served = provider.waitForFirstLiveSegment(timeout: 3) } + XCTAssertTrue(served) + XCTAssertGreaterThanOrEqual(first, 0.45, "the first request pays the grace, as before") + + let second = seconds { served = provider.waitForFirstLiveSegment(timeout: 3) } + XCTAssertTrue(served) + XCTAssertLessThan(second, 0.2, "a manifest has gone out; the gate is open") + } + + /// A source the engine cuts itself keeps 7.25.1's gate: the second request waits its own grace. + func testOnASelfCutSourceTheSecondRequestStillWaitsItsGrace() { + let (provider, cache) = makeProvider(ingest: false) + defer { cache.close() } + provider.appendLiveSegment(index: 0, startSeconds: 0, durationSeconds: 0.2) + provider.appendLiveSegment(index: 1, startSeconds: 0.2, durationSeconds: 0.2) + + var served = false + let first = seconds { served = provider.waitForFirstLiveSegment(timeout: 3) } + XCTAssertTrue(served) + XCTAssertGreaterThanOrEqual(first, 0.45) + let second = seconds { served = provider.waitForFirstLiveSegment(timeout: 3) } + XCTAssertTrue(served) + XCTAssertGreaterThanOrEqual(second, 0.45, "unchanged from 7.25.1") + } + + /// The latch is on a SERVED manifest. A gate that gave up with nothing cut has served none. + func testAnUnservedGateStaysAGate() { + let (provider, cache) = makeProvider(ingest: true) + defer { cache.close() } + XCTAssertFalse(provider.waitForFirstLiveSegment(timeout: 0.1)) + XCTAssertFalse(provider.waitForFirstLiveSegment(timeout: 0.1)) + } +} diff --git a/Tests/AetherEngineTests/Issue684IngestSealCoversUpstreamSegmentTests.swift b/Tests/AetherEngineTests/Issue684IngestSealCoversUpstreamSegmentTests.swift new file mode 100644 index 000000000..da8fc4af9 --- /dev/null +++ b/Tests/AetherEngineTests/Issue684IngestSealCoversUpstreamSegmentTests.swift @@ -0,0 +1,309 @@ +// Tests/AetherEngineTests/Issue684IngestSealCoversUpstreamSegmentTests.swift +// AE#684: a live HLS ingest under `.fastZap` whose upstream cuts 6 s segments sealed the served +// TARGETDURATION at 4 (`max EXTINF 2.000s, measured floor 6.000s needs 4s of patience`), because the +// engine re-cuts each upstream segment at its 2 s GOPs and the upstream's own segment duration entered +// only through the cadence floor, as `ceil(6 / 1.5)`. That division belongs to a MEASURED gap, a robust +// maximum that is a worst case already. An upstream segment duration is the opposite: the period the +// source delivers at when nothing is late, so `1.5 x TD` came out at exactly one period, 6.0 s, and every +// ordinary delivery (6.17 to 6.83 s in the field capture, 63 of 110 gaps above 6.0 s) was past the +// client's patience. The session drew -12888, AVPlayer then skipped a playlist reload (5.03 s between two +// polls instead of 2 s), and the 12 s holdback ran out 2.4 s after the content it was waiting for had +// been listed. +// +// The upstream delivers in whole segments however finely they are re-cut here, so the served playlist +// changes once per upstream segment and the seal asks for what a playlist of those segments promises: +// TARGETDURATION >= the longest upstream segment. These pin what is ASKED. What is sealed is that, +// as far as the join pays for it (`Issue684JoinBoundSealTests`), and the join is not deepened. +import XCTest +@testable import AetherEngine + +private final class ScriptedIngest: @unchecked Sendable { + var now: Double = 0 + var cadence: Double? + var closedCadence: Double? + var segmentDuration: Double? +} + +final class Issue684IngestSealCoversUpstreamSegmentTests: XCTestCase { + + private let fastZapCut = HLSVideoEngine.liveCutTargetSeconds(for: .fastZap) + private let standardCut = HLSVideoEngine.liveCutTargetSeconds(for: .standard) + + private func makePolicy(_ s: ScriptedIngest, advertised: Double?) -> LiveCadencePolicy { + LiveCadencePolicy( + observe: { s.cadence }, + cutTargetSeconds: 0.5, + observeSealEvidence: { + LiveCadenceEvidence(closedCadenceSeconds: s.closedCadence, + servedSegmentDurationSeconds: s.segmentDuration) + }, + selfReportedTargetDurationSeconds: advertised, + clock: { s.now } + ) + } + + // MARK: - The reported seal + + /// 6 s upstream segments, re-cut at 2.000 s GOPs, sealed inside the join burst. + func testSixSecondUpstreamRecutAtTwoSecondGOPsSealsAtSix() { + let td = LiveEdgePolicy.targetDurationSeconds(maxSegmentDuration: 2.0, + cutTargetSeconds: fastZapCut, + cadenceFloorSeconds: 6.0, + upstreamSegmentSeconds: 6.0) + XCTAssertEqual(td, 6) + XCTAssertEqual(LiveEdgePolicy.holdBackSeconds(targetDuration: td), 18.0, accuracy: 1e-9) + } + + /// The deliveries the field capture shows for one upstream segment, and the later candidate the + /// session itself logged (`measured floor 8.492s needs 6s of patience`). + func testPatienceCoversTheFieldDeliveries() { + let td = LiveEdgePolicy.targetDurationSeconds(maxSegmentDuration: 2.0, + cutTargetSeconds: fastZapCut, + cadenceFloorSeconds: 6.0, + upstreamSegmentSeconds: 6.0) + let patience = Double(td) * LiveEdgePolicy.unchangedPlaylistPatienceMultiplier + for gap in [6.17, 6.54, 6.83, 7.36, 8.32, 8.492] { + XCTAssertGreaterThan(patience, gap) + } + XCTAssertGreaterThanOrEqual(td, LiveEdgePolicy.targetDurationForCadence(8.492), + "the seal must not be under what the session measured afterwards") + } + + /// A period is not a worst case: the patience has to clear it by the same half the RFC gives a + /// playlist of those segments, for every segment length and not only where `ceil` happens to help. + func testPatienceIsOneAndAHalfUpstreamSegmentsAtEveryLength() { + for upstream in [1.0, 1.92, 2.0, 3.0, 4.0, 4.5, 6.0, 9.0, 10.0] { + let td = LiveEdgePolicy.targetDurationSeconds(maxSegmentDuration: min(upstream, 2.0), + cutTargetSeconds: fastZapCut, + cadenceFloorSeconds: upstream, + upstreamSegmentSeconds: upstream) + XCTAssertGreaterThanOrEqual(Double(td) * LiveEdgePolicy.unchangedPlaylistPatienceMultiplier, + upstream * 1.5, "upstream \(upstream)s sealed at \(td)s") + } + } + + // MARK: - What stays + + /// AE#447: 2.000 s and 1.920 s ingests keep TARGETDURATION 2 and a 6 s holdback. + func testTwoSecondIngestKeepsTargetDurationTwo() { + for upstream in [2.0, 1.92] { + let td = LiveEdgePolicy.targetDurationSeconds(maxSegmentDuration: upstream, + cutTargetSeconds: fastZapCut, + cadenceFloorSeconds: 2.142, + upstreamSegmentSeconds: upstream) + XCTAssertEqual(td, 2, "upstream \(upstream)s") + } + } + + /// A measured gap is a robust maximum and still enters as the patience it needs. + func testClosedCadenceStillEntersAsThePatienceItNeeds() { + XCTAssertEqual(LiveEdgePolicy.targetDurationSeconds(maxSegmentDuration: 2.0, + cutTargetSeconds: fastZapCut, + cadenceFloorSeconds: 20.0, + upstreamSegmentSeconds: 2.0), 14) + } + + /// No upstream playlist, no term: AE#670's self-cut seal and the raw-TS seal are what they were. + func testSelfCutSessionsAreUnchanged() { + XCTAssertEqual(LiveEdgePolicy.targetDurationSeconds(maxSegmentDuration: 1.0, + cutTargetSeconds: fastZapCut, + cadenceFloorSeconds: nil, + segmentsAreCutHere: true, + upstreamSegmentSeconds: nil), 2) + XCTAssertEqual(LiveEdgePolicy.targetDurationSeconds(maxSegmentDuration: 5.76, + cutTargetSeconds: standardCut, + cadenceFloorSeconds: nil, + segmentsAreCutHere: true, + upstreamSegmentSeconds: nil), 6) + } + + /// `.standard` on the same 6 s upstream already sat at 6 through its `1.5 x cut target` floor. + func testStandardProfileOnSixSecondUpstreamIsUnchanged() { + XCTAssertEqual(LiveEdgePolicy.targetDurationSeconds(maxSegmentDuration: 6.0, + cutTargetSeconds: standardCut, + cadenceFloorSeconds: 6.0, + upstreamSegmentSeconds: 6.0), 6) + } + + /// Taken at the served resolution like every other term (AE#447 round 2). + func testUpstreamSegmentIsTakenAtTheServedResolution() { + XCTAssertEqual(LiveEdgePolicy.targetDurationForUpstreamSegment(6.0000000000000009), 6) + XCTAssertEqual(LiveEdgePolicy.targetDurationForUpstreamSegment(6.006), 7) + XCTAssertEqual(LiveEdgePolicy.targetDurationForUpstreamSegment(.infinity), + Int(LiveEdgePolicy.upstreamSegmentSealCeilingSeconds)) + XCTAssertEqual(LiveEdgePolicy.targetDurationForUpstreamSegment(.nan), 0) + } + + // MARK: - `.standard`, which the term reaches above 6 s + + /// `.standard` re-cuts too (4 s cut target), so a provider above its `1.5 x cut target` floor is + /// delivered in units longer than anything the floor covers. This is what the terms ASK for; the + /// three-segment join then pays 9 of the 10 (`Issue684JoinBoundSealTests`). Measured on a 10 s + /// origin with jittered deliveries, three 60 s runs per arm and window depth: at TD 7 the item + /// was rebuilt in 6 of 6 runs, at TD 9 in none. + func testStandardProfileSealsLongUpstreamSegmentsWhole() { + func seal(_ upstream: Double) -> Int { + LiveEdgePolicy.targetDurationSeconds(maxSegmentDuration: 4.0, + cutTargetSeconds: standardCut, + cadenceFloorSeconds: upstream, + upstreamSegmentSeconds: upstream) + } + XCTAssertEqual(seal(4.0), 6, "under the floor: unchanged") + XCTAssertEqual(seal(6.0), 6, "at the floor: unchanged") + XCTAssertEqual(seal(8.0), 8, "was 6") + XCTAssertEqual(seal(10.0), 10, "was 7") + XCTAssertEqual(LiveEdgePolicy.holdBackSeconds(targetDuration: seal(10.0)), 30.0, accuracy: 1e-9) + } + + // MARK: - The ceiling + + /// A degenerate upstream: a 20 s segment taken whole would be a 60 s holdback, the whole + /// live-only window. Above the ceiling the term stops growing and the seal is what it was. + func testVeryLongUpstreamSegmentsAreNotTakenWhole() { + let ceiling = Int(LiveEdgePolicy.upstreamSegmentSealCeilingSeconds) + for upstream in [12.0, 15.0, 20.0, 28.0] { + let td = LiveEdgePolicy.targetDurationSeconds(maxSegmentDuration: 2.0, + cutTargetSeconds: fastZapCut, + cadenceFloorSeconds: upstream, + upstreamSegmentSeconds: upstream) + XCTAssertEqual(td, max(ceiling, LiveEdgePolicy.targetDurationForCadence(upstream)), + "upstream \(upstream)s") + XCTAssertLessThan(LiveEdgePolicy.holdBackSeconds(targetDuration: td), + LiveWindowSizing.liveOnlyFloorSeconds, "upstream \(upstream)s") + } + XCTAssertEqual(LiveEdgePolicy.targetDurationSeconds(maxSegmentDuration: 2.0, + cutTargetSeconds: fastZapCut, + cadenceFloorSeconds: 20.0, + upstreamSegmentSeconds: 20.0), 14, + "a 20 s segment seals what it sealed before the term existed") + } + + /// The term on its own never asks for more than half the live-only window of holdback. + func testTheTermAloneStaysWellInsideTheLiveWindow() { + let most = LiveEdgePolicy.targetDurationForUpstreamSegment(.greatestFiniteMagnitude) + XCTAssertLessThanOrEqual(LiveEdgePolicy.holdBackSeconds(targetDuration: most), + LiveWindowSizing.liveOnlyFloorSeconds / 2) + } + + /// A fractional segment costs a whole second, like every other term (AE#447 round 2). + func testFractionalSegmentCost() { + let td = LiveEdgePolicy.targetDurationSeconds(maxSegmentDuration: 2.0, + cutTargetSeconds: fastZapCut, + cadenceFloorSeconds: 6.006, + upstreamSegmentSeconds: 6.006) + XCTAssertEqual(td, 7) + XCTAssertEqual(LiveEdgePolicy.holdBackSeconds(targetDuration: td), 21.0, accuracy: 1e-9) + let coverage = HLSPlaylistTracker.loopbackCushionCoverageSeconds(segments: (0..<8).map { _ in + HLSMediaSegment(uri: "s", duration: 6.006, discontinuityBefore: false) + }) + XCTAssertEqual(coverage, 19.0, accuracy: 1e-9, + "the join is not sized from the seal: ceil(6.006 / 1.5) = 5, 15 s and a 4 s GOP") + } + + // MARK: - As far as the join can pay + + func testAnExhaustedJoinSealsWhatItsWindowCovers() { + // Three 6 s segments: 18 s joined, 16 s cut, one 2 s GOP open. + XCTAssertEqual(LiveEdgePolicy.targetDurationTheJoinCanPay( + full: 6, withoutUpstreamSegment: 4, finalizedSeconds: 16.0), 5) + // Three 10 s segments: 28 s cut. + XCTAssertEqual(LiveEdgePolicy.targetDurationTheJoinCanPay( + full: 10, withoutUpstreamSegment: 7, finalizedSeconds: 28.0), 9) + // A join deep enough pays in full. + XCTAssertEqual(LiveEdgePolicy.targetDurationTheJoinCanPay( + full: 6, withoutUpstreamSegment: 4, finalizedSeconds: 22.0), 6) + // Never under what the seal was without the term, however little was joined. + XCTAssertEqual(LiveEdgePolicy.targetDurationTheJoinCanPay( + full: 6, withoutUpstreamSegment: 4, finalizedSeconds: 10.0), 4) + // Total. + XCTAssertEqual(LiveEdgePolicy.targetDurationTheJoinCanPay( + full: 6, withoutUpstreamSegment: 4, finalizedSeconds: .nan), 4) + XCTAssertEqual(LiveEdgePolicy.targetDurationTheJoinCanPay( + full: 6, withoutUpstreamSegment: 4, finalizedSeconds: .infinity), 4) + } + + // MARK: - Where the term comes from + + /// Measured, not advertised: the longest segment the upstream actually served. + func testPolicyReportsTheLongestServedUpstreamSegment() { + let s = ScriptedIngest() + let policy = makePolicy(s, advertised: 9) + XCTAssertNil(policy.upstreamSegmentDurationSeconds, "an advert alone is not a served segment") + s.now = 1; s.cadence = 0.2; s.segmentDuration = 4.0 + XCTAssertEqual(try XCTUnwrap(policy.upstreamSegmentDurationSeconds), 4.0, accuracy: 1e-9) + s.now = 2; s.segmentDuration = 6.0 + XCTAssertEqual(try XCTUnwrap(policy.upstreamSegmentDurationSeconds), 6.0, accuracy: 1e-9) + s.now = 3; s.segmentDuration = nil + XCTAssertEqual(try XCTUnwrap(policy.upstreamSegmentDurationSeconds), 6.0, accuracy: 1e-9, + "monotonic, like the floor it sits beside") + } + + // MARK: - The join keeps its depth + + /// Owner's rule: the join is never extended for the upstream-segment term. One more upstream + /// segment is one more download before the first picture on every zap (AE#678 declined it), so + /// the coverage is what 7.24.0 sized it to and the seal rises only as far as that pays. + func testJoinIsNotExtendedForTheUpstreamSegmentTerm() { + func coverage(_ durations: [Double]) -> Double { + HLSPlaylistTracker.loopbackCushionCoverageSeconds(segments: durations.map { + HLSMediaSegment(uri: "s", duration: $0, discontinuityBefore: false) + }) + } + XCTAssertEqual(coverage([6, 6, 6]), 16) // ceil(6 / 1.5) = 4, 12 s and a 4 s GOP: three segments + XCTAssertEqual(coverage([10, 10, 10]), 25) // 7, 21 s + 4 s: three segments + XCTAssertEqual(coverage([2, 2, 2]), 8) + XCTAssertEqual(coverage([20, 20, 20]), 46) + var tracker = HLSPlaylistTracker() + let uris = ["a", "b", "c", "d", "e", "f", "g", "h"] + let joined = tracker.newSegments(in: HLSMediaPlaylist( + targetDuration: 6, + mediaSequence: 0, + segments: uris.map { HLSMediaSegment(uri: $0, duration: 6, discontinuityBefore: false) }, + hasEndList: false, + isEncrypted: false, + hasUnsupportedEncryption: false, + hasMap: false)) + XCTAssertEqual(joined.map(\.uri), ["f", "g", "h"], "three 6 s segments, as before this issue") + } + + // MARK: - The seal says why + + func testSealAccountNamesTheUpstreamSegmentTerm() { + let derivation = LiveTargetDurationDerivation( + value: 6, maxSegmentDuration: 2.0, cutTargetFloor: fastZapCut, + cadenceFloor: .measured(6.0), upstreamSegment: 6.0, selfReported: 6.0) + let account = derivation.account + XCTAssertTrue(account.contains("sealed at 6s"), account) + XCTAssertTrue(account.contains("holdback 18.000s"), account) + XCTAssertTrue(account.contains("upstream segment 6.000s"), account) + XCTAssertTrue(account.contains("upstream advertises 6.000s (reported, not used)"), account) + } + + /// AE#447 round 2's property, with the new term in the sum: the line adds up to its own number. + func testSealAccountStillRecomputesTheValueItReports() throws { + func number(after label: String, in text: String) -> Double? { + guard let r = text.range(of: label + "[0-9]+\\.?[0-9]*", options: .regularExpression) else { + return nil + } + return Double(text[r].dropFirst(label.count)) + } + for upstream in [2.0, 4.0, 6.0, 6.006, 10.0] { + for floor in [upstream, 8.492, 20.0] { + let value = LiveEdgePolicy.targetDurationSeconds( + maxSegmentDuration: 2.0, cutTargetSeconds: fastZapCut, + cadenceFloorSeconds: floor, upstreamSegmentSeconds: upstream) + let account = LiveTargetDurationDerivation( + value: value, maxSegmentDuration: 2.0, cutTargetFloor: fastZapCut, + cadenceFloor: .measured(floor), upstreamSegment: upstream, selfReported: 6.0).account + var recomputed = Int(ceil(try XCTUnwrap(number(after: "max EXTINF ", in: account)))) + recomputed = max(recomputed, + Int(ceil(try XCTUnwrap(number(after: "1.5 x cut target ", in: account))))) + recomputed = max(recomputed, + Int(ceil(try XCTUnwrap(number(after: "upstream segment ", in: account))))) + recomputed = max(recomputed, Int(try XCTUnwrap(number(after: "needs ", in: account)))) + XCTAssertEqual(Int(try XCTUnwrap(number(after: "sealed at ", in: account))), recomputed, + account) + } + } + } +} diff --git a/Tests/AetherEngineTests/Issue684JoinBoundSealTests.swift b/Tests/AetherEngineTests/Issue684JoinBoundSealTests.swift new file mode 100644 index 000000000..882619122 --- /dev/null +++ b/Tests/AetherEngineTests/Issue684JoinBoundSealTests.swift @@ -0,0 +1,264 @@ +// Tests/AetherEngineTests/Issue684JoinBoundSealTests.swift +// AE#684: the seal over the upstream segment, as far as the join can pay for it. +// +// The join is three upstream segments on anything longer than about 5 s and is not deepened for the +// seal. Three 6 s segments are 18 s joined; the last GOP stays open until the next upstream delivery, +// so 16 s are cut, and the 18 s holdback of a TARGETDURATION 6 is not reachable from the join at all. +// Waiting for it is waiting for the next upstream segment: measured 4.25 s to first picture where +// 7.25.1 took 0.18 s. A join that has handed over everything it will seals the largest value its cut +// content covers, between what the seal was without the upstream term and the full one. +// +// "Has handed over everything" is a fact the reader states. Round 1 derived it from EXTINF sums with +// a tolerance, and a playlist whose EXTINF over-reports its media (6.3 for 6 s, measured in review) +// never added up: TD 7, a 21 s holdback, bounded start after the grace, 2.23 s to first picture. +import XCTest +@testable import AetherEngine + +/// Locked because a test flips `joinSpent` from another queue while the provider reads it. +private final class JoinedUpstream: @unchecked Sendable { + private let lock = NSLock() + private var _segmentDuration: Double? + private var _joinBacklog: Double? + private var _joinSpent: Bool? + var segmentDuration: Double? { + get { lock.withLock { _segmentDuration } } + set { lock.withLock { _segmentDuration = newValue } } + } + var joinBacklog: Double? { + get { lock.withLock { _joinBacklog } } + set { lock.withLock { _joinBacklog = newValue } } + } + var joinSpent: Bool? { + get { lock.withLock { _joinSpent } } + set { lock.withLock { _joinSpent = newValue } } + } +} + +final class Issue684JoinBoundSealTests: XCTestCase { + + private func makeProvider(_ upstream: JoinedUpstream, cutTarget: Double, + boundedStart: Bool) -> (VideoSegmentProvider, SegmentCache) { + let cache = SegmentCache(forwardWindow: 10, backwardWindow: 10) + let policy = LiveCadencePolicy( + observe: { 0.1 }, + cutTargetSeconds: cutTarget, + observeSealEvidence: { + LiveCadenceEvidence(closedCadenceSeconds: nil, + servedSegmentDurationSeconds: upstream.segmentDuration, + joinBacklogSeconds: upstream.joinBacklog, + joinIsSpent: upstream.joinSpent) + }, + selfReportedTargetDurationSeconds: upstream.segmentDuration, + clock: { 0 } + ) + let provider = VideoSegmentProvider( + cache: cache, + segments: [], + codecsString: "avc1.4D001E,mp4a.40.2", + supplementalCodecs: nil, + resolution: (720, 576), + videoRange: .sdr, + frameRate: 25, + hdcpLevel: nil, + sourceBitrate: 1_500_000, + isLive: true, + liveWindowSizing: LiveWindowSizing(targetSegmentDurationSeconds: cutTarget, dvrWindowSeconds: nil), + allowsBoundedDegradedStart: boundedStart, + liveCadencePolicy: policy + ) + return (provider, cache) + } + + private func append(_ provider: VideoSegmentProvider, count: Int, each: Double) { + for index in 0.. Void) -> Double { + let start = DispatchTime.now() + body() + return Double(DispatchTime.now().uptimeNanoseconds - start.uptimeNanoseconds) / 1_000_000_000 + } + + /// The reviewer's shape: three 6 s segments, fastZap, eight 2 s GOPs cut. + func testShallowFastZapJoinServesAtOnceOnTheSealItCanPay() { + let upstream = JoinedUpstream() + upstream.segmentDuration = 6.0 + upstream.joinBacklog = 18.0 + upstream.joinSpent = true + let (provider, cache) = makeProvider(upstream, cutTarget: 0.5, boundedStart: true) + defer { cache.close() } + append(provider, count: 8, each: 2.0) + + var served = false + let waited = seconds { served = provider.waitForFirstLiveSegment(timeout: 5) } + XCTAssertTrue(served) + XCTAssertLessThan(waited, 0.5, "no grace: the join has nothing more to give") + let td = provider.liveTargetDurationSeconds(maxSegmentDuration: 2.0) + XCTAssertEqual(td, 5, "16 s of window pays for 5, between the old 4 and the full 6") + XCTAssertLessThanOrEqual(LiveEdgePolicy.holdBackSeconds(targetDuration: td), 16.0, + "and the holdback it advertises is one the window holds") + } + + /// A join that does cover the full holdback pays in full (the reporting channel's own shape: + /// 6 s and 4 s segments alternating, four joined, 18 s cut or more). + func testAJoinThatCoversTheHoldbackKeepsTheFullSeal() { + let upstream = JoinedUpstream() + upstream.segmentDuration = 6.0 + upstream.joinBacklog = 24.0 + upstream.joinSpent = true + let (provider, cache) = makeProvider(upstream, cutTarget: 0.5, boundedStart: true) + defer { cache.close() } + append(provider, count: 11, each: 2.0) + XCTAssertTrue(provider.waitForFirstLiveSegment(timeout: 5)) + XCTAssertEqual(provider.liveTargetDurationSeconds(maxSegmentDuration: 2.0), 6) + } + + /// A join still arriving is not spent: the gate keeps waiting for the cushion, as before. + func testAJoinStillArrivingIsNotSealedShort() { + let upstream = JoinedUpstream() + upstream.segmentDuration = 6.0 + upstream.joinBacklog = 24.0 + upstream.joinSpent = false + let (provider, cache) = makeProvider(upstream, cutTarget: 0.5, boundedStart: false) + defer { cache.close() } + append(provider, count: 5, each: 2.0) + let derivation = provider.firstServeTargetDuration((count: 5, summed: 10.0, maxDuration: 2.0), joinIsSpent: upstream.joinSpent) + XCTAssertEqual(derivation.value, 6) + XCTAssertNil(derivation.joinBound) + } + + /// `.standard` on a 10 s provider listing three segments: 4 s cuts, 28 s of them. It has no + /// bounded start, so without this it waits for the next upstream segment, up to 10 s of wall clock. + func testShallowStandardJoinOnTenSecondSegmentsServesAtOnce() { + let upstream = JoinedUpstream() + upstream.segmentDuration = 10.0 + upstream.joinBacklog = 30.0 + upstream.joinSpent = true + let (provider, cache) = makeProvider(upstream, cutTarget: 4.0, boundedStart: false) + defer { cache.close() } + append(provider, count: 7, each: 4.0) + + var served = false + let waited = seconds { served = provider.waitForFirstLiveSegment(timeout: 5) } + XCTAssertTrue(served) + XCTAssertLessThan(waited, 0.5) + XCTAssertEqual(provider.liveTargetDurationSeconds(maxSegmentDuration: 4.0), 9, + "28 s of window pays for 9, between the old 7 and the full 10") + } + + /// `.standard` on 6 s segments was at 6 before the term and is at 6 with it: nothing to pay down. + func testStandardOnSixSecondSegmentsIsWhatItWas() { + let upstream = JoinedUpstream() + upstream.segmentDuration = 6.0 + upstream.joinBacklog = 18.0 + upstream.joinSpent = true + let (provider, cache) = makeProvider(upstream, cutTarget: 4.0, boundedStart: false) + defer { cache.close() } + append(provider, count: 4, each: 4.0) + let derivation = provider.firstServeTargetDuration((count: 4, summed: 16.0, maxDuration: 4.0), joinIsSpent: upstream.joinSpent) + XCTAssertEqual(derivation.value, 6) + XCTAssertNil(derivation.joinBound) + } + + // MARK: - EXTINF and media need not agree + + /// The re-review's shape: 6 s of media per segment listed as 6.3 (advertised 7). EXTINF says + /// 18.9 s were joined; 16 s are cut and nothing more is coming. Arithmetic on the sum (16 + 2 + + /// tolerance against 18.9) said "still arriving" and sealed the full 7. + func testInflatedExtinfStillSealsWhatTheJoinPays() { + let upstream = JoinedUpstream() + upstream.segmentDuration = 6.3 + upstream.joinBacklog = 18.9 + upstream.joinSpent = true + let (provider, cache) = makeProvider(upstream, cutTarget: 0.5, boundedStart: true) + defer { cache.close() } + append(provider, count: 8, each: 2.0) + + var served = false + let waited = seconds { served = provider.waitForFirstLiveSegment(timeout: 5) } + XCTAssertTrue(served) + XCTAssertLessThan(waited, 0.5) + XCTAssertEqual(provider.liveTargetDurationSeconds(maxSegmentDuration: 2.0), 5, + "what 7.25.1 sealed here too: ceil(6.3 / 1.5)") + } + + /// The other way round: the media runs longer than its EXTINF says. The sum is already reached + /// (16 + 2 against 18) while the reader is still handing media over, so the arithmetic called + /// the join exhausted and would have sealed short of what the rest of it pays for. + func testMediaLongerThanExtinfIsNotSealedBeforeTheJoinIsSpent() { + let upstream = JoinedUpstream() + upstream.segmentDuration = 6.0 + upstream.joinBacklog = 18.0 + upstream.joinSpent = false + let (provider, cache) = makeProvider(upstream, cutTarget: 0.5, boundedStart: false) + defer { cache.close() } + append(provider, count: 8, each: 2.0) + let arriving = provider.firstServeTargetDuration((count: 8, summed: 16.0, maxDuration: 2.0), joinIsSpent: upstream.joinSpent) + XCTAssertEqual(arriving.value, 6) + XCTAssertNil(arriving.joinBound) + + // One more GOP lands, the reader runs dry: 18 s cut pays for the full 6. + provider.appendLiveSegment(index: 8, startSeconds: 16.0, durationSeconds: 2.0) + upstream.joinSpent = true + XCTAssertTrue(provider.waitForFirstLiveSegment(timeout: 5)) + XCTAssertEqual(provider.liveTargetDurationSeconds(maxSegmentDuration: 2.0), 6) + } + + /// The gate has no event for "the cutter parked on an empty reader", so it has to look again by + /// itself: a join that turns spent while a request is waiting is served then, not at the grace. + func testAWaitingRequestIsServedWhenTheJoinTurnsSpent() { + let upstream = JoinedUpstream() + upstream.segmentDuration = 10.0 + upstream.joinBacklog = 30.0 + upstream.joinSpent = false + let (provider, cache) = makeProvider(upstream, cutTarget: 4.0, boundedStart: false) + defer { cache.close() } + append(provider, count: 7, each: 4.0) + DispatchQueue.global().asyncAfter(deadline: .now() + 0.2) { upstream.joinSpent = true } + var served = false + let waited = seconds { served = provider.waitForFirstLiveSegment(timeout: 5) } + XCTAssertTrue(served) + XCTAssertGreaterThanOrEqual(waited, 0.15) + XCTAssertLessThan(waited, 1.5, "not the 5 s deadline: the gate noticed") + XCTAssertEqual(provider.liveTargetDurationSeconds(maxSegmentDuration: 4.0), 9) + } + + /// The reader's half of the fact: nothing queued and its consumer parked. + func testAnEmptyFIFOWithAParkedReaderIsTheSpentSignal() { + let fifo = ByteFIFO(capacity: 1024) + XCTAssertFalse(fifo.isEmptyWithReaderParked, "nobody is reading") + XCTAssertTrue(fifo.write(Data([1, 2, 3]))) + XCTAssertFalse(fifo.isEmptyWithReaderParked, "bytes are waiting") + let buffer = UnsafeMutablePointer.allocate(capacity: 16) + defer { buffer.deallocate() } + XCTAssertEqual(fifo.read(into: buffer, maxLength: 16), 3) + XCTAssertFalse(fifo.isEmptyWithReaderParked, "drained, but the consumer has not come back for more") + let done = expectation(description: "reader returned") + Thread.detachNewThread { + let scratch = UnsafeMutablePointer.allocate(capacity: 16) + _ = fifo.read(into: scratch, maxLength: 16) + scratch.deallocate() + done.fulfill() + } + while fifo.parkedWaiterCount == 0 { usleep(200) } + XCTAssertTrue(fifo.isEmptyWithReaderParked) + fifo.cancel() + wait(for: [done], timeout: 2) + XCTAssertFalse(fifo.isEmptyWithReaderParked, "a cancelled reader is not a spent join") + } + + /// The seal line says what was paid and what was asked, and the drift line names the term too. + func testSealAccountStatesWhatTheJoinPaid() { + var derivation = LiveTargetDurationDerivation( + value: 5, maxSegmentDuration: 2.0, cutTargetFloor: 0.5, + cadenceFloor: .measured(6.0), upstreamSegment: 6.0, selfReported: 6.0) + derivation.joinBound = (backlogSeconds: 18.0, finalizedSeconds: 16.0, full: 6) + let account = derivation.account + XCTAssertTrue(account.contains("sealed at 5s (holdback 15.000s)"), account) + XCTAssertTrue(account.contains("upstream segment 6.000s"), account) + XCTAssertTrue(account.contains("of which the join pays 5s of 6s (16.000s cut of the 18.000s it listed)"), + account) + } +} diff --git a/Tests/AetherEngineTests/Issue684LiveItemStartAccountTests.swift b/Tests/AetherEngineTests/Issue684LiveItemStartAccountTests.swift new file mode 100644 index 000000000..40f188309 --- /dev/null +++ b/Tests/AetherEngineTests/Issue684LiveItemStartAccountTests.swift @@ -0,0 +1,69 @@ +// Tests/AetherEngineTests/Issue684LiveItemStartAccountTests.swift +// AE#684, second half: a viewer saw sound and picture slightly apart after an item rebuild placed +// mid-segment (`segment 26 + 1.91s`) and together again after the next one (`segment 129 + 0.03s`). +// The served media measures aligned and macOS cannot observe AVPlayer's audio renderer, so the session +// states what each native live item started on: the segment, the offset into it, and where that +// segment's sound begins against its picture. These pin the line and the lookup it reads. +import XCTest +@testable import AetherEngine + +final class Issue684LiveItemStartAccountTests: XCTestCase { + + func testAccountNamesThePlacementAndTheSoundAgainstThePicture() { + let line = AetherEngine.liveItemStartAccount( + item: 7, itemSeconds: 53.908, + heads: (index: 26, secondsIntoSegment: 1.908, pictureStart: 52.0, sound: (first: 51.921, last: 53.905))) + XCTAssertTrue(line.contains("#684 item #7 starts at its own 53.908s"), line) + XCTAssertTrue(line.contains("segment 26 + 1.908s"), line) + XCTAssertTrue(line.contains("picture opens at 52.000s"), line) + XCTAssertTrue(line.contains("sound runs 51.921..53.905s"), line) + XCTAssertTrue(line.contains("opens 79 ms before the picture"), line) + } + + func testAccountSaysWhenTheSoundOpensAfterThePicture() { + let line = AetherEngine.liveItemStartAccount( + item: 1, itemSeconds: 6.0, + heads: (index: 3, secondsIntoSegment: 0, pictureStart: 6.0, sound: (first: 6.013, last: 7.997))) + XCTAssertTrue(line.contains("opens 13 ms after the picture"), line) + } + + /// An absence is stated, not printed as a zero that would read as "aligned". + func testAccountDoesNotInventWhatWasNotRecorded() { + let unrecorded = AetherEngine.liveItemStartAccount( + item: 2, itemSeconds: 4.0, + heads: (index: 2, secondsIntoSegment: 0, pictureStart: 4.0, sound: nil)) + XCTAssertTrue(unrecorded.contains("sound was not recorded"), unrecorded) + XCTAssertFalse(unrecorded.contains(" ms "), unrecorded) + let unlisted = AetherEngine.liveItemStartAccount(item: 2, itemSeconds: 400.0, heads: nil) + XCTAssertTrue(unlisted.contains("in no segment the producer lists yet"), unlisted) + } + + func testProviderResolvesAPositionToItsSegmentAndItsSound() throws { + let provider = VideoSegmentProvider( + cache: SegmentCache(forwardWindow: 10, backwardWindow: 10), + segments: [], + codecsString: "avc1.4D001E,mp4a.40.2", + supplementalCodecs: nil, + resolution: (720, 576), + videoRange: .sdr, + frameRate: 25, + hdcpLevel: nil, + sourceBitrate: 1_500_000, + isLive: true, + liveWindowSizing: LiveWindowSizing(targetSegmentDurationSeconds: 0.5, dvrWindowSeconds: nil), + liveCadencePolicy: nil + ) + for index in 0..<4 { + provider.appendLiveSegment(index: index, startSeconds: Double(index) * 2, durationSeconds: 2) + provider.noteLiveSegmentSound(index: index, + firstSeconds: Double(index) * 2 - 0.079, + lastSeconds: Double(index) * 2 + 1.899) + } + let heads = try XCTUnwrap(provider.liveSegmentHeads(atOutputSeconds: 5.908)) + XCTAssertEqual(heads.index, 2) + XCTAssertEqual(heads.secondsIntoSegment, 1.908, accuracy: 1e-9) + XCTAssertEqual(heads.pictureStart, 4.0, accuracy: 1e-9) + XCTAssertEqual(try XCTUnwrap(heads.sound).first, 3.921, accuracy: 1e-9) + XCTAssertNil(provider.liveSegmentHeads(atOutputSeconds: 8.5), "past the last listed segment") + } +} diff --git a/Tests/AetherEngineTests/Issue684LiveJoinNotReadyTests.swift b/Tests/AetherEngineTests/Issue684LiveJoinNotReadyTests.swift new file mode 100644 index 000000000..866af0ae8 --- /dev/null +++ b/Tests/AetherEngineTests/Issue684LiveJoinNotReadyTests.swift @@ -0,0 +1,90 @@ +// Tests/AetherEngineTests/Issue684LiveJoinNotReadyTests.swift +// AE#684: the AE#440 forced start was taken on a rejoin item that could not play yet. The capture: +// +// 14:51:15.650 #454 placing the fresh item at 31871.11s in its own playlist: segment 26 + 1.91s +// 14:51:15.716 #7 AE#440 live join: leaving the stall-avoidance wait alone (buffer ahead 0.00s, +// empty=true ...) (playhead 48.00s), and the item's own status has not left unknown +// 14:51:15.731 #7 AE#440 live join: cutting the stall-avoidance wait short at rate 1.0 +// (buffer ahead 4.00s) +// 14:51:15.767 #7 item.status=readyToPlay +// 14:51:15.778 #454 the playlist already placed this item at its own 53.91s +// +// The item was placed at 53.908 s by `EXT-X-START`, AVPlayer fetched from 48.00 s first (its own +// lookback), and the guard measured its cushion from `currentTime()`, which before readiness is that +// 48.00 s: two fetched segments, 4.00 s, all of it behind the point playback was about to start at. +// It is the only one of the five item starts in that capture the engine forced, and the one the +// viewer reported out of sync. AE#440 round 7 had named the shape before any capture showed it. +import AVFoundation +import Testing +@testable import AetherEngine + +@Suite("AE#684: the AE#440 start is not forced on an item that cannot play yet") +struct Issue684LiveJoinNotReadyTests { + + @Test("the captured reading is refused: 4.00 s ahead of a playhead the item has not taken yet") + func refusesTheCapturedReading() { + #expect(!NativeAVPlayerHost.shouldStartLiveJoinImmediately( + armed: true, alreadySpent: false, hostWantsToPlay: true, + isWaitingToMinimizeStalls: true, playbackBufferEmpty: false, bufferedAheadSeconds: 4.0, + itemIsReadyToPlay: false)) + } + + @Test("the same hold on a ready item is still the case AE#440 exists for") + func firesOnceTheItemCanPlay() { + #expect(NativeAVPlayerHost.shouldStartLiveJoinImmediately( + armed: true, alreadySpent: false, hostWantsToPlay: true, + isWaitingToMinimizeStalls: true, playbackBufferEmpty: false, bufferedAheadSeconds: 4.0, + itemIsReadyToPlay: true)) + } + + /// The reading is asynchronous and the readiness sink's question is dropped while one is in + /// flight, so a refusal taken on a stale `unknown` has to notice the item is ready now. + @Test("a not-ready refusal asks once more when the item turned ready under it") + func asksAgainOnceWhenTheItemTurnedReady() { + #expect(NativeAVPlayerHost.liveJoinAsksAgainAfterNotReadyRefusal( + readingWasNotReady: true, itemIsReadyNow: true, alreadyAskedAgain: false)) + #expect(!NativeAVPlayerHost.liveJoinAsksAgainAfterNotReadyRefusal( + readingWasNotReady: true, itemIsReadyNow: false, alreadyAskedAgain: false)) + #expect(!NativeAVPlayerHost.liveJoinAsksAgainAfterNotReadyRefusal( + readingWasNotReady: true, itemIsReadyNow: true, alreadyAskedAgain: true)) + } + + /// A reading that is both thin and not ready is no less stale than a deep one: the item it was + /// taken on could not say where it starts. It keeps the thin-buffer line and still asks again. + @Test("a thin reading on a not-ready item asks again too, and a refusal on a ready item does not") + func thinAndNotReadyStillAsksAgain() { + let thin = NativeAVPlayerHost.LiveJoinBufferReading( + bufferEmpty: true, aheadSeconds: 0, playheadSeconds: 48.0, + loadedRangeCount: 0, nearestRangeOffsetSeconds: nil, itemStatus: .unknown) + #expect(NativeAVPlayerHost.liveJoinNotReadyRefusal(reading: thin) == nil) + #expect(NativeAVPlayerHost.liveJoinAsksAgainAfterNotReadyRefusal( + readingWasNotReady: thin.itemStatus != .readyToPlay, itemIsReadyNow: true, + alreadyAskedAgain: false)) + #expect(!NativeAVPlayerHost.liveJoinAsksAgainAfterNotReadyRefusal( + readingWasNotReady: false, itemIsReadyNow: true, alreadyAskedAgain: false)) + } + + @Test("the refusal names the playhead the cushion was read from") + func refusalNamesThePlayhead() throws { + let reading = NativeAVPlayerHost.LiveJoinBufferReading( + bufferEmpty: false, aheadSeconds: 4.0, playheadSeconds: 48.0, + loadedRangeCount: 1, nearestRangeOffsetSeconds: nil, itemStatus: .unknown) + let line = try #require(NativeAVPlayerHost.liveJoinNotReadyRefusal(reading: reading)) + #expect(line.contains("buffer ahead 4.00s of playhead 48.00s")) + #expect(line.contains("cannot play yet")) + #expect(line.contains("asked again at readiness")) + } + + /// The thin-buffer line already accounts for these; a second line would say the same refusal twice. + @Test("no extra line for a refusal the depth explains, or for a ready item") + func refusalIsOnlyForTheDepthThatWouldHaveFired() { + let thin = NativeAVPlayerHost.LiveJoinBufferReading( + bufferEmpty: true, aheadSeconds: 0, playheadSeconds: 48.0, + loadedRangeCount: 0, nearestRangeOffsetSeconds: nil, itemStatus: .unknown) + #expect(NativeAVPlayerHost.liveJoinNotReadyRefusal(reading: thin) == nil) + let ready = NativeAVPlayerHost.LiveJoinBufferReading( + bufferEmpty: false, aheadSeconds: 4.0, playheadSeconds: 53.908, + loadedRangeCount: 1, nearestRangeOffsetSeconds: nil, itemStatus: .readyToPlay) + #expect(NativeAVPlayerHost.liveJoinNotReadyRefusal(reading: ready) == nil) + } +} diff --git a/Tests/AetherEngineTests/Issue684MixedLengthJoinTests.swift b/Tests/AetherEngineTests/Issue684MixedLengthJoinTests.swift new file mode 100644 index 000000000..f68a6cca1 --- /dev/null +++ b/Tests/AetherEngineTests/Issue684MixedLengthJoinTests.swift @@ -0,0 +1,164 @@ +// Tests/AetherEngineTests/Issue684MixedLengthJoinTests.swift +// AE#684, the reporter's own channel: its upstream alternates 6 s and 4 s segments. The join counts +// seconds back from the newest segment, so its depth depended on which of the two was newest. With a +// 4 s one newest, `4 + 6 + 4` is under the 16 s coverage target and a fourth segment is taken: 20 s +// listed, 18 s cut, seal 6. With a 6 s one newest, `6 + 4 + 6` meets the target at three: 16 s listed, +// 14 s cut, seal 4, 7.25.1's value and its stalls. Measured on `--durs 6,4 --window 8`: `--prefill 8` +// sealed 6 with no -12888 and no stall, `--prefill 7` sealed 4 in both arms and stalled as before. +// About four tunes in ten land in that phase. +// +// The join is equalised over the phases of one listing: a tune never loads more than the deepest tune +// of the same channel already did. A first version of this took one segment more on ANY mixed-length +// join that could not pay its seal, which also deepened shapes with no phase asymmetry at all, on +// every tune: 10 s / 8 s from three to four segments (6.19 s to 8.12 s to first picture behind +// 8 Mbit/s) and 6 s / 6 s / 4 s likewise (3.81 s to 4.98 s). Those keep their join now. +import XCTest +@testable import AetherEngine + +final class Issue684MixedLengthJoinTests: XCTestCase { + + /// The join a fresh tracker takes from a playlist listing these durations, oldest first. + private func joined(_ durations: [Double]) -> [Double] { + var tracker = HLSPlaylistTracker() + let playlist = HLSMediaPlaylist( + targetDuration: (durations.max() ?? 0).rounded(.up), + mediaSequence: 100, + segments: durations.enumerated().map { + HLSMediaSegment(uri: "s\($0.offset)", duration: $0.element, discontinuityBefore: false) + }, + hasEndList: false, + isEncrypted: false, + hasUnsupportedEncryption: false, + hasMap: false + ) + return tracker.newSegments(in: playlist).map(\.duration) + } + + /// What the gate seals from a spent join of these segments at 2 s GOPs under `.fastZap`. + private func seal(_ join: [Double]) -> Int { + let longest = join.max() ?? 0 + let cut = join.reduce(0, +) - 2.0 + let full = LiveEdgePolicy.targetDurationSeconds(maxSegmentDuration: 2.0, cutTargetSeconds: 0.5, + cadenceFloorSeconds: longest, + upstreamSegmentSeconds: longest) + let base = LiveEdgePolicy.targetDurationSeconds(maxSegmentDuration: 2.0, cutTargetSeconds: 0.5, + cadenceFloorSeconds: longest) + return LiveEdgePolicy.targetDurationTheJoinCanPay(full: full, withoutUpstreamSegment: base, + finalizedSeconds: cut) + } + + // MARK: - Both phases of the reported channel + + func testSixFourAlternationJoinsFourSegmentsInBothPhases() { + let newestIsFour = joined([6, 4, 6, 4, 6, 4, 6, 4]) + XCTAssertEqual(newestIsFour, [6, 4, 6, 4], "under the coverage target at three, as before") + let newestIsSix = joined([4, 6, 4, 6, 4, 6, 4, 6]) + XCTAssertEqual(newestIsSix, [4, 6, 4, 6], "three met the target; the fourth is this rule's") + XCTAssertEqual(seal(newestIsFour), 6) + XCTAssertEqual(seal(newestIsSix), 6) + } + + /// The phase is where the tune lands, not how the playlist began. + func testFourSixOrderIsTheSameChannel() { + XCTAssertEqual(joined([4, 6, 4, 6, 4, 6, 4]).reduce(0, +), 20) + XCTAssertEqual(joined([6, 4, 6, 4, 6, 4, 6]).reduce(0, +), 20) + XCTAssertEqual(joined([4, 6, 4, 6, 4, 6, 4]).count, 4) + XCTAssertEqual(joined([6, 4, 6, 4, 6, 4, 6]).count, 4) + } + + // MARK: - Shapes with no deeper phase keep their join + + /// Three segments are 16 s in every phase, so no tune of this channel ever loaded a fourth, and + /// none does now. Its seal is what three pay: 14 s cut, 4. + func testSixSixFourPatternKeepsThreeSegmentsInEveryPhase() { + let phases: [[Double]] = [ + [6, 6, 4, 6, 6, 4, 6, 6, 4], + [6, 4, 6, 6, 4, 6, 6, 4, 6], + [4, 6, 6, 4, 6, 6, 4, 6, 6], + ] + for phase in phases { + let join = joined(phase) + XCTAssertEqual(join.count, 3, "\(phase)") + XCTAssertEqual(join.reduce(0, +), 16, "\(phase)") + XCTAssertEqual(seal(join), 4, "\(phase)") + } + } + + /// 28 s and 26 s both meet the 25 s target at three. Seal 8 in both phases (26 s and 24 s cut). + func testTenEightAlternationKeepsThreeSegmentsInBothPhases() { + let newestIsTen = joined([8, 10, 8, 10, 8, 10, 8, 10]) + let newestIsEight = joined([10, 8, 10, 8, 10, 8, 10, 8]) + XCTAssertEqual(newestIsTen, [10, 8, 10]) + XCTAssertEqual(newestIsEight, [8, 10, 8]) + XCTAssertEqual(seal(newestIsTen), 8) + XCTAssertEqual(seal(newestIsEight), 8) + } + + /// Equalising phases is about depth, not about the seal: 6 s / 5 s joins three either way + /// (17 s and 16 s), and its seal still follows the phase, 5 and 4. + func testAShapeWhosePhasesJoinAlikeKeepsItsPhaseDependentSeal() { + let newestIsSix = joined([5, 6, 5, 6, 5, 6, 5, 6]) + let newestIsFive = joined([6, 5, 6, 5, 6, 5, 6, 5]) + XCTAssertEqual(newestIsSix.count, 3) + XCTAssertEqual(newestIsFive.count, 3) + XCTAssertEqual(seal(newestIsSix), 5) + XCTAssertEqual(seal(newestIsFive), 4) + } + + // MARK: - Uniform upstreams keep the 7.24.0 join + + func testUniformSixAndTenSecondJoinsAreUnchanged() { + XCTAssertEqual(joined(Array(repeating: 6, count: 8)), [6, 6, 6]) + XCTAssertEqual(joined(Array(repeating: 10, count: 8)), [10, 10, 10]) + XCTAssertEqual(joined(Array(repeating: 2, count: 8)), [2, 2, 2, 2]) + XCTAssertEqual(seal([6, 6, 6]), 5) + XCTAssertEqual(seal([10, 10, 10]), 9) + } + + /// EXTINF that wanders by frames around its nominal value is a uniform upstream. + func testFrameJitterInExtinfIsNotAPhase() { + XCTAssertEqual(joined([6.0, 5.96, 5.92, 6.0, 5.96, 5.92, 6.0, 5.96]).count, 3) + XCTAssertEqual(joined([5.96, 5.92, 6.0, 5.96, 5.92, 6.0, 5.96, 6.0]).count, 3) + } + + // MARK: - The rule's edges + + /// The depth is the deepest the coverage rule takes over the phases of the listing, and never + /// more than one above the tune's own. + func testDepthIsTheDeepestPhaseBoundedAtOneAboveTheTunesOwn() { + func depth(_ durations: [Double], coverage: Double) -> Int { + HLSPlaylistTracker.phaseEqualisedJoinDepth(durations: durations, coverage: coverage, limit: 8) + } + XCTAssertEqual(depth([4, 6, 4, 6, 4, 6, 4, 6], coverage: 16), 4) + XCTAssertEqual(depth([6, 4, 6, 4, 6, 4, 6, 4], coverage: 16), 4) + XCTAssertEqual(depth([6, 6, 6, 6, 6, 6, 6, 6], coverage: 16), 3) + // A run of short segments behind two long ones: the earlier phases need six and more, the + // tune's own two. Bounded at three. + XCTAssertEqual(depth([2, 2, 2, 2, 2, 2, 10, 10], coverage: 16), 3) + // The old rule alone, and a listing too short to decide a tune. + XCTAssertEqual(HLSPlaylistTracker.coverageJoinDepth(durations: [4, 6, 4, 6][...], coverage: 16, limit: 8), 3) + XCTAssertNil(HLSPlaylistTracker.coverageJoinDepth(durations: [4, 6][...], coverage: 16, limit: 8)) + } + + /// A lone short segment on a uniform upstream. While it is among the newest three the join is + /// four, as it always was (15 s is under the target). For the two tunes after that the join + /// still contains the newest segment of a tune that took four, so it stays four. After that it + /// is three again, although the short segment is still listed. + func testALoneShortSegmentOnAUniformUpstream() { + XCTAssertEqual(joined([6, 6, 6, 6, 6, 6, 6, 3]), [6, 6, 6, 3], "as before") + XCTAssertEqual(joined([6, 6, 6, 6, 6, 6, 3, 6]), [6, 6, 3, 6], "as before") + XCTAssertEqual(joined([6, 6, 6, 6, 6, 3, 6, 6]), [6, 3, 6, 6], "as before") + XCTAssertEqual(joined([6, 6, 6, 6, 3, 6, 6, 6]), [3, 6, 6, 6], "one tune after: lifted to four") + XCTAssertEqual(joined([6, 6, 6, 3, 6, 6, 6, 6]), [6, 6, 6, 6], "two tunes after: lifted to four") + XCTAssertEqual(joined([6, 6, 3, 6, 6, 6, 6, 6]), [6, 6, 6], "three after: its own depth again") + XCTAssertEqual(joined([6, 6, 6, 6, 6, 6, 6, 6]).count, 3) + } + + /// One more, once, and never past what the playlist offers or the eviction margin allows. + func testTheExtraSegmentRespectsTheWindow() { + XCTAssertEqual(joined([4, 6, 4, 6]), [6, 4, 6], "four listed: the oldest stays where it is") + XCTAssertEqual(joined([6, 4, 6]), [6, 4, 6], "three listed: all of them, nothing more to take") + XCTAssertEqual(joined([6, 4, 6, 4, 6]).count, 4) + } + +} diff --git a/Tests/AetherEngineTests/Issue684PumpJoinSpentTests.swift b/Tests/AetherEngineTests/Issue684PumpJoinSpentTests.swift new file mode 100644 index 000000000..c7f92701d --- /dev/null +++ b/Tests/AetherEngineTests/Issue684PumpJoinSpentTests.swift @@ -0,0 +1,322 @@ +// Tests/AetherEngineTests/Issue684PumpJoinSpentTests.swift +// AE#684: "the join has nothing more to give" is a fact the reader states, and the seal on every +// ingest start hangs on it. Two ways it was wrong or untested: +// +// - It was read off the VIDEO reader alone. With a demuxed audio rendition the cutter merges two +// readers on one thread and parks on whichever runs dry first. A rendition whose segments end 0.2 s +// before the video's runs dry first every time, the video reader is never parked, and the fact never +// became true: measured on a three-segment window as the full seal over a window that cannot hold it, +// 2.17 to 2.24 s to first picture under `.fastZap` and 10.4 to 10.7 s under `.standard`, against +// 0.18 to 0.20 s on 7.25.1. +// - It was only ever tested through a provider fake. These drive the reader itself against a loopback +// origin whose responses the test releases by hand, so nothing here depends on how long a sleep is. +import XCTest +@testable import AetherEngine + +/// Loopback origin: a master, a video and an audio media playlist of three 6 s segments each, and +/// TS-shaped segment bodies. A path named in `hold` is answered only once `release` names it. +private final class HeldSegmentOrigin: @unchecked Sendable { + let port: UInt16 + private let listenFD: Int32 + private let lock = NSCondition() + private var held: Set + private var stopped = false + + static let segmentBytes = 4 * 188 + + init?(hold: Set) { + held = hold + let fd = socket(AF_INET, SOCK_STREAM, 0) + guard fd >= 0 else { return nil } + var one: Int32 = 1 + setsockopt(fd, SOL_SOCKET, SO_REUSEADDR, &one, socklen_t(MemoryLayout.size)) + var addr = sockaddr_in() + addr.sin_family = sa_family_t(AF_INET) + addr.sin_port = 0 + addr.sin_addr.s_addr = inet_addr("127.0.0.1") + let bound = withUnsafePointer(to: &addr) { + $0.withMemoryRebound(to: sockaddr.self, capacity: 1) { + bind(fd, $0, socklen_t(MemoryLayout.size)) + } + } + guard bound == 0, listen(fd, 16) == 0 else { close(fd); return nil } + var name = sockaddr_in() + var len = socklen_t(MemoryLayout.size) + let named = withUnsafeMutablePointer(to: &name) { + $0.withMemoryRebound(to: sockaddr.self, capacity: 1) { getsockname(fd, $0, &len) } + } + guard named == 0 else { close(fd); return nil } + listenFD = fd + port = UInt16(bigEndian: name.sin_port) + Thread.detachNewThread { [weak self] in self?.acceptLoop() } + } + + func url(_ path: String) -> URL { URL(string: "http://127.0.0.1:\(port)/\(path)")! } + + func release(_ path: String) { + lock.lock() + held.remove(path) + lock.broadcast() + lock.unlock() + } + + func stop() { + lock.lock() + stopped = true + held.removeAll() + lock.broadcast() + lock.unlock() + close(listenFD) + } + + private func acceptLoop() { + while true { + let conn = accept(listenFD, nil, nil) + guard conn >= 0 else { return } + var noSigPipe: Int32 = 1 + setsockopt(conn, SOL_SOCKET, SO_NOSIGPIPE, &noSigPipe, socklen_t(MemoryLayout.size)) + Thread.detachNewThread { [weak self] in self?.serve(conn) } + } + } + + private func mediaPlaylist(prefix: String) -> Data { + var lines = ["#EXTM3U", "#EXT-X-VERSION:3", "#EXT-X-TARGETDURATION:6", "#EXT-X-MEDIA-SEQUENCE:0"] + for index in 0..<3 { + lines.append("#EXTINF:6.000,") + lines.append("\(prefix)\(index).ts") + } + return Data((lines.joined(separator: "\n") + "\n").utf8) + } + + private func segment(tag: UInt8) -> Data { + var data = Data(capacity: Self.segmentBytes) + for packet in 0..<4 { + var ts = [UInt8](repeating: tag, count: 188) + ts[0] = 0x47 + ts[1] = UInt8(packet) + data.append(contentsOf: ts) + } + return data + } + + private func serve(_ conn: Int32) { + defer { close(conn) } + var request = Data() + var buf = [UInt8](repeating: 0, count: 4096) + while request.range(of: Data("\r\n\r\n".utf8)) == nil { + let n = read(conn, &buf, buf.count) + guard n > 0 else { return } + request.append(contentsOf: buf[0.. 64 * 1024 { return } + } + guard let head = String(data: request, encoding: .utf8), + let line = head.components(separatedBy: "\r\n").first else { return } + let parts = line.components(separatedBy: " ") + guard parts.count >= 2 else { return } + let path = String(parts[1].dropFirst()) + + lock.lock() + while held.contains(path), !stopped { lock.wait() } + lock.unlock() + + let body: Data + switch path { + case "master.m3u8": + body = Data(""" + #EXTM3U + #EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="a",NAME="de",LANGUAGE="de",DEFAULT=YES,AUTOSELECT=YES,URI="a.m3u8" + #EXT-X-STREAM-INF:BANDWIDTH=1800000,CODECS="avc1.4d401e,mp4a.40.2",RESOLUTION=720x576,AUDIO="a" + v.m3u8 + + """.utf8) + case "v.m3u8": body = mediaPlaylist(prefix: "v") + case "a.m3u8": body = mediaPlaylist(prefix: "a") + default: + guard path.hasSuffix(".ts"), let index = Int(path.dropFirst(1).dropLast(3)) else { + let missing = "HTTP/1.1 404 Not Found\r\nContent-Length: 0\r\nConnection: close\r\n\r\n" + _ = missing.withCString { write(conn, $0, strlen($0)) } + return + } + body = segment(tag: UInt8(truncatingIfNeeded: (path.hasPrefix("a") ? 100 : 0) + index)) + } + let header = "HTTP/1.1 200 OK\r\nContent-Length: \(body.count)\r\nConnection: close\r\n\r\n" + _ = header.withCString { write(conn, $0, strlen($0)) } + body.withUnsafeBytes { raw in + var offset = 0 + while offset < raw.count { + let n = write(conn, raw.baseAddress!.advanced(by: offset), raw.count - offset) + guard n > 0 else { return } + offset += n + } + } + } +} + +/// Reads a reader on a thread of its own, optionally stopping at a byte budget WITHOUT parking (a +/// consumer that holds what it has and asks for no more, like the cutter's lookahead). +private final class ReaderDrain: @unchecked Sendable { + private let lock = NSLock() + private var _bytes = 0 + var bytes: Int { lock.lock(); defer { lock.unlock() }; return _bytes } + + init(_ reader: IOReader, stopAfter budget: Int? = nil) { + Thread.detachNewThread { [self] in + var buf = [UInt8](repeating: 0, count: 64) + while true { + if let budget, bytes >= budget { return } + let want = budget.map { min(buf.count, $0 - bytes) } ?? buf.count + let n = buf.withUnsafeMutableBufferPointer { reader.read($0.baseAddress, size: Int32(want)) } + if n <= 0 { return } + lock.lock() + _bytes += Int(n) + lock.unlock() + } + } + } +} + +final class Issue684PumpJoinSpentTests: XCTestCase { + + private func waitUntil(_ what: String, timeout: TimeInterval = 60, _ condition: () -> Bool) { + let deadline = Date().addingTimeInterval(timeout) + while !condition() { + if Date() >= deadline { return XCTFail("timed out waiting for: \(what)") } + usleep(2_000) + } + } + + // MARK: - The rule + + func testVideoOnlyIsTheVideoReaderParkedOnACommittedJoin() { + XCTAssertTrue(HLSLiveIngestReader.pumpJoinIsSpent(main: (true, true), companion: nil)) + XCTAssertFalse(HLSLiveIngestReader.pumpJoinIsSpent(main: (true, false), companion: nil)) + XCTAssertFalse(HLSLiveIngestReader.pumpJoinIsSpent(main: (false, true), companion: nil), + "parked between two segments of a join still arriving") + } + + /// The reported shape: the audio rendition ends before the video does, so the cutter parks on + /// the AUDIO reader while video bytes are still queued. That is a spent join. + func testShortAudioParksTheCutterOnTheCompanion() { + XCTAssertTrue(HLSLiveIngestReader.pumpJoinIsSpent( + main: (committed: true, parked: false), companion: (started: true, committed: true, parked: true))) + } + + /// The audio playlist publishes a little after the video one, so at the join the companion's + /// batch is still arriving while the cutter already sits on an empty reader. + func testAudioPublishedLaterIsNotSpentUntilItsOwnJoinIsCommitted() { + XCTAssertFalse(HLSLiveIngestReader.pumpJoinIsSpent( + main: (true, true), companion: (started: true, committed: false, parked: false))) + XCTAssertFalse(HLSLiveIngestReader.pumpJoinIsSpent( + main: (true, false), companion: (started: true, committed: false, parked: true)), + "parked on the companion between two of its join segments") + XCTAssertTrue(HLSLiveIngestReader.pumpJoinIsSpent( + main: (true, true), companion: (started: true, committed: true, parked: false))) + } + + func testACompanionNobodyReadsDoesNotCount() { + XCTAssertTrue(HLSLiveIngestReader.pumpJoinIsSpent( + main: (true, true), companion: (started: false, committed: false, parked: false))) + } + + func testBothCommittedAndNeitherParkedIsACutterStillWorking() { + XCTAssertFalse(HLSLiveIngestReader.pumpJoinIsSpent( + main: (true, false), companion: (started: true, committed: true, parked: false))) + } + + // MARK: - The reader itself + + /// A consumer parked on an empty reader is not a spent join while the join batch is still being + /// fetched: the third segment is held at the origin, the reader has drained the first two and + /// waits, and the fact has to stay false until that segment is committed and drained too. + func testReaderIsNotSpentWhileItsJoinBatchIsStillArriving() throws { + let origin = try XCTUnwrap(HeldSegmentOrigin(hold: ["v2.ts"])) + defer { origin.stop() } + let reader = HLSLiveIngestReader(playlistURL: origin.url("v.m3u8")) + defer { reader.close() } + XCTAssertFalse(reader.joinIsSpent, "nothing has been read") + + let drain = ReaderDrain(reader) + let two = 2 * HeldSegmentOrigin.segmentBytes + waitUntil("two segments drained and the consumer parked") { + drain.bytes == two && reader.pumpJoinState.parked + } + XCTAssertFalse(reader.pumpJoinState.committed) + XCTAssertFalse(reader.joinIsSpent, "empty and parked, but the join is not all here") + + origin.release("v2.ts") + waitUntil("the join to be spent") { reader.joinIsSpent } + XCTAssertEqual(drain.bytes, 3 * HeldSegmentOrigin.segmentBytes) + XCTAssertTrue(reader.pumpJoinState.committed) + } + + /// Demuxed audio, the reported shape: the cutter holds video it cannot place and waits on the + /// audio reader. The video reader is never parked. + func testReaderWithACompanionIsSpentWhenTheCutterParksOnTheAudio() throws { + let origin = try XCTUnwrap(HeldSegmentOrigin(hold: [])) + defer { origin.stop() } + let reader = HLSLiveIngestReader(playlistURL: origin.url("master.m3u8")) + defer { reader.close() } + + // The video side takes one segment and asks for no more: two stay queued, nobody is parked. + let video = ReaderDrain(reader, stopAfter: HeldSegmentOrigin.segmentBytes) + waitUntil("the companion to be installed") { reader.companionAudioReader != nil } + waitUntil("the video join to be committed") { reader.pumpJoinState.committed } + waitUntil("the video side to stop") { video.bytes == HeldSegmentOrigin.segmentBytes } + XCTAssertFalse(reader.pumpJoinState.parked) + XCTAssertFalse(reader.joinIsSpent, "video queued, nobody waiting, audio not yet read") + + let companion = try XCTUnwrap(reader.companionAudioReader) + let audio = ReaderDrain(companion) + waitUntil("the join to be spent on the audio side") { reader.joinIsSpent } + XCTAssertEqual(audio.bytes, 3 * HeldSegmentOrigin.segmentBytes) + XCTAssertFalse(reader.pumpJoinState.parked, "the video reader never parked, and it did not have to") + } + + /// Audio published a moment after video: the cutter has drained the video and parked on it + /// while the audio rendition's last join segment is still at the origin. + func testReaderWithACompanionWaitsForTheAudioJoinToBeCommitted() throws { + let origin = try XCTUnwrap(HeldSegmentOrigin(hold: ["a2.ts"])) + defer { origin.stop() } + let reader = HLSLiveIngestReader(playlistURL: origin.url("master.m3u8")) + defer { reader.close() } + + let video = ReaderDrain(reader) + waitUntil("the companion to be installed") { reader.companionAudioReader != nil } + let companion = try XCTUnwrap(reader.companionAudioReader) + let audio = ReaderDrain(companion) + let three = 3 * HeldSegmentOrigin.segmentBytes + waitUntil("the video drained and parked, the audio parked two segments in") { + video.bytes == three && reader.pumpJoinState.parked + && audio.bytes == 2 * HeldSegmentOrigin.segmentBytes + } + XCTAssertFalse(reader.joinIsSpent, "the audio rendition's join is not all here") + + origin.release("a2.ts") + waitUntil("the join to be spent") { reader.joinIsSpent } + XCTAssertEqual(audio.bytes, three) + } + + // MARK: - The gate asks in the right order + + /// The fact is monotone, so a snapshot taken AFTER it read true holds everything the join will + /// cut. Taken before, the cutter can append its last segment and park between the two reads and + /// the seal is paid from the sum without it. A race two reads wide is not something a policy + /// test can stage, so this reads the site, as `Issue588RefusalLatchLifetimeTests` does. + func testTheGateAsksSpentBeforeItTakesTheSnapshot() throws { + let source = URL(fileURLWithPath: #filePath) + .deletingLastPathComponent().deletingLastPathComponent().deletingLastPathComponent() + .appendingPathComponent("Sources/AetherEngine/Video/VideoSegmentProvider.swift") + let text = try String(contentsOf: source, encoding: .utf8) + let gateStart = try XCTUnwrap(text.range(of: "func waitForFirstLiveSegment(timeout: TimeInterval) -> Bool {")) + let gateEnd = try XCTUnwrap(text.range(of: "private func accountForFirstServe(", + range: gateStart.upperBound.. (VideoSegmentProvider, SegmentCache) { + let cache = SegmentCache(forwardWindow: 10, backwardWindow: 10) + let provider = VideoSegmentProvider( + cache: cache, + segments: [], + codecsString: "avc1.4D001E,mp4a.40.2", + supplementalCodecs: nil, + resolution: (720, 576), + videoRange: .sdr, + frameRate: 25, + hdcpLevel: nil, + sourceBitrate: 1_500_000, + isLive: true, + liveWindowSizing: LiveWindowSizing(targetSegmentDurationSeconds: 0.5, dvrWindowSeconds: nil), + allowsBoundedDegradedStart: true, + firstServeLatchCoversEngineCut: latchArm + ) + return (provider, cache) + } + + private func seconds(_ body: () -> Void) -> Double { + let start = DispatchTime.now() + body() + return Double(DispatchTime.now().uptimeNanoseconds - start.uptimeNanoseconds) / 1_000_000_000 + } + + func testTheLatchIsOnUnlessTheEnvironmentOptsOut() { + XCTAssertEqual(LiveEdgePolicy.firstServeLatchAllArmed, + ProcessInfo.processInfo.environment["AETHER_FIRST_SERVE_LATCH_ALL"] != "0") + } + + /// Two 0.2 s segments against a 3 s holdback: the bounded start, grace 0.5 s. + func testArmedASelfCutSourceLetsTheSecondRequestThrough() { + let (provider, cache) = makeProvider(latchArm: true) + defer { cache.close() } + provider.appendLiveSegment(index: 0, startSeconds: 0, durationSeconds: 0.2) + provider.appendLiveSegment(index: 1, startSeconds: 0.2, durationSeconds: 0.2) + + var served = false + let first = seconds { served = provider.waitForFirstLiveSegment(timeout: 3) } + XCTAssertTrue(served) + XCTAssertGreaterThanOrEqual(first, 0.45, "the first request still pays the grace") + + let second = seconds { served = provider.waitForFirstLiveSegment(timeout: 3) } + XCTAssertTrue(served) + XCTAssertLessThan(second, 0.2, "the arm latches the gate once a manifest has gone out") + } + + func testArmedAnUnservedGateStaysAGate() { + let (provider, cache) = makeProvider(latchArm: true) + defer { cache.close() } + XCTAssertFalse(provider.waitForFirstLiveSegment(timeout: 0.1)) + XCTAssertFalse(provider.waitForFirstLiveSegment(timeout: 0.1)) + } + + /// The repeat line is the measurement both arms are compared on, so it must appear in the + /// unarmed one too, once, with the held interval. + func testTheRepeatRequestIsAccountedOnceInEitherArm() { + for armed in [false, true] { + let (provider, cache) = makeProvider(latchArm: armed) + defer { cache.close() } + provider.appendLiveSegment(index: 0, startSeconds: 0, durationSeconds: 0.2) + provider.appendLiveSegment(index: 1, startSeconds: 0.2, durationSeconds: 0.2) + + let capture = EngineLogCapture() + defer { capture.end() } + + XCTAssertTrue(provider.waitForFirstLiveSegment(timeout: 3)) + XCTAssertTrue(provider.waitForFirstLiveSegment(timeout: 3)) + let afterSecond = capture.matching("repeat live manifest request held").count + XCTAssertTrue(provider.waitForFirstLiveSegment(timeout: 3)) + + let repeats = capture.matching("repeat live manifest request held") + XCTAssertGreaterThanOrEqual(afterSecond, 1, "armed=\(armed)") + XCTAssertEqual(repeats.count, afterSecond, "armed=\(armed): a third request adds no line") + if armed { + XCTAssertTrue(repeats.contains { $0.contains("first-serve latch") }) + } else { + XCTAssertTrue(repeats.contains { $0.contains("fastZap bounded start") }) + } + } + } +} diff --git a/Tests/AetherEngineTests/Issue687RetentionClaimsTests.swift b/Tests/AetherEngineTests/Issue687RetentionClaimsTests.swift new file mode 100644 index 000000000..af446d216 --- /dev/null +++ b/Tests/AetherEngineTests/Issue687RetentionClaimsTests.swift @@ -0,0 +1,83 @@ +// Tests/AetherEngineTests/Issue687RetentionClaimsTests.swift +// #687: every session sized its retention from the raw free space, so sessions started together each +// took a quarter of the same space. A claim is now sized from what the other running claims leave. +import Foundation +import Testing +@testable import AetherEngine + +@Suite("Retention claims across concurrent sessions") +struct Issue687RetentionClaimsTests { + + private func session(_ ledger: RetentionClaims, free: Int64?) -> RetentionClaims.Claim { + ledger.claim(volumeAvailableBytes: free) { + HLSVideoEngine.sessionRetentionBudgetBytes(volumeAvailableBytes: $0) + } + } + + @Test("A session alone gets what it always got") + func aloneIsUnchanged() { + let ledger = RetentionClaims() + let claim = session(ledger, free: 4 << 30) + #expect(claim.bytes == 1 << 30) + #expect(claim.heldBackBytes == 0) + } + + @Test("Four sessions on a tight volume no longer claim all of it") + func fourSessionsLeaveTheVolumeRoom() { + let ledger = RetentionClaims() + let free: Int64 = 4 << 30 + let claims = (0..<4).map { _ in session(ledger, free: free) } + let total = claims.reduce(0) { $0 + $1.bytes } + #expect(claims[0].bytes == 1 << 30) + #expect(claims[1].bytes == (3 << 30) / 4) + #expect(Double(total) / Double(free) < 0.69) + #expect(claims.allSatisfy { $0.bytes > 0 }) + } + + @Test("A roomy volume still gives every session the cap") + func roomyVolumeKeepsTheCap() { + let ledger = RetentionClaims() + let claims = (0..<4).map { _ in session(ledger, free: 32 << 30) } + #expect(claims.allSatisfy { $0.bytes == 2 << 30 }) + } + + @Test("What a session has written is not held back twice") + func writtenBytesAreNotHeldBack() { + let ledger = RetentionClaims() + let first = session(ledger, free: 4 << 30) + first.track { 1 << 30 } + // The first session's GiB is on disk, so the measurement already misses it. + let second = session(ledger, free: 3 << 30) + #expect(second.heldBackBytes == 0) + #expect(second.bytes == (3 << 30) / 4) + } + + @Test("A released claim frees its share, and a dropped one does too") + func releaseFreesTheShare() { + let ledger = RetentionClaims() + let first = session(ledger, free: 4 << 30) + first.release() + first.release() + #expect(session(ledger, free: 4 << 30).bytes == 1 << 30) + #expect(session(ledger, free: 4 << 30).bytes == 1 << 30) + } + + @Test("Unknown capacity stays unknown and keeps the cap") + func unknownCapacityKeepsTheCap() { + let ledger = RetentionClaims() + let first = session(ledger, free: nil) + let second = session(ledger, free: nil) + #expect(first.bytes == 2 << 30) + #expect(second.bytes == 2 << 30) + } + + @Test("The software live ring keeps its floor when others hold everything back") + func liveRingKeepsItsFloor() { + let ledger = RetentionClaims() + let first = session(ledger, free: 1 << 30) + let ring = ledger.claim(volumeAvailableBytes: Int64(first.bytes) / 2) { + PacketRingBuffer.liveByteBudget(volumeAvailableBytes: $0) + } + #expect(ring.bytes == PacketRingBuffer.minimumLiveByteBudget) + } +} diff --git a/Tests/AetherEngineTests/Issue693TransientRefusalTests.swift b/Tests/AetherEngineTests/Issue693TransientRefusalTests.swift new file mode 100644 index 000000000..5db50ede2 --- /dev/null +++ b/Tests/AetherEngineTests/Issue693TransientRefusalTests.swift @@ -0,0 +1,108 @@ +// #693: a transient refusal at byte 0. KIPTV's Xtream origin answered the ranged open of a series +// episode with a 403, and the unranged GET the reader sent right after it, to the same resource, was +// answered 200. #378 had read every 401/403/404/410 at byte 0 as "the origin's answer to the +// RESOURCE", so that one 403 decided the whole session: the reader streamed forward-only, the load +// promoted the source to a sequential origin, the saved position was dropped and every seek past +// the downloaded window snapped back. Ten hours later the same origin answered every ranged request +// with a 206. +// +// A refusal that the next request does not repeat says nothing about the resource. The open now asks +// the same range once more before it settles, and only a second refusal sends it down the #378 path. +import Foundation +import Testing +@testable import AetherEngine + +@Suite("#693 a refusal at byte 0 that is not repeated keeps the source seekable", .serialized) +struct Issue693TransientRefusalTests { + + private static let fileSize = 8 * 1024 * 1024 + + private final class Counter: @unchecked Sendable { + private let lock = NSLock() + private var value = 0 + func next() -> Int { + lock.lock(); defer { lock.unlock() } + value += 1 + return value + } + } + + /// A range-capable origin. `refuseByteZero` decides, per ranged request at byte 0 (1-based), whether + /// that one is answered 403. + private static func origin(refuseByteZero: @escaping @Sendable (Int) -> Bool) throws -> ScriptedOriginServer { + let byteZero = Counter() + return try #require(ScriptedOriginServer { request in + let total = Int64(fileSize) + guard let range = request.range else { + return .init(status: 200, declaredLength: total, bodyBytes: fileSize) + } + if range.hasPrefix("bytes=-") { + return .init(status: 206, declaredLength: 65_536, + contentRange: "bytes \(total - 65_536)-\(total - 1)/\(total)", bodyBytes: 65_536) + } + let bounds = range.dropFirst("bytes=".count).split(separator: "-", omittingEmptySubsequences: false) + let start = Int64(bounds.first ?? "") ?? 0 + let end = min(Int64(bounds.count > 1 ? bounds[1] : "") ?? (total - 1), total - 1) + if start == 0, refuseByteZero(byteZero.next()) { + return .init(status: 403, declaredLength: 0) + } + let length = Int(end - start + 1) + return .init(status: 206, declaredLength: Int64(length), + contentRange: "bytes \(start)-\(end)/\(total)", bodyBytes: length) + }) + } + + @Test("a 403 that the second ranged open does not repeat leaves the source seekable, without an unranged GET") + func transientRefusalKeepsSeekability() throws { + let server = try Self.origin { $0 == 1 } + defer { server.stop() } + let url = URL(string: "http://127.0.0.1:\(server.port)/series/episode.mp4")! + + let reader = AVIOReader(url: url) + defer { reader.markClosed(); reader.close() } + try reader.open() + #expect(reader.isSeekable, "one 403 at byte 0 turned a range-capable origin forward-only") + + var buffer = [UInt8](repeating: 0, count: 64 * 1024) + let read = buffer.withUnsafeMutableBufferPointer { reader.read(into: $0.baseAddress!, size: Int32($0.count)) } + #expect(read > 0, "the second ranged open delivered nothing: \(read)") + + let requests = server.requests + #expect(!requests.contains { $0.method == "GET" && $0.range == nil }, + "settled on the unranged GET although the range form was served: \(requests)") + #expect(!requests.contains { $0.method == "HEAD" }) + #expect(requests.filter { $0.range?.hasPrefix("bytes=0-") == true }.count == 2, + "expected the refused ranged open and exactly one more: \(requests)") + } + + @Test("a 403 that the second ranged open repeats settles forward-only on one unranged GET, as #378 does") + func repeatedRefusalSettlesForwardOnly() throws { + let server = try Self.origin { _ in true } + defer { server.stop() } + let url = URL(string: "http://127.0.0.1:\(server.port)/series/episode.mp4")! + + let reader = AVIOReader(url: url) + defer { reader.markClosed(); reader.close() } + try reader.open() + #expect(!reader.isSeekable) + + let requests = server.requests + #expect(requests.filter { $0.range?.hasPrefix("bytes=0-") == true }.count == 2, + "the range form was not asked a second time, or more than that: \(requests)") + #expect(requests.filter { $0.method == "GET" && $0.range == nil }.count == 1, "\(requests)") + #expect(!requests.contains { $0.method == "HEAD" }) + #expect(!requests.contains { $0.range == "bytes=0-1" }) + } + + // MARK: - What the host sees when the session does settle + + @MainActor + @Test("isSequentialOrigin does not outlive the session it described") + func sequentialFlagClearsOnStop() throws { + let engine = try AetherEngine() + #expect(!engine.isSequentialOrigin) + engine.isSequentialOrigin = true + engine.stop() + #expect(!engine.isSequentialOrigin, "the next load would start out reporting a sequential origin") + } +} diff --git a/Tests/AetherEngineTests/Issue694AudioHostEndOfMediaParkTests.swift b/Tests/AetherEngineTests/Issue694AudioHostEndOfMediaParkTests.swift new file mode 100644 index 000000000..ef5571358 --- /dev/null +++ b/Tests/AetherEngineTests/Issue694AudioHostEndOfMediaParkTests.swift @@ -0,0 +1,143 @@ +import Testing +import Foundation +@testable import AetherEngine + +/// #694: the audio-only host reported end of media while its synchronizer kept rate 1, so +/// `currentTime` walked past `duration` without bound. AE#374 closed the same defect on the software +/// host; this pins it on the audio-only one, end to end against a real decode. +@Suite("Audio host parks its clock at end of media (#694)", .serialized) +struct Issue694AudioHostEndOfMediaParkTests { + + @MainActor + @Test("after end of media the clock stands still on the last sample, even when the park runs late") + func clockParksOnTheLastSample() async throws { + let host = AudioPlaybackHost() + let demuxer = Demuxer() + try demuxer.open(reader: DataIOReader(data: makeWAV(seconds: 1))) + try await host.load(demuxer: demuxer, startPosition: nil, audioSourceStreamIndex: nil) + defer { host.stop() } + host.play() + + let deadline = Date().addingTimeInterval(8) + while !host.didReachEnd, Date() < deadline { + try await Task.sleep(nanoseconds: 50_000_000) + } + #expect(host.didReachEnd) + // What a loaded runner does to the deferred park (CI: parked at 1.149 s): hold the main actor + // past the queued tail, so the park runs after the clock has walked on. + usleep(500_000) + + let parkDeadline = Date().addingTimeInterval(2) + while host.clockRateForTesting != 0, Date() < parkDeadline { + try await Task.sleep(nanoseconds: 20_000_000) + } + #expect(host.clockRateForTesting == 0) + #expect(host.rate == 0) + + let first = try #require(host.clockSecondsForTesting) + try await Task.sleep(nanoseconds: 500_000_000) + let second = try #require(host.clockSecondsForTesting) + #expect(abs(second - first) < 0.01) + #expect(abs(first - 1.0) < 0.1) + } + + @Test("a park that runs after the clock passed its target puts it back on the target") + func lateParkLandsOnTheTarget() { + #expect(SoftwareEndOfMediaClock.parkSeconds(clockSeconds: 1.149, notAfter: 1.0) == 1.0) + #expect(SoftwareEndOfMediaClock.parkSeconds(clockSeconds: 0.98, notAfter: 1.0) == nil) + #expect(SoftwareEndOfMediaClock.parkSeconds(clockSeconds: 1.149, notAfter: .infinity) == nil) + #expect(SoftwareEndOfMediaClock.parkSeconds(clockSeconds: .nan, notAfter: 1.0) == nil) + } + + @Test("the deferred park waits out the tail in source time, and polls a stopped clock") + func parkWaitIsSourceTime() { + // 0.25 s of tail at 0.5x is half a second of wall time, not a quarter. + #expect(SoftwareEndOfMediaClock.parkWaitSeconds(clockSeconds: 0.75, rate: 0.5, target: 1.0) == 0.5) + #expect(SoftwareEndOfMediaClock.parkWaitSeconds(clockSeconds: 0.75, rate: 1, target: 1.0) == 0.25) + #expect(SoftwareEndOfMediaClock.parkWaitSeconds(clockSeconds: 0.75, rate: 0, target: 1.0) + == SoftwareEndOfMediaClock.stoppedClockPollSeconds) + #expect(SoftwareEndOfMediaClock.parkWaitSeconds(clockSeconds: 1.0, rate: 0.5, target: 1.0) == nil) + #expect(SoftwareEndOfMediaClock.parkWaitSeconds(clockSeconds: .nan, rate: 1, target: 1.0) == nil) + } + + @MainActor + @Test("below 1x the clock still parks on the last sample, not short of it") + func halfSpeedParksOnTheLastSample() async throws { + let host = AudioPlaybackHost() + let demuxer = Demuxer() + try demuxer.open(reader: DataIOReader(data: makeWAV(seconds: 1))) + try await host.load(demuxer: demuxer, startPosition: nil, audioSourceStreamIndex: nil) + defer { host.stop() } + host.setResumeRate(0.5) + host.play() + + try await waitForParkedClock(host, within: 8) + let first = try #require(host.clockSecondsForTesting) + // Parked on wall time, the clock stopped about 0.11 s short of the end at 0.5x. + #expect(abs(first - 1.0) < 0.02) + } + + @MainActor + @Test("a pause during the tail holds the park until the clock reaches the last sample") + func pauseDuringTheTailDefersThePark() async throws { + // The playthrough wait releases up to 0.25 s of source before the end, and the pause has to + // land in that tail, before the park. At 0.25x the tail lasts about a second of wall time; + // a run a loaded machine still delays past it (the clock already parked on the last + // sample, which a resume would carry on from) starts over. + for _ in 0..<4 { + let host = AudioPlaybackHost() + let demuxer = Demuxer() + try demuxer.open(reader: DataIOReader(data: makeWAV(seconds: 1))) + try await host.load(demuxer: demuxer, startPosition: nil, audioSourceStreamIndex: nil) + defer { host.stop() } + host.setResumeRate(0.25) + host.play() + + let deadline = Date().addingTimeInterval(16) + while !host.didReachEnd, Date() < deadline { + try await Task.sleep(nanoseconds: 10_000_000) + } + try #require(host.didReachEnd) + host.pause() + guard try #require(host.clockSecondsForTesting) < 0.99 else { continue } + try await Task.sleep(nanoseconds: 600_000_000) + host.play() + + // A park that fired during the pause left nothing to stop the resumed clock. + try await waitForParkedClock(host, within: 4) + let first = try #require(host.clockSecondsForTesting) + #expect(abs(first - 1.0) < 0.02) + return + } + Issue.record("the pause never landed inside the tail in four runs") + } + + @MainActor + private func waitForParkedClock(_ host: AudioPlaybackHost, within seconds: TimeInterval) async throws { + let deadline = Date().addingTimeInterval(seconds) + while !(host.didReachEnd && host.clockRateForTesting == 0), Date() < deadline { + try await Task.sleep(nanoseconds: 20_000_000) + } + #expect(host.didReachEnd) + #expect(host.clockRateForTesting == 0) + #expect(host.rate == 0) + let first = try #require(host.clockSecondsForTesting) + try await Task.sleep(nanoseconds: 300_000_000) + let second = try #require(host.clockSecondsForTesting) + #expect(abs(second - first) < 0.01) + } + + private func makeWAV(seconds: Double) -> Data { + let sampleRate = 48_000, channels = 2 + let pcm = Data(count: Int(Double(sampleRate) * seconds) * channels * 2) + var d = Data() + func str(_ s: String) { d.append(s.data(using: .ascii)!) } + func u32(_ v: UInt32) { withUnsafeBytes(of: v.littleEndian) { d.append(contentsOf: $0) } } + func u16(_ v: UInt16) { withUnsafeBytes(of: v.littleEndian) { d.append(contentsOf: $0) } } + str("RIFF"); u32(UInt32(36 + pcm.count)); str("WAVE") + str("fmt "); u32(16); u16(1); u16(UInt16(channels)); u32(UInt32(sampleRate)) + u32(UInt32(sampleRate * channels * 2)); u16(UInt16(channels * 2)); u16(16) + str("data"); u32(UInt32(pcm.count)); d.append(pcm) + return d + } +} diff --git a/Tests/AetherEngineTests/Issue93ItemDeathReviveTests.swift b/Tests/AetherEngineTests/Issue93ItemDeathReviveTests.swift index 91b13029c..3a49a6298 100644 --- a/Tests/AetherEngineTests/Issue93ItemDeathReviveTests.swift +++ b/Tests/AetherEngineTests/Issue93ItemDeathReviveTests.swift @@ -78,59 +78,6 @@ struct Issue93ItemDeathReviveTests { consumerIsPaused: false, allowPausedConsumer: false)) } - // MARK: - Viewer pause before the death - - private static let second: UInt64 = 1_000_000_000 - - @Test("an item that died under a viewer's pause is reloaded paused") - func deathUnderViewerPause() { - // The field report: paused on an Apple TV, the item died minutes later and the reload - // started playback with nobody touching the remote. - #expect(NativeAVPlayerHost.transportPausedBeforeFailure( - pausedSinceUptime: 10 * Self.second, failureUptime: 460 * Self.second)) - } - - @Test("an item that died while rolling is reloaded playing") - func deathWhileRolling() { - #expect(!NativeAVPlayerHost.transportPausedBeforeFailure( - pausedSinceUptime: nil, failureUptime: 460 * Self.second)) - } - - @Test("the dead item's own pause, landing just before the notification, is not the viewer's") - func deathParksItsOwnPause() { - #expect(!NativeAVPlayerHost.transportPausedBeforeFailure( - pausedSinceUptime: 460 * Self.second - Self.second / 20, - failureUptime: 460 * Self.second)) - } - - @Test("a pause stamped after the notification is not the viewer's") - func pauseAfterNotification() { - #expect(!NativeAVPlayerHost.transportPausedBeforeFailure( - pausedSinceUptime: 461 * Self.second, failureUptime: 460 * Self.second)) - } - - @Test("with no press since the death, the transport it died in decides") - func reloadFollowsTransportAtDeath() { - #expect(!NativeAVPlayerHost.itemDeathReloadResumesPlaying( - diedUnderPause: true, commandSinceFailure: nil, transportRolling: false)) - #expect(NativeAVPlayerHost.itemDeathReloadResumesPlaying( - diedUnderPause: false, commandSinceFailure: nil, transportRolling: false)) - } - - @Test("a Play or Pause pressed while the death is confirmed outranks the transport it died in") - func pressDuringConfirmationDecides() { - #expect(NativeAVPlayerHost.itemDeathReloadResumesPlaying( - diedUnderPause: true, commandSinceFailure: true, transportRolling: false)) - #expect(!NativeAVPlayerHost.itemDeathReloadResumesPlaying( - diedUnderPause: false, commandSinceFailure: false, transportRolling: true)) - } - - @Test("a Play from outside the engine after a paused death resumes the reload") - func externalPlayAfterPausedDeath() { - #expect(NativeAVPlayerHost.itemDeathReloadResumesPlaying( - diedUnderPause: true, commandSinceFailure: nil, transportRolling: true)) - } - // MARK: - Host-side counting decision @Test("loopback path counts an end failure after playback was established") @@ -151,3 +98,68 @@ struct Issue93ItemDeathReviveTests { surfaceEndFailures: false, hasEverPlayed: false)) } } + +/// #93: item death parks AVPlayer at `.paused` whatever the viewer wanted, so the stage-2 reload runs +/// for a paused consumer too. Whether the fresh item then PLAYS is the viewer's call, read from the +/// host's durable intent (#122), which the in-place swap keeps. +/// +/// Field log, Apple TV 4K 3rd gen, tvOS 27.0, HDR10+ HEVC Matroska: paused at 2605.23 s, the item died +/// with -11868 two minutes later as the tvOS screensaver took the display, and the recovery called +/// `play()` on the fresh item. The title started itself and the screensaver was dismissed. +@Suite("#93: the stage-2 reload resumes only a viewer who was playing") +@MainActor +struct ItemDeathRecoveryTransportTests { + + private let url = URL(fileURLWithPath: "/nonexistent-item-death-recovery-transport-test.m3u8") + + /// An engine holding a native host with one mounted item, the state the recovery reloads from. + private func engineWithMountedItem() throws -> (AetherEngine, NativeAVPlayerHost) { + let engine = try AetherEngine() + let host = NativeAVPlayerHost() + engine.nativeHost = host + engine.currentAVPlayer = host.avPlayer + host.load(url: url, startPosition: 2605.23, contract: .init()) + return (engine, host) + } + + @Test("A paused viewer stays paused through the reload") + func pausedViewerStaysPaused() throws { + let (engine, host) = try engineWithMountedItem() + defer { host.tearDown() } + host.pause() + + engine.forceStalledConsumerReloadForTesting() + + #expect(!host.transportIntentIsPlaying) + #expect(host.avPlayer.rate == 0) + } + + @Test("A playing viewer is resumed on the fresh item, as before") + func playingViewerResumes() throws { + let (engine, host) = try engineWithMountedItem() + defer { host.tearDown() } + host.play() + + engine.forceStalledConsumerReloadForTesting() + + #expect(host.transportIntentIsPlaying) + } + + /// The #98 media fallback needs a live loopback session to run, so this reads its call site, as + /// the #535 latch test does. + @Test("The media fallback asks the same intent before it plays") + func fallbackAsksTheIntent() throws { + let source = URL(fileURLWithPath: #filePath) + .deletingLastPathComponent() + .deletingLastPathComponent() + .deletingLastPathComponent() + .appendingPathComponent("Sources/AetherEngine/AetherEngine.swift") + let text = try #require(try? String(contentsOf: source, encoding: .utf8)) + let fn = try #require(text.range(of: "func fallBackToMediaPlaylist(")) + let end = try #require(text[fn.upperBound...].range(of: "\n }\n")) + let body = String(text[fn.lowerBound.. SoftwarePathEscalation.Availability { + SoftwarePathEscalation.Availability( + alreadyEscalated: false, preferredDecodePath: path, nativeRemoteHLS: false, + hostAllowsEscalation: hostAllows) + } + + @Test("The join verdict is offered the software path like a media failure") + func joinVerdictEscalates() { + #expect(SoftwarePathEscalation.shouldEscalate( + errorDomain: SoftwarePathEscalation.liveJoinErrorDomain, availability: Self.availability())) + } + + @Test("A host that declined the rung, or a session already on software, is not escalated") + func declinedOrPinned() { + #expect(!SoftwarePathEscalation.shouldEscalate( + errorDomain: SoftwarePathEscalation.liveJoinErrorDomain, + availability: Self.availability(hostAllows: false))) + #expect(!SoftwarePathEscalation.shouldEscalate( + errorDomain: SoftwarePathEscalation.liveJoinErrorDomain, + availability: Self.availability(path: .software))) + } +} diff --git a/Tests/AetherEngineTests/LiveRecordingAPITests.swift b/Tests/AetherEngineTests/LiveRecordingAPITests.swift index 480f2c38e..b945e2847 100644 --- a/Tests/AetherEngineTests/LiveRecordingAPITests.swift +++ b/Tests/AetherEngineTests/LiveRecordingAPITests.swift @@ -63,6 +63,8 @@ struct LiveRecordingAPITests { #expect(host.installedSink != nil) engine.endRecordingIfRunning(reason: .sourceReset) + #expect(host.installedSink == nil, "the sink comes off the route at once, before the drain") + await engine.recordingFinish?.value #expect(engine.recordingState == .ended(.sourceReset)) #expect(host.installedSink == nil, "the sink must be removed from the route") } @@ -77,6 +79,7 @@ struct LiveRecordingAPITests { try engine._testStartRecordingWithStubHost(to: url, host: host) engine.endRecordingIfRunning(reason: .stoppedByHost) engine.endRecordingIfRunning(reason: .sessionEnded) + await engine.recordingFinish?.value #expect(engine.recordingState == .ended(.stoppedByHost)) } @@ -105,4 +108,60 @@ struct LiveRecordingAPITests { await engine.stopRecording() #expect(engine.recordingState == .ended(.stoppedByHost)) } + + /// Audit CORE-5: the writer reports a failure from its teardown queue, after the host may have + /// ended that recording and started the next. + @Test("a late failure of the previous recording does not end the next one") + func staleFailureDoesNotEndTheNextRecording() async throws { + let engine = try AetherEngine() + let host = AetherEngine.TestRecordingHost() + let first = tempURL() + let second = tempURL() + defer { + try? FileManager.default.removeItem(at: first) + try? FileManager.default.removeItem(at: second) + } + + try engine._testStartRecordingWithStubHost(to: first, host: host) + let firstGeneration = engine.recordingGeneration + engine.endRecordingIfRunning(reason: .sessionEnded) + try engine._testStartRecordingWithStubHost(to: second, host: host) + let running = engine.activeRecording + + engine.recordingDidFail(.writeTooSlow(bytesWritten: 1, queuedBytesDropped: 1), + generation: firstGeneration) + #expect(engine.activeRecording === running) + #expect(host.installedSink != nil) + #expect({ if case .recording = engine.recordingState { return true }; return false }()) + + engine.recordingDidFail(.writeTooSlow(bytesWritten: 1, queuedBytesDropped: 1), + generation: engine.recordingGeneration) + #expect(engine.activeRecording == nil) + #expect(engine.recordingState == .failed(.writeTooSlow(bytesWritten: 1, queuedBytesDropped: 1))) + running?.finish(reason: .sessionEnded) + } + + /// Audit REC-1: the drain and the trailer run off the main actor, and `.ended` still means the + /// file is closed. A recording started in the meantime keeps the state; the late `.ended` of + /// the one before it must not overwrite `.recording`. + @Test("a stop followed at once by a new recording leaves the new one's state standing") + func lateEndedDoesNotOverwriteTheNextRecording() async throws { + let engine = try AetherEngine() + let host = AetherEngine.TestRecordingHost() + let first = tempURL() + let second = tempURL() + defer { + try? FileManager.default.removeItem(at: first) + try? FileManager.default.removeItem(at: second) + } + + try engine._testStartRecordingWithStubHost(to: first, host: host) + engine.endRecordingIfRunning(reason: .sourceReset) + try engine._testStartRecordingWithStubHost(to: second, host: host) + await engine.recordingFinish?.value + #expect({ if case .recording(let p) = engine.recordingState { return p.url == second }; return false }()) + await engine.stopRecording() + #expect(engine.recordingState == .ended(.stoppedByHost)) + #expect(engine.activeRecording == nil) + } } diff --git a/Tests/AetherEngineTests/LiveRecordingQueueTests.swift b/Tests/AetherEngineTests/LiveRecordingQueueTests.swift index a11d994b7..6bc637ed9 100644 --- a/Tests/AetherEngineTests/LiveRecordingQueueTests.swift +++ b/Tests/AetherEngineTests/LiveRecordingQueueTests.swift @@ -98,4 +98,28 @@ struct LiveRecordingQueueTests { q.finish() #expect(q.offer(packet(64)) == false) } + + /// Audit REC-1: `finish` used to drain what was left on the calling thread, which for a stop was + /// the main actor, one `removeFirst()` per packet. + @Test("finish drains a large backlog in order on the writer queue, not the caller's thread") + func finishDrainsBacklogOnTheWriterQueue() { + let gate = Latch() + let seen = Locked<[Int64]>([]) + let offQueue = Locked(0) + let q = LiveRecordingQueue(ceilingBytes: 1 << 24) { item in + gate.wait() + let label = String(cString: __dispatch_queue_get_label(nil)) + if label != "de.superuser404.aether.recording.write" { offQueue.withLock { $0 += 1 } } + seen.withLock { $0.append(item.pts) } + } + let count = 100_000 + for i in 0.. URL { + FileManager.default.temporaryDirectory.appendingPathComponent("rec-\(UUID().uuidString).ts") + } + + private func isRecording(_ state: RecordingState) -> Bool { + if case .recording = state { return true } + return false + } + + private func annexB(_ size: Int, nalType: UInt8) -> [UInt8] { + var bytes: [UInt8] = [0x00, 0x00, 0x00, 0x01, nalType] + bytes.append(contentsOf: [UInt8](repeating: 0x88, count: max(0, size - bytes.count))) + return bytes + } + + // MARK: - FEA-105 + + @Test("a stop inside a scheduled teardown waits for its drain and publishes its failure") + func stopJoinsAScheduledTeardown() async throws { + let engine = try AetherEngine() + let host = AetherEngine.TestRecordingHost() + let url = tempURL() + defer { try? FileManager.default.removeItem(at: url) } + + try engine._testStartRecordingWithStubHost(to: url, host: host, ceilingBytes: 4096) + let writer = try #require(engine.activeRecording) + let gate = Latch() + // A cancelled wait below skips the open; the held writer must not outlive the test. + defer { gate.open() } + writer.beforeWriteForTesting = { gate.wait() } + + // The first packet enters the held drain, the second breaks the ceiling: the demux thread's + // offer is refused and the teardown is scheduled off it. + annexB(2000, nalType: 0x65).withUnsafeBytes { + writer.accept(packetBytes: $0, sourceStreamIndex: 0, pts: 0, dts: 0, duration: 3000, isKeyframe: true) + } + annexB(3000, nalType: 0x41).withUnsafeBytes { + writer.accept(packetBytes: $0, sourceStreamIndex: 0, pts: 3000, dts: 3000, duration: 3000, isKeyframe: false) + } + try await waitFor { writer.isClosed } + + engine.endRecordingIfRunning(reason: .stoppedByHost) + let publishedEarly = try await waitFor(upTo: .milliseconds(300)) { !self.isRecording(engine.recordingState) } + #expect(!publishedEarly, "the file is still being drained, so nothing may claim it closed") + + gate.open() + await engine.recordingFinish?.value + #expect(engine.recordingState == .failed(.writeTooSlow(bytesWritten: 0, queuedBytesDropped: 3000))) + #expect(engine.activeRecording == nil) + } + + @Test("a clean stop still ends with .ended") + func cleanStopStillEnds() async throws { + let engine = try AetherEngine() + let host = AetherEngine.TestRecordingHost() + let url = tempURL() + defer { try? FileManager.default.removeItem(at: url) } + + try engine._testStartRecordingWithStubHost(to: url, host: host) + await engine.stopRecording() + #expect(engine.recordingState == .ended(.stoppedByHost)) + } + + // MARK: - FEA-106 + + /// Holds `recordingFinish` open the way a slow drain does, and returns the handle that releases it. + private func holdPreviousFinish(_ engine: AetherEngine) -> AsyncStream.Continuation { + let (stream, release) = AsyncStream.makeStream() + engine.recordingFinish = Task { for await _ in stream { break } } + return release + } + + @MainActor private final class Reached { var value = false } + + /// Starts a recording and returns once the start is parked in its `recordingFinish` wait. + /// `startRecording` takes its snapshots and reaches that wait without suspending, so a task that + /// has entered it, seen from the main actor, is already waiting. `Task.yield()` would leave the + /// order to the executor. + private func startParkedInTheWait(_ engine: AetherEngine, to url: URL) async throws + -> Task { + let entered = Reached() + let starter = Task { @MainActor in + entered.value = true + try await engine.startRecording(to: url) + } + try await waitFor { entered.value } + return starter + } + + @Test("a start that a stop overtook while it waited does not begin recording") + func stopDuringTheStartsWaitCancelsIt() async throws { + let engine = try AetherEngine() + engine._testSetLiveRoute(isLive: true, route: .loopback) + let release = holdPreviousFinish(engine) + + let url = tempURL() + let starter = try await startParkedInTheWait(engine, to: url) + let serial = engine.recordingStopSerial + let stopper = Task { @MainActor in await engine.stopRecording() } + // The stop has to be counted before the release lets the start look again. + try await waitFor { engine.recordingStopSerial != serial } + + release.yield(()) + release.finish() + let outcome = await starter.result + await stopper.value + + #expect({ if case .failure(let error) = outcome { return error is CancellationError }; return false }(), + "the stop came after the start, so the start must not win: \(outcome)") + #expect(engine.activeRecording == nil) + #expect(!isRecording(engine.recordingState)) + } + + @Test("a start that a new load overtook while it waited does not record the new channel") + func loadDuringTheStartsWaitCancelsIt() async throws { + let engine = try AetherEngine() + engine._testSetLiveRoute(isLive: true, route: .loopback) + let release = holdPreviousFinish(engine) + + let url = tempURL() + let starter = try await startParkedInTheWait(engine, to: url) + engine.loadGeneration &+= 1 + + release.yield(()) + release.finish() + let outcome = await starter.result + + #expect({ if case .failure(let error) = outcome { return error is CancellationError }; return false }(), + "a zap ends the session the start was asked for: \(outcome)") + #expect(engine.activeRecording == nil) + } +} diff --git a/Tests/AetherEngineTests/LiveRecoveryPlaylistGrowthTests.swift b/Tests/AetherEngineTests/LiveRecoveryPlaylistGrowthTests.swift new file mode 100644 index 000000000..b912569aa --- /dev/null +++ b/Tests/AetherEngineTests/LiveRecoveryPlaylistGrowthTests.swift @@ -0,0 +1,92 @@ +import Foundation +import Testing +@testable import AetherEngine + +/// Audit HLS-101: only the session's first live producer was wired to the provider, so a producer +/// that replaced a dead one (live reopen, the in-place muxerFailed rebuild, the AE#222 live rebuild) +/// cut segments into the cache that the playlist never listed. The playlist ended where the dead +/// producer stopped, and "live reopen succeeded" was logged over a frozen picture. +@Suite("A live playlist keeps growing after an in-engine recovery", .serialized) +struct LiveRecoveryPlaylistGrowthTests { + + private final class Observed: @unchecked Sendable { + private let lock = NSLock() + private weak var _engine: HLSVideoEngine? + private var _countAtFirstReopen: Int? + private var _sourceResets = 0 + + func watch(_ engine: HLSVideoEngine) { lock.withLock { _engine = engine } } + + /// Runs inside the reopen, after the dead pump's own segments were reported. + func noteReopen() { + let engine = lock.withLock { _engine } + let count = engine?.liveSegmentCountSnapshot + lock.withLock { if _countAtFirstReopen == nil { _countAtFirstReopen = count } } + } + + func noteSourceReset() { lock.withLock { _sourceResets += 1 } } + + var countAtFirstReopen: Int? { lock.withLock { _countAtFirstReopen } } + var sourceResets: Int { lock.withLock { _sourceResets } } + } + + private static func openLiveDemuxer() throws -> Demuxer { + let dem = Demuxer() + try dem.open(reader: TinyTransportStreamFixture.LiveReader(), + formatHint: "mpegts", isLive: true) + return dem + } + + @Test("segments a reopened live producer cuts reach the playlist", .timeLimit(.minutes(1))) + func reopenedProducerGrowsThePlaylist() async throws { + let observed = Observed() + let engine = HLSVideoEngine( + url: URL(string: "aether-custom://source")!, + dvModeAvailable: false, + isLiveSession: true, + liveCutTargetSeconds: 1, + preopenedDemuxer: try Self.openLiveDemuxer(), + sourceReopenableByURL: false, + customSourceReopenFactory: { + observed.noteReopen() + return (reader: TinyTransportStreamFixture.LiveReader(), formatHint: "mpegts") + }) + observed.watch(engine) + _ = try engine.start() + defer { engine.stop() } + + // The reader ends after one pass, which is the source loss; the factory is the reconnect. + try await waitFor { observed.countAtFirstReopen != nil } + let beforeLoss = try #require(observed.countAtFirstReopen) + #expect(beforeLoss > 0, "the first producer never reported a segment") + + try await waitFor { (engine.liveSegmentCountSnapshot ?? 0) > beforeLoss } + } + + @Test("a producer rebuilt in place reports into the same playlist", .timeLimit(.minutes(1))) + func rebuiltProducerReportsIntoThePlaylist() async throws { + let observed = Observed() + let engine = HLSVideoEngine( + url: URL(string: "aether-custom://source")!, + dvModeAvailable: false, + isLiveSession: true, + liveCutTargetSeconds: 1, + preopenedDemuxer: try Self.openLiveDemuxer(), + sourceReopenableByURL: false) + engine.onLiveSourceReset = { observed.noteSourceReset() } + _ = try engine.start() + defer { engine.stop() } + + try await waitFor { observed.sourceResets > 0 } + let failed = try #require(engine.producer) + engine.rebuildLiveProducerInPlace(failed: failed) + let rebuilt = try #require(engine.producer) + #expect(rebuilt !== failed) + #expect(rebuilt.liveResidentCapProvider != nil) + + let continuation = try #require(engine.provider?.liveContinuationPoint()) + let report = try #require(rebuilt.onLiveSegmentFinalized, "the rebuilt producer reports to nobody") + report(continuation.nextIndex, 1.0, continuation.outputEndSeconds, true) + #expect(engine.liveSegmentCountSnapshot == continuation.nextIndex + 1) + } +} diff --git a/Tests/AetherEngineTests/LiveReopenAfterTeardownTests.swift b/Tests/AetherEngineTests/LiveReopenAfterTeardownTests.swift new file mode 100644 index 000000000..7d9b6425d --- /dev/null +++ b/Tests/AetherEngineTests/LiveReopenAfterTeardownTests.swift @@ -0,0 +1,59 @@ +import Testing +import Foundation +import AetherLibavcodec +import AetherLibavutil +@testable import AetherEngine + +/// Audit HLS-2: a live reopen that slept through a zap must not open the old channel. +@Suite("Live reopen after teardown") +struct LiveReopenAfterTeardownTests { + + @Test("a reopen demuxer is only registered while the failed producer is still the session's") + func registrationRequiresCurrentProducer() throws { + var codecpar: UnsafeMutablePointer? = try #require(avcodec_parameters_alloc()) + defer { avcodec_parameters_free(&codecpar) } + codecpar!.pointee.codec_id = AV_CODEC_ID_H264 + codecpar!.pointee.codec_type = AVMEDIA_TYPE_VIDEO + let engine = HLSVideoEngine(url: URL(fileURLWithPath: "/nonexistent/live.ts"), + dvModeAvailable: false) + let failed = try HLSSegmentProducer( + demuxer: Demuxer(), videoStreamIndex: 0, + video: .init(codecpar: UnsafePointer(codecpar!), + timeBase: AVRational(num: 1, den: 90_000), codecTagOverride: nil), + cache: SegmentCache(forwardWindow: 4, backwardWindow: 4), + videoFallbackDurationPts: 3_000, desiredFirstVideoTfdtPts: 0, + segmentBoundaries: [0, 540_000]) + + engine.producer = failed + #expect(engine.registerReopenDemuxer(Demuxer(), failedProducer: failed)) + engine.stop() + #expect(!engine.registerReopenDemuxer(Demuxer(), failedProducer: failed), + "after stop() the reopen must not register (and so must not open)") + } + + private final class CountingReader: IOReader, @unchecked Sendable { + private let lock = NSLock() + private var _reads = 0 + var reads: Int { lock.lock(); defer { lock.unlock() }; return _reads } + func read(_ buffer: UnsafeMutablePointer?, size: Int32) -> Int32 { + lock.lock(); _reads += 1; lock.unlock() + return 0 + } + func seek(offset: Int64, whence: Int32) -> Int64 { -1 } + func close() {} + func cancel() {} + func makeIndependentReader() -> IOReader? { nil } + var discImageProbeEnabled: Bool { false } + } + + @Test("a demuxer marked closed before its open never touches the source") + func markClosedBeforeOpenAbortsTheOpen() { + let demuxer = Demuxer() + let reader = CountingReader() + demuxer.markClosed() + #expect(throws: DemuxerError.self) { + try demuxer.open(reader: reader, formatHint: "mpegts", isLive: true) + } + #expect(reader.reads == 0) + } +} diff --git a/Tests/AetherEngineTests/LiveSubtitleRenditionHardeningTests.swift b/Tests/AetherEngineTests/LiveSubtitleRenditionHardeningTests.swift new file mode 100644 index 000000000..d8e0e3ae9 --- /dev/null +++ b/Tests/AetherEngineTests/LiveSubtitleRenditionHardeningTests.swift @@ -0,0 +1,258 @@ +import Foundation +import Testing +@testable import AetherEngine + +/// Audit FEA-104 / FEA-107 / SUB-106: the live subtitle rendition loop fetched, parsed and merged on +/// the main actor, through a session with a 7-day resource timeout, no status check and no cap, and +/// `WebVTTSegmentParser.merged` compared every cue against everything published (40k cues took 3.9 s) +/// while `inf` and `nan` timestamps were accepted and a `+inf` end was never pruned. +struct LiveSubtitleRenditionHardeningTests { + + private func cue(_ id: Int, _ text: String, _ start: Double, _ end: Double) -> SubtitleCue { + SubtitleCue(id: id, startTime: start, endTime: end, body: .text(text)) + } + + /// The fold `merged` was before the per-text index, kept as the oracle. + private func legacyMerged(into existing: [SubtitleCue], adding: [SubtitleCue]) -> [SubtitleCue] { + var result = existing + for cue in adding { + guard case .text(let text) = cue.body else { continue } + let match = result.firstIndex { candidate in + guard case .text(let candidateText) = candidate.body, candidateText == text else { return false } + return cue.startTime <= candidate.endTime + 0.25 + && cue.endTime >= candidate.startTime - 0.25 + } + if let match { + let old = result[match] + result[match] = SubtitleCue(id: old.id, + startTime: min(old.startTime, cue.startTime), + endTime: max(old.endTime, cue.endTime), + body: old.body, + placement: old.placement) + } else { + result.append(cue) + } + } + return result + } + + private struct Snapshot: Equatable { + let id: Int + let start: Double + let end: Double + let text: String? + + init(_ cue: SubtitleCue) { + id = cue.id + start = cue.startTime + end = cue.endTime + text = cue.text + } + } + + private struct SplitMix64: RandomNumberGenerator { + var state: UInt64 + mutating func next() -> UInt64 { + state &+= 0x9E37_79B9_7F4A_7C15 + var z = state + z = (z ^ (z >> 30)) &* 0xBF58_476D_1CE4_E5B9 + z = (z ^ (z >> 27)) &* 0x94D0_49BB_1331_11EB + return z ^ (z >> 31) + } + } + + // MARK: - merged + + @Test("the indexed merge gives what the scan gave, on colliding texts and touching ranges") + func mergeMatchesTheOldScan() { + var rng = SplitMix64(state: 0x5EED_0106) + let texts = ["a", "b", "c", "a b", "", "\u{00E9}"] + for round in 0..<300 { + func make(_ count: Int, firstID: Int) -> [SubtitleCue] { + (0.. String { + "WEBVTT\nX-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:0\n\n\(cues)" + } + + @Test("a cue with a non-finite or negative timestamp is dropped and the others stay") + func nonFiniteTimestampsAreDropped() throws { + let text = segment(""" + 00:00:01.000 --> 00:00:02.000 + good + + 00:00:inf --> 00:00:03.000 + start infinity + + 00:00:01.000 --> 00:00:inf + end infinity + + 00:00:nan --> 00:00:03.000 + start nan + + 00:00:01.000 --> 00:00:nan + end nan + + 00:00:01.000 --> 00:00:1e999 + end overflow + + -00:00:05.000 --> 00:00:03.000 + negative + + 00:00:04.000 --> 00:00:05.000 + also good + """) + let parsed = try #require(WebVTTSegmentParser.parse(text)) + #expect(parsed.cues.map(\.text) == ["good", "also good"]) + #expect(parsed.cues.allSatisfy { $0.start.isFinite && $0.end.isFinite }) + } + + @Test("ordinary timestamps still parse, with a comma as the decimal mark too") + func ordinaryTimestampsStillParse() throws { + let parsed = try #require(WebVTTSegmentParser.parse(segment(""" + 164:04:24.000 --> 164:04:25.440 + long clock + + 01:02,500 --> 01:03.000 + short form + """))) + #expect(parsed.cues.count == 2) + // Typed apart: Swift 6.2 cannot type-check the literal sum inside the macro in time. + let longClock: Double = 164 * 3600 + 4 * 60 + 24 + let longClockError: Double = abs(parsed.cues[0].start - longClock) + let shortFormError: Double = abs(parsed.cues[1].start - 62.5) + #expect(longClockError < 0.001) + #expect(shortFormError < 0.001) + } + + @Test("a segment holds at most 4096 cues") + func cuesPerSegmentAreCapped() throws { + var body = "" + for index in 0..<10_000 { body += "00:00:\(String(format: "%02d", index % 60)).000 --> 00:59:00.000\ncue \(index)\n\n" } + let parsed = try #require(WebVTTSegmentParser.parse(segment(body))) + #expect(parsed.cues.count == WebVTTSegmentParser.maxCuesPerSegment) + #expect(parsed.cues.first?.text == "cue 0") + } + + // MARK: - the fetch, off the main actor + + private func media(_ origin: CannedHTTPOrigin, _ path: String) -> URL { + URL(string: origin.baseURL + path)! + } + + @Test("a rendition playlist is fetched and parsed from a detached task") + func playlistFetchRunsOffTheMainActor() async throws { + let origin = try #require(CannedHTTPOrigin()) + defer { origin.stop() } + origin.route("/subs.m3u8", .body(""" + #EXTM3U + #EXT-X-TARGETDURATION:6 + #EXT-X-PROGRAM-DATE-TIME:2026-09-30T10:00:00.000Z + #EXTINF:6.0, + seg0.vtt + """, contentType: "application/vnd.apple.mpegurl")) + let url = media(origin, "/subs.m3u8") + let playlist = try await Task.detached { + try await AetherEngine.fetchLiveSubtitleRenditionPlaylist(url, headers: [:]) + }.value + #expect(playlist?.segments.map(\.uri) == ["seg0.vtt"]) + #expect(playlist?.segments.first?.programDateTime != nil) + } + + @Test("a master where a media playlist belongs is reported as none, an error status throws") + func playlistFetchRefusals() async throws { + let origin = try #require(CannedHTTPOrigin()) + defer { origin.stop() } + origin.route("/master.m3u8", .body(""" + #EXTM3U + #EXT-X-STREAM-INF:BANDWIDTH=1000 + v.m3u8 + """, contentType: "application/vnd.apple.mpegurl")) + origin.route("/gone.m3u8", .status(404)) + let master = media(origin, "/master.m3u8") + let gone = media(origin, "/gone.m3u8") + let asMaster = try await Task.detached { + try await AetherEngine.fetchLiveSubtitleRenditionPlaylist(master, headers: [:]) + }.value + #expect(asMaster == nil) + await #expect(throws: HLSIngestError.playlistUnreachable(status: 404)) { + _ = try await Task.detached { + try await AetherEngine.fetchLiveSubtitleRenditionPlaylist(gone, headers: [:]) + }.value + } + } + + @Test("a WebVTT segment is parsed off the main actor, an oversized or failing one is dropped") + func segmentFetch() async throws { + let origin = try #require(CannedHTTPOrigin()) + defer { origin.stop() } + origin.route("/ok.vtt", .body(segment("00:00:01.000 --> 00:00:02.000\nhello"), contentType: "text/vtt")) + let filler = String(repeating: "x", count: 1_200_000) + origin.route("/huge.vtt", .body(segment("00:00:01.000 --> 00:00:02.000\n\(filler)"), contentType: "text/vtt")) + origin.route("/gone.vtt", .status(500)) + let ok = media(origin, "/ok.vtt") + let huge = media(origin, "/huge.vtt") + let gone = media(origin, "/gone.vtt") + let parsed = await Task.detached { + await AetherEngine.fetchLiveSubtitleRenditionSegment(ok, headers: [:]) + }.value + #expect(parsed?.cues.map(\.text) == ["hello"]) + let oversized = await Task.detached { + await AetherEngine.fetchLiveSubtitleRenditionSegment(huge, headers: [:]) + }.value + #expect(oversized == nil) + let failing = await Task.detached { + await AetherEngine.fetchLiveSubtitleRenditionSegment(gone, headers: [:]) + }.value + #expect(failing == nil) + } + + @Test("the rendition session has a finite resource timeout") + func sessionTimeoutsAreFinite() { + let configuration = AetherEngine.liveSubtitleRenditionSessionConfiguration() + #expect(configuration.timeoutIntervalForResource <= 60) + #expect(configuration.timeoutIntervalForRequest <= 30) + } + + @Test("the published cue list is capped") + func publishedCuesAreCapped() { + let cues = (0..<5_000).map { cue($0, "c\($0)", Double($0), Double($0) + 1) } + let kept = AetherEngine.capLiveSubtitleCues(cues) + #expect(kept.count == AetherEngine.maxLiveSubtitleCues) + #expect(kept.last?.id == 4_999) + } +} diff --git a/Tests/AetherEngineTests/LiveWindowObservedCadenceTests.swift b/Tests/AetherEngineTests/LiveWindowObservedCadenceTests.swift index 106acfc1e..2d43bdd50 100644 --- a/Tests/AetherEngineTests/LiveWindowObservedCadenceTests.swift +++ b/Tests/AetherEngineTests/LiveWindowObservedCadenceTests.swift @@ -61,8 +61,8 @@ final class LiveWindowObservedCadenceTests: XCTestCase { let provider = makeFastZapProvider() // Unsealed: legacy 18s (3 x fallback TD 6) so non-live/test conformers keep today's bound. XCTAssertEqual(provider.liveBlockingReloadHoldSeconds, 18.0, accuracy: 0.001) - // Seal at fastZap cadence: max segment 1.92s -> TD=2 -> hold 6s. - XCTAssertEqual(provider.liveTargetDurationSeconds(maxSegmentDuration: 1.92), 2) + // Seal at fastZap cadence: max segment 0.96s, cut here -> TD=ceil(1.5 x 0.96)=2 (AE#670) -> hold 6s. + XCTAssertEqual(provider.liveTargetDurationSeconds(maxSegmentDuration: 0.96), 2) XCTAssertEqual(provider.liveBlockingReloadHoldSeconds, 6.0, accuracy: 0.001) // Seal is session-stable: a later, longer segment cannot stretch the hold. _ = provider.liveTargetDurationSeconds(maxSegmentDuration: 5.76) diff --git a/Tests/AetherEngineTests/LiveWindowSizingTotalityTests.swift b/Tests/AetherEngineTests/LiveWindowSizingTotalityTests.swift new file mode 100644 index 000000000..57cf995cf --- /dev/null +++ b/Tests/AetherEngineTests/LiveWindowSizingTotalityTests.swift @@ -0,0 +1,34 @@ +import Testing +@testable import AetherEngine + +/// Audit SEG-107: `LoadOptions.dvrWindowSeconds` reached `Int(ceil(window / cadence))` unvalidated, so a +/// host expressing "keep everything" as `.infinity` trapped on the first live manifest request. +@Suite("Live window sizing is total over the host's DVR window") +struct LiveWindowSizingTotalityTests { + + @Test("An unbounded window serves the playlist ceiling", arguments: [ + Double.infinity, 1e300, Double.greatestFiniteMagnitude, 5e18, + ]) + func unboundedWindow(window: Double) { + let sizing = LiveWindowSizing(targetSegmentDurationSeconds: 4, dvrWindowSeconds: window) + #expect(sizing.windowSegmentCount == LiveWindowSizing.maxWindowSegments) + #expect(sizing.requestedSegmentCount(observedSegmentDurationSeconds: 2) > LiveWindowSizing.maxWindowSegments) + } + + @Test("A NaN window sizes like no window, a negative one like the smallest") + func nanAndNegativeWindows() { + let nan = LiveWindowSizing(targetSegmentDurationSeconds: 4, dvrWindowSeconds: .nan) + let none = LiveWindowSizing(targetSegmentDurationSeconds: 4, dvrWindowSeconds: nil) + #expect(nan.windowSegmentCount == none.windowSegmentCount) + let negative = LiveWindowSizing(targetSegmentDurationSeconds: 4, dvrWindowSeconds: -.infinity) + #expect(negative.windowSegmentCount == LiveWindowSizing.minSafeSegments) + } + + @Test("Ordinary windows are unchanged") + func ordinaryWindows() { + #expect(LiveWindowSizing(targetSegmentDurationSeconds: 4, dvrWindowSeconds: 1800) + .requestedSegmentCount(observedSegmentDurationSeconds: nil) == 450) + #expect(LiveWindowSizing(targetSegmentDurationSeconds: 4, dvrWindowSeconds: 86_400) + .requestedSegmentCount(observedSegmentDurationSeconds: 6) == 14_400) + } +} diff --git a/Tests/AetherEngineTests/LogRedactionTests.swift b/Tests/AetherEngineTests/LogRedactionTests.swift index 2ef55888a..b59eb5134 100644 --- a/Tests/AetherEngineTests/LogRedactionTests.swift +++ b/Tests/AetherEngineTests/LogRedactionTests.swift @@ -6,8 +6,7 @@ import Testing /// the access token in that same query, and `EngineLog` emits with `.public` privacy into OSLog plus /// whatever handler the host installed, so an unredacted line is a live credential in a Console.app /// capture, a sysdiagnose, and every in-app log a host builds on the handler. -/// Serialized: `EngineLog.handler` is process-global, so two of these running at once would -/// each install over the other and read an empty capture. +/// Serialized: the secret-registry tests mutate process-global redaction state. @Suite("EngineLog credential stripping", .serialized) struct LogRedactionTests { @@ -170,14 +169,266 @@ struct LogRedactionTests { #expect(LogRedaction.redact(line) == line) } + @Test("an Xtream Codes path loses its password and keeps the account name", arguments: [ + ("http://h:8080/live/john/S3cretPass/12345.m3u8", "http://h:8080/live/john//12345.m3u8"), + ("http://h:8080/movie/john/S3cretPass/678.mkv", "http://h:8080/movie/john//678.mkv"), + ("http://h:8080/series/john/S3cretPass/901.mkv", "http://h:8080/series/john//901.mkv"), + ("http://h:8080/timeshift/john/S3cretPass/60/2026-09-24:20-00/12345.ts", + "http://h:8080/timeshift/john//60/2026-09-24:20-00/12345.ts"), + ("http://h:8080/hls/a1b2c3d4e5/12345_3.ts", "http://h:8080/hls//12345_3.ts"), + ("http://h:8080/hlsr/a1b2c3d4e5/john/S3cretPass/12345/1/7.ts", "http://h:8080/hlsr//12345/1/7.ts"), + ]) + func xtreamPath(url: String, expected: String) { + let line = LogRedaction.redact("[AetherEngine] load url=\(url) source-format=hls") + #expect(line == "[AetherEngine] load url=\(expected) source-format=hls") + } + + @Test("an ordinary path under the same prefixes is left alone", arguments: [ + "https://origin.example/live/master.m3u8", + "https://origin.example/live/channel1/index.m3u8", + "https://origin.example/movie/trailer.mp4", + "https://origin.example/live/ch1/index.m3u8?x=1", + "https://jellyfin.example/LiveTv/LiveStreamFiles/abc/stream.ts", + ]) + func ordinaryPrefixedPathsSurvive(url: String) { + #expect(LogRedaction.redact("[x] url=\(url) ok") == "[x] url=\(url) ok") + } + + /// `'` `,` and `)` are legal raw in a URL path, so a password holding one read as too few segments + /// for the layout and went out whole. The same mark after the URL is prose and still ends it. + @Test("an Xtream password holding ' , or ) goes whole, and the mark after the URL stays prose", + arguments: ["'", ",", ")"]) + func xtreamPasswordWithPathPunctuation(mark: String) { + #expect(LogRedaction.redact("[x] url=http://h:8080/live/john/pa\(mark)ss/12345.ts ok") + == "[x] url=http://h:8080/live/john//12345.ts ok") + #expect(LogRedaction.redact("[x] url=http://h:8080/hlsr/a1b2\(mark)c3/john/pa\(mark)ss/12345/1/7.ts ok") + == "[x] url=http://h:8080/hlsr//12345/1/7.ts ok") + #expect(LogRedaction.redact("[x] (http://h:8080/live/john/S3cretPass/12345.ts\(mark)) retrying") + == "[x] (http://h:8080/live/john//12345.ts\(mark)) retrying") + let ordinary = "[x] (see https://origin.example/live/channel1/index.m3u8\(mark)), ok" + #expect(LogRedaction.redact(ordinary) == ordinary) + } + + @Test("a URL logged percent-encoded inside another URL's query loses its token (audit NET-1)") + func percentEncodedNestedURL() { + // The exact shape the pre-NET-1 origin relay logged on every request. + let origin = "https://jf.example.com/Videos/abc/master.m3u8?MediaSourceId=x&api_key=\(token)&Tag=7" + let encoded = origin.addingPercentEncoding(withAllowedCharacters: .alphanumerics)! + let lines = [ + "[HLSLocalServer] GET /deadbeef/aether-origin-relay?origin=\(encoded) HTTP/1.1 fd=12", + "[NativeAVPlayerHost] #3 load url=http://127.0.0.1:50123/deadbeef/aether-origin-relay?origin=\(encoded)", + ] + for line in lines { + let out = LogRedaction.redact(line) + #expect(!out.contains(token), "leaked: \(out)") + #expect(out.contains("api%5Fkey%3D%26Tag%3D7"), "\(out)") + #expect(out.contains("MediaSourceId%3Dx"), "diagnostic context went with it: \(out)") + } + let tail = LogRedaction.redact("GET /x?origin=\(encoded) HTTP/1.1 fd=12") + #expect(tail.hasSuffix(" HTTP/1.1 fd=12")) + } + + @Test("a doubly encoded token is stripped too") + func doublyEncodedNestedURL() { + let once = "https://s/a?b=1&X-Emby-Token=\(token)&keep=1" + .addingPercentEncoding(withAllowedCharacters: .alphanumerics)! + let twice = once.addingPercentEncoding(withAllowedCharacters: .alphanumerics)! + let out = LogRedaction.redact("[x] outer?u=\(twice)&z=2") + #expect(!out.contains(token), "leaked: \(out)") + #expect(out.contains("%2526keep%253D1&z=2"), "\(out)") + } + + @Test("an encoded separator inside a plain query value stays part of the value") + func encodedAmpersandInPlainValue() { + // At depth 0 a `%26` is data in the value, not the `&` that ends it, so the whole value goes. + let out = LogRedaction.redact("[x] https://s/a?api_key=abc%26def&keep=1") + #expect(out == "[x] https://s/a?api_key=&keep=1") + } + + @Test("an escape that decodes to a letter is no boundary, and prose with percent signs is left alone") + func encodedBoundaryRules() { + // `%73` is `s`, so this reads `hasToken=` and must stay, like its plain form. + let letter = "[x] ha%73Token=visible" + #expect(LogRedaction.redact(letter) == letter) + let prose = "[x] buffer 100% full, 5%token budget, 12%3 left" + #expect(LogRedaction.redact(prose) == prose) + #expect(LogRedaction.redact("[x] a%2Ftoken%3Asecretvalue done") == "[x] a%2Ftoken%3A done") + } + + @Test("a registered secret goes wherever it sits, raw or percent-encoded, until unregistered") + func registeredSecret() { + #expect(EngineLog.registerSecret("p@ss w0rd")) + let raw = LogRedaction.redact("[x] http://h:8080/john/p%40ss%20w0rd/123 alt=p@ss w0rd.") + #expect(raw == "[x] http://h:8080/john//123 alt=.") + EngineLog.unregisterSecret("p@ss w0rd") + #expect(LogRedaction.redact("[x] alt=p@ss w0rd") == "[x] alt=p@ss w0rd") + } + + @Test("a value too short to match literally is refused") + func shortSecretRefused() { + #expect(!EngineLog.registerSecret("abc")) + #expect(LogRedaction.redact("[x] abc") == "[x] abc") + } + + /// Audit SUB-109: `registerSecret` was a set, so the second owner of a value unregistering it + /// (logout of one of two profiles sharing a password) unmasked it for the first. + @Test("a secret registered twice stays redacted until both registrations are gone") + func registrationsAreCounted() { + let secret = "sh4redPassw0rd" + #expect(EngineLog.registerSecret(secret)) + #expect(EngineLog.registerSecret(secret)) + EngineLog.unregisterSecret(secret) + #expect(LogRedaction.redact("[x] pw=\(secret)x") == "[x] pw=x") + EngineLog.unregisterSecret(secret) + #expect(LogRedaction.redact("[x] v=\(secret)x") == "[x] v=\(secret)x") + } + + /// A registered literal that is also a credential name hid the value behind it, because the scan + /// resumed past the literal; the value's span must not hide a registered literal either. + /// `passwd` rather than `token`: the registry is process-global and parallel suites log `token`. + @Test("a registered secret and a named credential value both go where they meet") + func registeredSecretMeetsACredentialName() { + #expect(EngineLog.registerSecret("passwd")) + #expect(EngineLog.registerSecret("w0rd tail")) + defer { + EngineLog.unregisterSecret("passwd") + EngineLog.unregisterSecret("w0rd tail") + } + #expect(LogRedaction.redact("[x] passwd=abcdef123456&keep=1") == "[x] &keep=1") + #expect(LogRedaction.redact("[x] password=w0rd tail ok") == "[x] password= ok") + // An escape anywhere in the line takes the decoded-view pass instead of the streaming one. + #expect(LogRedaction.redact("[x] %20 passwd=abcdef123456&keep=1") == "[x] %20 &keep=1") + #expect(LogRedaction.redact("[x] passwd alone") == "[x] alone") + } + + // MARK: - Nameless shapes inside a percent-encoded URL (audit SUB-104) + + /// An IPTV proxy or a debrid wrapper carries the upstream URL percent-encoded in its own query. + /// The key forms of that were covered by NET-1; the shapes that need no key only matched raw. + @Test("a nameless credential inside a percent-encoded URL goes", arguments: [ + ("url=http://proxy/x?u=http%3A%2F%2Fiptv.example%2Flive%2Falice%2FSECRETpass%2F1234.ts", + "url=http://proxy/x?u=http%3A%2F%2Fiptv.example%2Flive%2Falice%2F%2F1234.ts"), + ("u=http%3A%2F%2Faddon%2FeyJzdG9yZXMiOlsiYSJdLCJjIjoiU0VDUkVUeHl6IiwidCI6InQifQ%2Fmanifest.json", + "u=http%3A%2F%2Faddon%2F%2Fmanifest.json"), + ("u=smb%3A%2F%2Fbob%3ASECRETpw%40nas%2Fshare", "u=smb%3A%2F%2Fbob%3A%40nas%2Fshare"), + ("http://addon/v1-eyJzdG9yZXMiOlsiYSJdLCJjIjoiU0VDUkVUeHl6IiwidCI6InQifQ/manifest.json", + "http://addon/v1-/manifest.json"), + ]) + func namelessShapesThroughEscapes(input: String, expected: String) { + #expect(LogRedaction.redact(input) == expected) + } + + @Test("the same shapes encoded twice go too", arguments: [ + "http://iptv.example/live/alice/SECRETpass/1234.ts", + "http://addon/eyJzdG9yZXMiOlsiYSJdLCJjIjoiU0VDUkVUeHl6IiwidCI6InQifQ/manifest.json", + "smb://bob:SECRETpw@nas/share", + ]) + func namelessShapesEncodedTwice(upstream: String) { + let once = upstream.addingPercentEncoding(withAllowedCharacters: .alphanumerics)! + let twice = once.addingPercentEncoding(withAllowedCharacters: .alphanumerics)! + let out = LogRedaction.redact("[NativeAVPlayerHost] #1 load url=https://mfp.example/p?d=\(twice) startPos=nil") + for secret in ["SECRETpass", "SECRETpw", "U0VDUkVUeHl6"] { #expect(!out.contains(secret), "\(out)") } + #expect(out.contains("")) + #expect(out.hasSuffix(" startPos=nil")) + } + + /// NET-114: `"\(error)"` of a URLError prints the failing URL twice through its userInfo. + @Test("an interpolated URLError loses the encoded upstream credential of its failing URL") + func urlErrorDescription() { + let upstream = "http://iptv/live/alice/SECRETpass/1.ts" + .addingPercentEncoding(withAllowedCharacters: .alphanumerics)! + let failing = "http://127.0.0.1:1/live/playlist.m3u8?u=\(upstream)" + let error = NSError(domain: NSURLErrorDomain, code: NSURLErrorTimedOut, + userInfo: [NSURLErrorFailingURLStringErrorKey: failing, + NSURLErrorFailingURLErrorKey: URL(string: failing)!]) + let out = LogRedaction.redact("[HLSVODIngest] carriage probe inconclusive: \(error)") + #expect(!out.contains("SECRETpass"), "\(out)") + } + + @Test("an error summary names the URL error code and host, never the failing URL") + func errorSummaryDropsTheURL() throws { + let failing = try #require(URL(string: "http://iptv.example:8080/get.php?username=u&password=Pa:ss")) + let urlError = URLError(.timedOut, userInfo: [NSURLErrorFailingURLErrorKey: failing, + NSURLErrorFailingURLStringErrorKey: failing.absoluteString]) + let summary = EngineLog.summary(of: urlError) + #expect(summary == "NSURLError \(NSURLErrorTimedOut) from iptv.example") + #expect(EngineLog.summary(of: HLSIngestError.playlistUnreachable(status: 403)) + == "playlistUnreachable(403)") + } + + @Test("an escape-heavy line with nothing secret in it comes back unchanged", arguments: [ + "[ffmpeg] Opening 'https://s/Videos/My%20Movie%20(2009)/stream.mkv' for reading", + "[x] url=https://s/Shows/Show%20Name/Season%2001/Show%20Name%20-%20S01E01.mkv ok", + "[x] path=%2Fmedia%2Flive%2Fchannel1%2Findex.m3u8 ok", + "[HLSLocalServer] GET /0123/aether-origin-relay?ref=eW91IGNhbm5vdCByZWFkIHRoaXM_3kJ-qZ HTTP/1.1 fd=9", + ]) + func escapesWithoutSecretsSurvive(line: String) { + #expect(LogRedaction.redact(line) == line) + } + + // MARK: - Query values that hold a terminator (audit SUB-108) + + @Test("a query password holding : ; , ) or > goes whole", arguments: [":", ";", ",", ")", ">"]) + func queryPasswordWithPunctuation(mark: String) { + let out = LogRedaction.redact("https://h/get.php?username=u&password=SECRET\(mark)tail123&type=m3u") + #expect(out == "https://h/get.php?username=u&password=&type=m3u") + } + + @Test("punctuation that prose puts after a value still ends it") + func proseAfterAValue() { + #expect(LogRedaction.redact("[x] fetch failed (api_key=abc), retrying") + == "[x] fetch failed (api_key=), retrying") + #expect(LogRedaction.redact("[x] seen ") == "[x] seen >") + } + + // MARK: - Credential names outside Jellyfin and Xtream (audit SUB-109) + + @Test("the other backends' credential names are covered", arguments: [ + ("Authorization: Bearer SECRETopaque12345", "Authorization: Bearer "), + ("Authorization: Basic dmluY2VudDpzZWNyZXQxMjM0", "Authorization: Basic "), + ("Cookie: PHPSESSID=SECRETsess123; other=1", "Cookie: ; other=1"), + ("Set-Cookie: session=SECRETsess123; Path=/", "Set-Cookie: ; Path=/"), + ("X-Api-Key: SECRETkey123", "X-Api-Key: "), + ("https://h/a?api-key=SECRETkey123&x=1", "https://h/a?api-key=&x=1"), + (#"{"api_key":"SECRETjson123"}"#, #"{"api_key":""}"#), + (#"["X-Emby-Token": "SECRETjson123"]"#, #"["X-Emby-Token": ""]"#), + ("https://h/p.php?user=u&pwd=SECRETkey123", "https://h/p.php?user=u&pwd="), + ("https://h/p.php?user=u&passwd=SECRETkey123", "https://h/p.php?user=u&passwd="), + ("authToken=SECRETrt123 done", "authToken= done"), + ("refreshToken=SECRETrt123 done", "refreshToken= done"), + ("sessionToken=SECRETrt123 done", "sessionToken= done"), + ("auth_token=SECRETrt123 done", "auth_token= done"), + ("https://h/a?sessionid=SECRETsid123&session_id=SECRETsid456", "https://h/a?sessionid=&session_id="), + ]) + func otherBackendNames(input: String, expected: String) { + #expect(LogRedaction.redact(input) == expected) + } + + @Test("prose around the new names is left alone", arguments: [ + "[http] 401 with WWW-Authenticate: Bearer realm=\"fixture\"", + "[auth] a bearer token was sent", + "[auth] basic auth failed", + "[x] X-Playback-Session-Id: 5A0C2D7E-1234", + "[x] no cookie was set", + ]) + func proseAroundNewNames(line: String) { + #expect(LogRedaction.redact(line) == line) + } + + /// Audit OPS-106: aetherctl printed the source URL in its own banner, outside the funnel, so the + /// first line of a pasted transcript undid the redaction of every engine line below it. + @Test("a line a tool prints itself gets the funnel's redaction") + func redactedForATool() { + #expect(EngineLog.redacted("aetherctl probe: http://h:8080/live/john/S3cretPass/1.ts?api_key=\(token)") + == "aetherctl probe: http://h:8080/live/john//1.ts?api_key=") + } + /// The point of putting this in EngineLog rather than in each host: the handler a host installs /// must never see the raw token, whether or not that host scrubs its own log. @Test("the host handler receives the redacted line") func handlerSeesRedactedLine() { - let box = LineBox() - let previous = EngineLog.handler - EngineLog.handler = { box.append($0) } - defer { EngineLog.handler = previous } + let box = EngineLogCapture() + defer { box.end() } EngineLog.emit("[test-496a] load url=https://s/v?api_key=\(token)&Static=true", category: .engine) @@ -194,10 +445,8 @@ struct LogRedactionTests { /// shared funnel and not on the `.info` branch alone. @Test("a verbose line is withheld from the handler") func verboseSkipsTheHandler() { - let box = LineBox() - let previous = EngineLog.handler - EngineLog.handler = { box.append($0) } - defer { EngineLog.handler = previous } + let box = EngineLogCapture() + defer { box.end() } EngineLog.emit("[test-496b] per-segment trace api_key=\(token)", category: .session, level: .verbose) @@ -205,20 +454,4 @@ struct LogRedactionTests { // an unrelated AVIOReader line from a parallel suite, which says nothing about `.verbose`. #expect(box.lines.filter { $0.contains("[test-496b]") }.isEmpty) } - - /// The handler is called on whatever thread emitted, so the capture needs its own lock. - private final class LineBox: @unchecked Sendable { - private let lock = NSLock() - private var storage: [String] = [] - - func append(_ line: String) { - lock.lock(); defer { lock.unlock() } - storage.append(line) - } - - var lines: [String] { - lock.lock(); defer { lock.unlock() } - return storage - } - } } diff --git a/Tests/AetherEngineTests/LostSourceReadErrorTests.swift b/Tests/AetherEngineTests/LostSourceReadErrorTests.swift new file mode 100644 index 000000000..20c380bb3 --- /dev/null +++ b/Tests/AetherEngineTests/LostSourceReadErrorTests.swift @@ -0,0 +1,140 @@ +import Testing +import Foundation +@testable import AetherEngine + +/// A source that is lost mid-read reports a read error, never end-of-file: the consumer treats EOF +/// as "played to the end" and never retries it (audit DMX-6, DMX-10). +@Suite("Lost source reads report EIO") +struct LostSourceReadErrorTests { + + /// Answers every request with a length-less 200 and every GET with `bodyBytes` of a body that + /// never completes. The reader can resolve no size and runs the forward-only streaming path. + private final class LengthlessResettingOrigin: @unchecked Sendable { + let port: UInt16 + private let listenFD: Int32 + private let bodyBytes: Int + private let lock = NSLock() + private var stopped = false + private var isStopped: Bool { + lock.lock() + defer { lock.unlock() } + return stopped + } + + init?(bodyBytes: Int) { + self.bodyBytes = bodyBytes + let fd = socket(AF_INET, SOCK_STREAM, 0) + guard fd >= 0 else { return nil } + var one: Int32 = 1 + setsockopt(fd, SOL_SOCKET, SO_REUSEADDR, &one, socklen_t(MemoryLayout.size)) + var addr = sockaddr_in() + addr.sin_family = sa_family_t(AF_INET) + addr.sin_port = 0 + addr.sin_addr.s_addr = inet_addr("127.0.0.1") + let bound = withUnsafePointer(to: &addr) { + $0.withMemoryRebound(to: sockaddr.self, capacity: 1) { + bind(fd, $0, socklen_t(MemoryLayout.size)) + } + } + guard bound == 0, listen(fd, 16) == 0 else { Darwin.close(fd); return nil } + var name = sockaddr_in() + var len = socklen_t(MemoryLayout.size) + _ = withUnsafeMutablePointer(to: &name) { + $0.withMemoryRebound(to: sockaddr.self, capacity: 1) { getsockname(fd, $0, &len) } + } + listenFD = fd + port = UInt16(bigEndian: name.sin_port) + Thread.detachNewThread { [self] in acceptLoop() } + } + + func stop() { + lock.lock() + stopped = true + lock.unlock() + shutdown(listenFD, SHUT_RDWR) + Darwin.close(listenFD) + } + + private func acceptLoop() { + while true { + let fd = accept(listenFD, nil, nil) + guard fd >= 0 else { return } + var one: Int32 = 1 + setsockopt(fd, SOL_SOCKET, SO_NOSIGPIPE, &one, socklen_t(MemoryLayout.size)) + Thread.detachNewThread { [self] in serve(fd) } + } + } + + private func serve(_ fd: Int32) { + var buf = [UInt8](repeating: 0, count: 16 * 1024) + var request = Data() + while request.range(of: Data("\r\n\r\n".utf8)) == nil { + let n = recv(fd, &buf, buf.count, 0) + guard n > 0 else { Darwin.close(fd); return } + request.append(contentsOf: buf[0.. 0 else { break } + sent += n + } + } + // Half-close, as `ThrottledOriginServer.serveThenDrop` does: a full close can turn into + // an RST that discards bytes the reader has not been handed yet. + shutdown(fd, SHUT_WR) + while !isStopped { usleep(50_000) } + Darwin.close(fd) + } + } + + private func drain(_ reader: AVIOReader) -> Int32 { + let buf = UnsafeMutablePointer.allocate(capacity: 64 * 1024) + defer { buf.deallocate() } + while true { + let n = reader.read(into: buf, size: 64 * 1024) + if n <= 0 { return n } + } + } + + @Test("a length-less stream cut off mid-body reports EIO, not EOF", .timeLimit(.minutes(1))) + func cutOffLengthlessStreamIsAnError() throws { + let origin = try #require(LengthlessResettingOrigin(bodyBytes: 256 * 1024)) + defer { origin.stop() } + let reader = AVIOReader(url: URL(string: "http://127.0.0.1:\(origin.port)/live.ts")!, + prefetchEnabled: false) + defer { reader.markClosed(); reader.close() } + try reader.open() + #expect(drain(reader) == FFmpegErr.eio) + } + + @Test("a seekable read whose fetch fails reports EIO, not EOF", .timeLimit(.minutes(1))) + func failedSeekableFetchIsAnError() throws { + let target: Int64 = 8 * 1024 * 1024 + let respond: @Sendable (Int, Int64, String) -> ThrottledOriginServer.Directive = { _, offset, _ in + offset == target ? .status(500) : .serve206 + } + let origin = ThrottledOriginServer(totalSize: 32 * 1024 * 1024, throttleUs: 0, respond: respond) + let server = try #require(origin) + defer { server.stop() } + let reader = AVIOReader(url: URL(string: "http://127.0.0.1:\(server.port)/movie.bin")!, + chunkSize: 256 * 1024, prefetchEnabled: false) + defer { reader.markClosed(); reader.close() } + try reader.open() + + #expect(reader.seek(offset: target, whence: SEEK_SET) == target) + let buf = UnsafeMutablePointer.allocate(capacity: 4096) + defer { buf.deallocate() } + #expect(reader.read(into: buf, size: 4096) == FFmpegErr.eio) + } +} diff --git a/Tests/AetherEngineTests/MasterFallbackDecisionTests.swift b/Tests/AetherEngineTests/MasterFallbackDecisionTests.swift index b087cb2c0..069769e4c 100644 --- a/Tests/AetherEngineTests/MasterFallbackDecisionTests.swift +++ b/Tests/AetherEngineTests/MasterFallbackDecisionTests.swift @@ -114,82 +114,3 @@ struct MasterFallbackPositionTests { #expect(body.contains("host.mountedStartPosition")) } } - -/// #93/#98: when a dead item's recovery reload is refused, the media fallback replaces it. Whether -/// the fallback then PLAYS is the viewer's call. The engine's intent (#122) misses a Play from AVKit, -/// Control Center or PiP, so AVPlayer's rate is read as well. -@Suite("#98: the media fallback resumes only a viewer who was playing") -@MainActor -struct MasterFallbackTransportTests { - private func resumes( - command: Bool? = nil, intent: Bool, rolled: Bool, pausedBefore: Bool - ) -> Bool { - NativeAVPlayerHost.mediaFallbackResumesPlaying( - commandSinceRejection: command, intentIsPlaying: intent, - rolledSinceEngineStop: rolled, pausedBeforeRejection: pausedBefore) - } - - @Test("An item the engine was playing is replaced playing") - func enginePlayResumes() { - #expect(resumes(intent: true, rolled: false, pausedBefore: false)) - } - - @Test("A Play from AVKit, Control Center or PiP, which leaves the intent clear, is kept") - func externalPlayResumes() { - #expect(resumes(intent: false, rolled: true, pausedBefore: false)) - } - - @Test("A viewer who paused before the refusal stays paused, whichever way the pause came") - func pauseBeforeRejectionStaysPaused() { - #expect(!resumes(intent: true, rolled: true, pausedBefore: true)) - #expect(!resumes(intent: false, rolled: true, pausedBefore: true)) - } - - /// An engine pause inside the one-second margin reads as the refusal's own stop, but it clears the - /// intent and the roll, so the viewer's pause still holds. - @Test("An engine pause just before the refusal stays paused") - func enginePauseInsideTheMarginStaysPaused() { - #expect(!resumes(intent: false, rolled: false, pausedBefore: false)) - } - - @Test("A Play or Pause through the engine after the refusal decides") - func commandAfterRejectionDecides() { - #expect(resumes(command: true, intent: true, rolled: false, pausedBefore: true)) - #expect(!resumes(command: false, intent: false, rolled: true, pausedBefore: false)) - } - - /// Engine commands that drive the host's records, on a real host with a mounted item. - @Test("The host's records follow engine and external transport") - func hostRecordsFollowTransport() { - let host = NativeAVPlayerHost() - defer { host.tearDown() } - host.load(url: URL(fileURLWithPath: "/nonexistent-master-fallback-transport-test.m3u8"), - startPosition: 0, contract: .init()) - // A fresh load nobody played: stays paused. - #expect(!host.mediaFallbackResumesPlaying()) - host.play() - #expect(host.mediaFallbackResumesPlaying()) - host.pause() - #expect(!host.mediaFallbackResumesPlaying()) - } - - /// The fallback needs a live loopback session to run, so this reads its call site, as the - /// placement test above does. - @Test("The media fallback plays only on the host's verdict, read before the swap") - func fallbackAsksTheHost() throws { - let source = URL(fileURLWithPath: #filePath) - .deletingLastPathComponent() - .deletingLastPathComponent() - .deletingLastPathComponent() - .appendingPathComponent("Sources/AetherEngine/AetherEngine.swift") - let text = try #require(try? String(contentsOf: source, encoding: .utf8)) - let fn = try #require(text.range(of: "func fallBackToMediaPlaylist(")) - let end = try #require(text[fn.upperBound...].range(of: "\n }\n")) - let body = String(text[fn.lowerBound.. Bool { + audioRendition: Bool = false, + iFrameRendition: Bool = false) -> Bool { HLSVideoEngine.resolveUseMasterPlaylist( videoRange: videoRange, effectiveDvMode: effectiveDvMode, panelIsInHDRMode: panelHDR, displaySupportsHDR: displayHDR, hasNativeSubs: nativeSubs, builtInPanelEngagesOnDemand: panelEngagesOnDemand, frameRateKnown: frameRateKnown, videoCodecNeedsMasterSignaling: hevcNeedsMasterSignaling, - hasAudioRendition: audioRendition) + hasAudioRendition: audioRendition, + hasIFrameRendition: iFrameRendition) } @Test("tvOS: HDR source on an SDR-parked panel stays media-direct (-11848 guard)") @@ -162,4 +164,20 @@ struct MasterRoutingTests { #expect(!HLSVideoEngine.builtInPanelEngagesOnDemand) #endif } + + @Test("AE#682: an I-frame rendition forces the master for an SDR source, like a subtitle rendition") + func iFrameRenditionForcesMasterOnSDR() { + #expect(!route(videoRange: .sdr)) + #expect(route(videoRange: .sdr, iFrameRendition: true)) + } + + @Test("AE#682: an I-frame rendition never forces an HDR master onto an unready panel") + func iFrameRenditionRespectsRoutingSafety() { + #expect(!route(videoRange: .pq, displayHDR: true, iFrameRendition: true)) + } + + @Test("AE#682: an I-frame rendition cannot rescue a PQ master without a frame rate (#130)") + func iFrameRenditionRespectsFrameRateGate() { + #expect(!route(videoRange: .pq, panelHDR: true, frameRateKnown: false, iFrameRendition: true)) + } } diff --git a/Tests/AetherEngineTests/MovTextSampleBuilderTests.swift b/Tests/AetherEngineTests/MovTextSampleBuilderTests.swift index f354ac26f..6312282c1 100644 --- a/Tests/AetherEngineTests/MovTextSampleBuilderTests.swift +++ b/Tests/AetherEngineTests/MovTextSampleBuilderTests.swift @@ -17,4 +17,19 @@ final class MovTextSampleBuilderTests: XCTestCase { func test_sanitize_trimsSurroundingWhitespace() { XCTAssertEqual(MovTextSampleBuilder.sanitize(" hello "), "hello") } + + func test_sanitize_keepsAnUnclosedBraceAndWhatFollows() { + XCTAssertEqual(MovTextSampleBuilder.sanitize("{\\b1}bold{unclosed tail"), "bold{unclosed tail") + XCTAssertEqual(MovTextSampleBuilder.sanitize("{a{b}c}"), "c}") + } + + /// Audit SUB-1: stripping this cue block by block took 525 s on an M-series Mac; one pass, 0.15 s. + func test_sanitize_isLinearOnAHostileCueOfManySmallBlocks() { + let hostile = String(repeating: "{\\1c&H00FF00&}x", count: 200_000) + let started = Date() + let clean = MovTextSampleBuilder.sanitize(hostile) + XCTAssertEqual(clean.count, 200_000) + XCTAssertTrue(clean.allSatisfy { $0 == "x" }) + XCTAssertLessThan(Date().timeIntervalSince(started), 2.0) + } } diff --git a/Tests/AetherEngineTests/MuxerBufferBoundTests.swift b/Tests/AetherEngineTests/MuxerBufferBoundTests.swift index ccfd756b8..15d1fbb8a 100644 --- a/Tests/AetherEngineTests/MuxerBufferBoundTests.swift +++ b/Tests/AetherEngineTests/MuxerBufferBoundTests.swift @@ -60,4 +60,12 @@ struct MuxerBufferBoundTests { #expect(MP4SegmentMuxer.bufferedTicksExceedsBound( firstDts: 0, currentDts: 10_000_000, boundTicks: 0) == false) } + + @Test("A span wider than Int64 counts as past the bound instead of trapping (audit SEG-101)") + func spanOverflow() { + #expect(MP4SegmentMuxer.bufferedTicksExceedsBound( + firstDts: Int64.min + 1, currentDts: Int64.max, boundTicks: 128_000)) + #expect(MP4SegmentMuxer.bufferedTicksExceedsBound( + firstDts: -(Int64(1) << 62), currentDts: Int64(1) << 62, boundTicks: Int64.max)) + } } diff --git a/Tests/AetherEngineTests/NALUnitChainTests.swift b/Tests/AetherEngineTests/NALUnitChainTests.swift index a38337884..8c15c7e2d 100644 --- a/Tests/AetherEngineTests/NALUnitChainTests.swift +++ b/Tests/AetherEngineTests/NALUnitChainTests.swift @@ -68,6 +68,31 @@ struct NALUnitChainTests { #expect(Self.run([0x00, 0x00, 0x01, 0x26, 0x01, 0xAA, 0xBB, 0xCC]) == nil) } + /// Audit BIT-1: a first NAL of 256 to 511 bytes has the 4-byte length `00 00 01 xx`, which the + /// head test alone reads as a start code, so the overlong length behind it reached the parser. + @Test("A 256-511 byte first NAL no longer hides an overrun once the track walked exactly") + func threeByteHeadIsALengthOnceConfirmed() { + let first = Self.nal([0x4E, 0x01] + [UInt8](repeating: 0x5A, count: 318)) + #expect(Array(first.prefix(3)) == [0x00, 0x00, 0x01]) + let broken: [UInt8] = [0x16, 0xE5, 0x7A, 0xB3] + [UInt8](repeating: 0x5A, count: 96) + let sample = first + broken + + #expect(Self.run(sample) == nil, "unconfirmed, the head still reads as Annex B") + let confirmed = sample.withUnsafeBytes { + NALUnitChain.completeRunLength($0, lengthPrefixSize: 4, framingConfirmed: true) + } + #expect(confirmed == first.count) + + // The healthy sample of the same shape is what confirms the framing in the first place. + #expect(first.withUnsafeBytes { NALUnitChain.walksExactly($0, lengthPrefixSize: 4) }) + // Annex B never confirms it, and a 4-byte start code stays refused even once confirmed. + let annexB: [UInt8] = [0x00, 0x00, 0x00, 0x01, 0x26, 0x01, 0xAA, 0xBB, 0x00, 0x00, 0x01, 0x02, 0x01] + #expect(!annexB.withUnsafeBytes { NALUnitChain.walksExactly($0, lengthPrefixSize: 4) }) + #expect(annexB.withUnsafeBytes { + NALUnitChain.completeRunLength($0, lengthPrefixSize: 4, framingConfirmed: true) + } == nil) + } + @Test("The prefix width is honoured, not assumed to be four") func widthIsHonoured() { let two = Self.nal([0x26, 0x01, 0xAA], lengthPrefixSize: 2) @@ -115,4 +140,66 @@ struct NALUnitChainTests { } private enum Codec { case hevc, h264, av1 } + + // MARK: - The verdict outlives the muxer (audit BIT-104) + + /// Audit BIT-104: every seek, restart and reload builds a fresh muxer, and the sample a restart + /// lands on is that muxer's first. With the BIT-1 verdict held per muxer, exactly that sample was + /// judged by the head test alone, so a damaged IRAP with a 256-511 byte first NAL reached movenc + /// uncut on every retry of the restart. + @Test("A muxer rebuilt mid-session inherits the track's confirmed framing") + func rebuiltMuxerInheritsConfirmedFraming() throws { + let data = try #require(Data(base64Encoded: AtmosDetectionProbeIntegrationTests.videoOnlyBase64, + options: .ignoreUnknownCharacters)) + let demuxer = Demuxer() + try demuxer.open(reader: DataIOReader(data: data), formatHint: "mp4") + defer { demuxer.close() } + let stream = try #require(demuxer.stream(at: demuxer.videoStreamIndex)) + let sessionDir = FileManager.default.temporaryDirectory + .appendingPathComponent("bit104-\(UUID().uuidString)") + try FileManager.default.createDirectory(at: sessionDir, withIntermediateDirectories: true) + defer { try? FileManager.default.removeItem(at: sessionDir) } + + func makeMuxer(_ index: Int, latch: NALFramingLatch?) throws -> MP4SegmentMuxer { + try MP4SegmentMuxer( + initialSegmentIndex: index, sessionDir: sessionDir, + video: .init(codecpar: UnsafePointer(stream.pointee.codecpar), + timeBase: stream.pointee.time_base, codecTagOverride: nil, + nalFramingLatch: latch), + audio: nil, onInitCaptured: { _ in }) + } + func write(_ bytes: [UInt8], to muxer: MP4SegmentMuxer) throws { + var packet: UnsafeMutablePointer? = try #require(trackedPacketAlloc()) + defer { trackedPacketFree(&packet) } + let pkt = packet! + #expect(av_new_packet(pkt, Int32(bytes.count)) >= 0) + bytes.withUnsafeBytes { _ = memcpy(pkt.pointee.data, $0.baseAddress, bytes.count) } + pkt.pointee.pts = 0 + pkt.pointee.dts = 0 + pkt.pointee.duration = 512 + pkt.pointee.flags |= AV_PKT_FLAG_KEY + pkt.pointee.stream_index = muxer.videoOutputStreamIndex + _ = muxer.writePacket(pkt) + } + + // What the restart lands on: a first NAL whose 4-byte length reads `00 00 01 40`, then a + // length far past the end of the sample. + let first = Self.nal([0x65, 0x88] + [UInt8](repeating: 0x5A, count: 318)) + #expect(Array(first.prefix(4)) == [0x00, 0x00, 0x01, 0x40]) + let damaged = first + [0x16, 0xE5, 0x7A, 0xB3] + [UInt8](repeating: 0x5A, count: 96) + + let session = NALFramingLatch() + let earlier = try makeMuxer(0, latch: session) + try write(Self.nal([0x65, 0x88, 0x84, 0x00]), to: earlier) // walks exactly + #expect(session.isConfirmed) + + let rebuilt = try makeMuxer(7, latch: session) + try write(damaged, to: rebuilt) + #expect(rebuilt.truncatedVideoSamples == 1, "the rebuilt muxer passed the overrun through uncut") + + // A track nobody has confirmed keeps the Annex B head guard. + let unconfirmed = try makeMuxer(7, latch: nil) + try write(damaged, to: unconfirmed) + #expect(unconfirmed.truncatedVideoSamples == 0) + } } diff --git a/Tests/AetherEngineTests/OriginRequestBudgetAbortTests.swift b/Tests/AetherEngineTests/OriginRequestBudgetAbortTests.swift new file mode 100644 index 000000000..440b7b184 --- /dev/null +++ b/Tests/AetherEngineTests/OriginRequestBudgetAbortTests.swift @@ -0,0 +1,125 @@ +import Testing +import Foundation +@testable import AetherEngine + +/// Audit DMX-112: a pump parked in `OriginRequestBudget.acquire` ignored `markClosed()`, so a torn +/// down session stayed in the origin's FIFO and in the race for the next pacer token, and could take +/// the very slot the new session was waiting for. `shouldAbort` is polled once per slice in both +/// waits, and an aborted caller consumes nothing. +@Suite("Origin request budget abort", .serialized) +struct OriginRequestBudgetAbortTests { + + private let url = URL(string: "https://abort.example.com:443/movie.mkv")! + + private final class Flag: @unchecked Sendable { + private let lock = NSLock() + private var _value = false + var value: Bool { lock.withLock { _value } } + func set() { lock.withLock { _value = true } } + } + + private final class Result: @unchecked Sendable { + private let lock = NSLock() + private var _done = false + private var _ticket: OriginRequestBudget.Ticket? + var done: Bool { lock.withLock { _done } } + var ticket: OriginRequestBudget.Ticket? { lock.withLock { _ticket } } + func finish(_ ticket: OriginRequestBudget.Ticket?) { lock.withLock { _ticket = ticket; _done = true } } + } + + private final class ManualClock: @unchecked Sendable { + private let lock = NSLock() + private var nanoseconds: UInt64 = 1_000_000_000 + func now() -> DispatchTime { lock.withLock { DispatchTime(uptimeNanoseconds: nanoseconds) } } + func advance(by seconds: TimeInterval) { + lock.withLock { nanoseconds += UInt64((seconds * 1_000_000_000).rounded()) } + } + } + + @Test("a caller parked for a slot leaves on abort without taking or holding one", + .timeLimit(.minutes(2))) + func slotWaitAborts() async throws { + let budget = OriginRequestBudget() + budget.setHostLimit(1, for: url) + let held = budget.acquire(for: url, label: "holder", timeout: 1) + let abort = Flag() + let outcome = Result() + let target = url + Thread.detachNewThread { + // 120 s: the abort, not this budget, is what must end the wait. + outcome.finish(budget.acquire(for: target, label: "pump", timeout: 120, + shouldAbort: { abort.value })) + } + try await waitFor { budget.snapshot(for: url)?.waiting == 1 } + + abort.set() + try await waitFor { outcome.done } + #expect(outcome.ticket == nil, "an aborted acquire must not hand out a ticket") + #expect(budget.snapshot(for: url)?.waiting == 0, "the aborted caller left the FIFO") + #expect(budget.snapshot(for: url)?.inflight == 1, "only the holder is on the books") + + // The slot the holder gives back goes to the next caller, not to a waiter that left. + budget.release(held) + let next = budget.acquire(for: url, label: "next", timeout: 1) + #expect(next?.granted == true) + #expect(next?.waitedMs == 0) + budget.release(next) + #expect(budget.snapshot(for: url)?.inflight == 0) + } + + @Test("a caller parked for the pacer leaves on abort and leaves the tokens alone", + .timeLimit(.minutes(2))) + func pacerWaitAborts() async throws { + let clock = ManualClock() + let budget = OriginRequestBudget(now: clock.now) + budget.noteRefusal(for: url, status: 429) + let abort = Flag() + let outcome = Result() + let target = url + Thread.detachNewThread { + outcome.finish(budget.acquire(for: target, label: "pump", timeout: 120, + shouldAbort: { abort.value })) + } + try await waitFor { budget.snapshot(for: url)?.paced == true && budget.snapshot(for: url)?.inflight == 0 } + + abort.set() + try await waitFor { outcome.done } + #expect(outcome.ticket == nil) + #expect(budget.snapshot(for: url)?.inflight == 0, "an aborted caller is not on the link") + + // The bucket holds its full burst of two once the quiet period has passed: the aborted + // caller consumed none of it. + clock.advance(by: 4) + for _ in 0..<2 { + let ticket = budget.tryAcquire(for: url, label: "next") + #expect(ticket?.granted == true, "a token went to the caller that had left") + budget.release(ticket) + } + } + + @Test("an abort that is already set returns before anything is taken") + func abortBeforeTheWait() { + let budget = OriginRequestBudget() + #expect(budget.acquire(for: url, label: "pump", timeout: 1, shouldAbort: { true }) == nil) + #expect(budget.snapshot(for: url) == nil, "nothing was counted for an aborted caller") + } + + @Test("a caller that is never aborted waits and is granted as before") + func noAbortKeepsTheOldBehaviour() async throws { + let budget = OriginRequestBudget() + budget.setHostLimit(1, for: url) + let held = budget.acquire(for: url, label: "holder", timeout: 1) + let outcome = Result() + let target = url + Thread.detachNewThread { + outcome.finish(budget.acquire(for: target, label: "pump", timeout: 120, + shouldAbort: { false })) + } + try await waitFor { budget.snapshot(for: url)?.waiting == 1 } + budget.release(held) + try await waitFor { outcome.done } + #expect(outcome.ticket?.granted == true) + budget.release(outcome.ticket) + #expect(budget.snapshot(for: url)?.inflight == 0) + } +} diff --git a/Tests/AetherEngineTests/OutputTimestampSanitizerTests.swift b/Tests/AetherEngineTests/OutputTimestampSanitizerTests.swift index 539647045..fd2fec9df 100644 --- a/Tests/AetherEngineTests/OutputTimestampSanitizerTests.swift +++ b/Tests/AetherEngineTests/OutputTimestampSanitizerTests.swift @@ -5,56 +5,56 @@ final class OutputTimestampSanitizerTests: XCTestCase { // MARK: - Healthy content is untouched - func testMonotonicAudioPassThroughUnchanged() { + func testMonotonicAudioPassThroughUnchanged() throws { var s = OutputTimestampSanitizer() for i in 0..<5 { let t = Int64(1000 + i * 1024) - let out = s.sanitize(streamIndex: 1, pts: t, dts: t) + let out = try XCTUnwrap(s.sanitize(streamIndex: 1, pts: t, dts: t)) XCTAssertEqual(out.dts, t) XCTAssertEqual(out.pts, t) } } - func testVideoBFramePtsAboveDtsPreserved() { + func testVideoBFramePtsAboveDtsPreserved() throws { var s = OutputTimestampSanitizer() - let out = s.sanitize(streamIndex: 0, pts: 2000, dts: 1000) + let out = try XCTUnwrap(s.sanitize(streamIndex: 0, pts: 2000, dts: 1000)) XCTAssertEqual(out.dts, 1000) XCTAssertEqual(out.pts, 2000) } // MARK: - The SSAI ad-boundary failure (exact device-log values) - func testAudioPtsBelowDtsIsLiftedToDts() { + func testAudioPtsBelowDtsIsLiftedToDts() throws { // Device log (Beverly Hills 90210 Pluto ad boundary): // pts (6391809) < dts (6679426) in stream 1 // Audio carries no reordering, so the fix is pts := dts. var s = OutputTimestampSanitizer() _ = s.sanitize(streamIndex: 1, pts: 6679425, dts: 6679425) // prior packet - let out = s.sanitize(streamIndex: 1, pts: 6391809, dts: 6679426) + let out = try XCTUnwrap(s.sanitize(streamIndex: 1, pts: 6391809, dts: 6679426)) XCTAssertEqual(out.dts, 6679426) XCTAssertEqual(out.pts, 6679426, "audio pts must be lifted to dts (no reordering)") XCTAssertGreaterThanOrEqual(out.pts, out.dts) } - func testEqualDtsCollisionIsBumped() { + func testEqualDtsCollisionIsBumped() throws { // Device log: "non monotonically increasing dts ... 6679425 >= 6679425" var s = OutputTimestampSanitizer() _ = s.sanitize(streamIndex: 1, pts: 6679425, dts: 6679425) - let out = s.sanitize(streamIndex: 1, pts: 6679425, dts: 6679425) + let out = try XCTUnwrap(s.sanitize(streamIndex: 1, pts: 6679425, dts: 6679425)) XCTAssertEqual(out.dts, 6679426, "colliding dts must be bumped past the last written") XCTAssertEqual(out.pts, 6679426) } - func testBackwardDtsResetIsForcedMonotonic() { + func testBackwardDtsResetIsForcedMonotonic() throws { // SSAI creative restarts source clock at 2^33; rescaled dts can land far below the last written value. var s = OutputTimestampSanitizer() _ = s.sanitize(streamIndex: 1, pts: 10_000, dts: 10_000) - let out = s.sanitize(streamIndex: 1, pts: 2_000, dts: 2_000) + let out = try XCTUnwrap(s.sanitize(streamIndex: 1, pts: 2_000, dts: 2_000)) XCTAssertEqual(out.dts, 10_001) XCTAssertEqual(out.pts, 10_001) } - func testWholeAdBurstStaysMonotonicAndPtsGEDts() { + func testWholeAdBurstStaysMonotonicAndPtsGEDts() throws { // Replays the Beverly Hills 90210 / Pluto stall shape: dts collides/regresses, pts trails dts; both pathologies at once. var s = OutputTimestampSanitizer() _ = s.sanitize(streamIndex: 1, pts: 6679425, dts: 6679425) @@ -62,7 +62,7 @@ final class OutputTimestampSanitizerTests: XCTestCase { for i in 0..<200 { let pts = Int64(6391809 + i * 1024) let dts = Int64(6679425) // constant, colliding - let out = s.sanitize(streamIndex: 1, pts: pts, dts: dts) + let out = try XCTUnwrap(s.sanitize(streamIndex: 1, pts: pts, dts: dts)) XCTAssertGreaterThan(out.dts, lastDts, "dts must strictly increase") XCTAssertGreaterThanOrEqual(out.pts, out.dts, "pts must be >= dts") lastDts = out.dts @@ -71,17 +71,27 @@ final class OutputTimestampSanitizerTests: XCTestCase { // MARK: - Per-stream independence - func testStreamsTrackedIndependently() { + func testStreamsTrackedIndependently() throws { var s = OutputTimestampSanitizer() - let a = s.sanitize(streamIndex: 1, pts: 5000, dts: 5000) - let v = s.sanitize(streamIndex: 0, pts: 100, dts: 100) + let a = try XCTUnwrap(s.sanitize(streamIndex: 1, pts: 5000, dts: 5000)) + let v = try XCTUnwrap(s.sanitize(streamIndex: 0, pts: 100, dts: 100)) XCTAssertEqual(a.dts, 5000) XCTAssertEqual(v.dts, 100, "video stream is independent of audio's high-water mark") } - func testNoptsDtsPassedThrough() { + // MARK: - Hostile input (audit SEG-101) + + func testDtsAtInt64MaxCannotBeBumpedAndIsDropped() { + var s = OutputTimestampSanitizer() + _ = s.sanitize(streamIndex: 0, pts: Int64.max, dts: Int64.max) + XCTAssertNil(s.sanitize(streamIndex: 0, pts: 0, dts: 0), "no dts after Int64.max exists, so the packet is dropped") + XCTAssertNil(s.sanitize(streamIndex: 0, pts: Int64.max, dts: Int64.max)) + XCTAssertEqual(s.sanitize(streamIndex: 1, pts: 5, dts: 5)?.dts, 5, "the other stream keeps its own high-water mark") + } + + func testNoptsDtsPassedThrough() throws { var s = OutputTimestampSanitizer() - let out = s.sanitize(streamIndex: 0, pts: 123, dts: Int64.min) + let out = try XCTUnwrap(s.sanitize(streamIndex: 0, pts: 123, dts: Int64.min)) XCTAssertEqual(out.dts, Int64.min) XCTAssertEqual(out.pts, 123) } diff --git a/Tests/AetherEngineTests/PacketRingBufferBehaviourTests.swift b/Tests/AetherEngineTests/PacketRingBufferBehaviourTests.swift new file mode 100644 index 000000000..34f0e6769 --- /dev/null +++ b/Tests/AetherEngineTests/PacketRingBufferBehaviourTests.swift @@ -0,0 +1,211 @@ +import Foundation +import Testing +@testable import AetherEngine + +/// The DVR ring's public behaviour, pinned independently of how it stores bytes: append and read by +/// sequence, the sequence bounds an evicting ring reports, keyframe-aligned eviction order, the +/// rewind lookups and the still run the scrub thumbnail decodes from. Written against the +/// file-per-packet ring first and kept unchanged across the chunk-spool rewrite (audit PERF-101). +@Suite("DVR ring behaviour") +struct PacketRingBufferBehaviourTests { + + private func makeScratch(_ dirs: ScratchDirs) -> URL { dirs.make(prefix: "prbbehave") } + + private func payload(_ i: Int, count: Int = 16) -> Data { + Data((0..= lastFirst, "firstSeq never moves backwards") + #expect(bounds.end == n + 1) + lastFirst = bounds.first + } + } + let bounds = ring.seqBounds + #expect(bounds.first > 0) + let first = try #require(ring.packet(atSeq: bounds.first)) + #expect(first.isKeyframe && first.isVideo, "the retained span opens on a keyframe") + #expect(first.bytes == payload(bounds.first)) + #expect(ring.packet(atSeq: bounds.first - 1) == nil, "the evicted prefix reads as gone") + #expect(ring.packet(atSeq: bounds.end - 1)?.bytes == payload(bounds.end - 1)) + + let edge = Double(60 * 25 - 1) / 25 + let oldest = try #require(ring.oldestPts) + #expect(oldest <= edge - 10, "never evicts inside the window") + #expect(oldest > edge - 10 - 2.0 - 0.001, "evicts up to the last keyframe at or before the cutoff") + } + + @Test("Nothing is evicted while the retained history fits the window, and an infinite window keeps everything") + func wholeWindowIsKept() throws { + let dirs = ScratchDirs() + let short = try PacketRingBuffer(windowSeconds: 100, scratch: makeScratch(dirs)) + defer { short.close() } + try fill(short, seconds: 30) + #expect(short.seqBounds.first == 0) + #expect(short.seqBounds.end == 30 * 25 * 2) + + let forever = try PacketRingBuffer(windowSeconds: .infinity, scratch: makeScratch(dirs)) + defer { forever.close() } + try fill(forever, seconds: 30) + #expect(forever.seqBounds.first == 0) + #expect(forever.seqBounds.end == 30 * 25 * 2) + } + + @Test("Rewind lookups: newest keyframe at or before a target, earliest keyframe as the floor") + func rewindLookups() throws { + let dirs = ScratchDirs() + let ring = try PacketRingBuffer(windowSeconds: 3600, scratch: makeScratch(dirs)) + defer { ring.close() } + // The session joined mid-GOP: two non-key packets precede the first keyframe. + try ring.append(pts: 4.8, isKeyframe: false, isVideo: true, bytes: Data([0])) + try ring.append(pts: 4.9, isKeyframe: false, isVideo: false, bytes: Data([1])) + try ring.append(pts: 5.0, isKeyframe: true, isVideo: true, bytes: Data([2])) + try ring.append(pts: 5.1, isKeyframe: false, isVideo: true, bytes: Data([3])) + try ring.append(pts: 7.0, isKeyframe: true, isVideo: true, bytes: Data([4])) + try ring.append(pts: 7.1, isKeyframe: false, isVideo: true, bytes: Data([5])) + + #expect(ring.firstKeyframeSeq() == 2, "the floor skips the mid-GOP leading entries") + #expect(ring.seq(forKeyframeAtOrBefore: 6.0) == 2) + #expect(ring.seq(forKeyframeAtOrBefore: 7.0) == 4) + #expect(ring.seq(forKeyframeAtOrBefore: 99) == 4) + #expect(ring.seq(forKeyframeAtOrBefore: 4.95) == nil, "before every keyframe") + #expect(try ring.keyframePts(atOrBefore: 6.0) == 5.0) + #expect(try ring.keyframePts(atOrBefore: 4.0) == nil) + #expect(ring.oldestPts == 4.8) + } + + @Test("Rewind lookups stay correct after eviction has moved the first sequence") + func rewindLookupsAfterEviction() throws { + let dirs = ScratchDirs() + let ring = try PacketRingBuffer(windowSeconds: 10, scratch: makeScratch(dirs)) + defer { ring.close() } + try fill(ring, seconds: 60, gopSeconds: 2) + let bounds = ring.seqBounds + #expect(bounds.first > 0) + let floor = try #require(ring.firstKeyframeSeq()) + #expect(floor >= bounds.first) + let floorPacket = try #require(ring.packet(atSeq: floor)) + #expect(floorPacket.isKeyframe && floorPacket.isVideo) + #expect(ring.seq(forKeyframeAtOrBefore: floorPacket.pts - 0.5) == nil) + #expect(ring.seq(forKeyframeAtOrBefore: floorPacket.pts) == floor) + let newest = try #require(ring.seq(forKeyframeAtOrBefore: .infinity)) + let newestPacket = try #require(ring.packet(atSeq: newest)) + #expect(newestPacket.isKeyframe && newestPacket.pts > 55) + } + + @Test("packets(fromPts:) replays from the first video keyframe, audio included") + func replayStartsOnAVideoKeyframe() throws { + let dirs = ScratchDirs() + let ring = try PacketRingBuffer(windowSeconds: 3600, scratch: makeScratch(dirs)) + defer { ring.close() } + try ring.append(pts: 1.0, isKeyframe: false, isVideo: false, bytes: Data([1])) + try ring.append(pts: 1.1, isKeyframe: false, isVideo: true, bytes: Data([2])) + try ring.append(pts: 1.2, isKeyframe: true, isVideo: true, bytes: Data([3])) + try ring.append(pts: 1.2, isKeyframe: false, isVideo: false, bytes: Data([4])) + try ring.append(pts: 1.3, isKeyframe: false, isVideo: true, bytes: Data([5])) + + let replay = try ring.packets(fromPts: 1.0) + #expect(replay.map { $0.bytes.first } == [3, 4, 5]) + #expect(try ring.packets(fromPts: 1.25).isEmpty, "a start after the keyframe has no video keyframe to open on") + #expect(try ring.packets(fromPts: 99).isEmpty) + } + + @Test("A still run opens on the keyframe, carries video only, and respects its bounds") + func stillRunShape() throws { + let dirs = ScratchDirs() + let ring = try PacketRingBuffer(windowSeconds: 3600, scratch: makeScratch(dirs)) + defer { ring.close() } + try fill(ring, seconds: 10, gopSeconds: 2) + + let run = try #require(ring.stillRun(target: 5.0, maxPackets: 500, maxSpanSeconds: 30, reorderTail: 0)) + #expect(run.first?.isKeyframe == true) + #expect(run.allSatisfy { $0.isVideo }) + #expect(run.first?.pts == 4.0) + #expect(run.last?.pts == 5.0) + #expect(run.count == 26) + + let tailed = try #require(ring.stillRun(target: 5.0, maxPackets: 500, maxSpanSeconds: 30, reorderTail: 2)) + #expect(tailed.count == 28) + + #expect(ring.stillRun(target: 5.0, maxPackets: 10, maxSpanSeconds: 30, reorderTail: 0) == nil) + #expect(ring.stillRun(target: 5.0, maxPackets: 500, maxSpanSeconds: 0.5, reorderTail: 0) == nil) + #expect(ring.stillRun(target: -1, maxPackets: 500, maxSpanSeconds: 30, reorderTail: 0) == nil) + + let atEdge = try #require(ring.stillRun(target: 99, maxPackets: 500, maxSpanSeconds: 300, reorderTail: 0)) + #expect(atEdge.last?.pts == 9.96, "a target beyond the newest packet clamps to it") + } + + @Test("close() clears the index at once and is idempotent") + func closeIsImmediateAndIdempotent() throws { + let dirs = ScratchDirs() + let ring = try PacketRingBuffer(windowSeconds: 60, scratch: makeScratch(dirs)) + try fill(ring, seconds: 4) + #expect(ring.seqBounds.end > 0) + ring.close() + #expect(ring.seqBounds.first == ring.seqBounds.end) + #expect(ring.packet(atSeq: 0) == nil) + #expect(ring.oldestPts == nil) + ring.close() + } +} + +/// Removes every directory it handed out when the test that owns it ends. `PacketRingBuffer.close()` +/// unlinks in the background, which a finished test process does not wait for. +private final class ScratchDirs: @unchecked Sendable { + private let lock = NSLock() + private var dirs: [URL] = [] + func make(prefix: String) -> URL { + let dir = FileManager.default.temporaryDirectory + .appendingPathComponent("\(prefix)-\(UUID().uuidString)", isDirectory: true) + try? FileManager.default.createDirectory(at: dir, withIntermediateDirectories: true) + lock.lock(); dirs.append(dir); lock.unlock() + return dir + } + deinit { for dir in dirs { try? FileManager.default.removeItem(at: dir) } } +} diff --git a/Tests/AetherEngineTests/PacketRingBufferChunkSpoolTests.swift b/Tests/AetherEngineTests/PacketRingBufferChunkSpoolTests.swift new file mode 100644 index 000000000..abf7de958 --- /dev/null +++ b/Tests/AetherEngineTests/PacketRingBufferChunkSpoolTests.swift @@ -0,0 +1,543 @@ +import Darwin +import Foundation +import Testing +@testable import AetherEngine + +/// Audit PERF-101 / SEG-106 / VPERF-101 / SEG-105: the software live DVR ring spools into a few +/// append-only chunk files behind a compact index, bounds its disk use by bytes as well as by time, +/// makes room when a write fails, and marks its directory live for the segment cache's sweep. +@Suite("DVR ring chunk spool") +struct PacketRingBufferChunkSpoolTests { + + private func makeScratch(_ dirs: ScratchDirs) -> URL { dirs.make(prefix: "prbspool") } + + private func packetFiles(in dir: URL) -> [URL] { + ((try? FileManager.default.contentsOfDirectory(at: dir, includingPropertiesForKeys: nil)) ?? []) + .filter { $0.lastPathComponent != "session.lock" } + } + + private func bytesOnDisk(in dir: URL) -> Int { + packetFiles(in: dir).reduce(0) { sum, url in + var info = stat() + return lstat(url.path, &info) == 0 ? sum + Int(info.st_size) : sum + } + } + + @Test("Packets spool into a few chunk files, not one file per packet") + func spoolsIntoChunkFiles() throws { + let dirs = ScratchDirs() + let scratch = makeScratch(dirs) + let ring = try PacketRingBuffer(windowSeconds: 3600, scratch: scratch) + defer { ring.close() } + let payload = Data(repeating: 0x5A, count: 1024) + for i in 0..<2000 { + try ring.append(pts: Double(i) * 0.04, isKeyframe: i % 50 == 0, isVideo: true, bytes: payload) + } + #expect(ring.seqBounds.end == 2000) + #expect(packetFiles(in: scratch).count <= 2, + "2 MB of packets fit one 4 MB chunk; a file per packet is the PERF-101 defect") + let last = try #require(ring.packet(atSeq: 1999)) + #expect(last.bytes == payload) + } + + @Test("A ring directory with a held marker survives the segment cache's stale sweep") + func heldRingDirectorySurvivesSweep() throws { + let base = URL(fileURLWithPath: NSTemporaryDirectory(), isDirectory: true) + .appendingPathComponent("prbsweep-\(UUID().uuidString)", isDirectory: true) + try FileManager.default.createDirectory(at: base, withIntermediateDirectories: true) + defer { try? FileManager.default.removeItem(at: base) } + let scratch = base.appendingPathComponent("dvr-\(UUID().uuidString)", isDirectory: true) + try FileManager.default.createDirectory(at: scratch, withIntermediateDirectories: true) + + let ring = try PacketRingBuffer(windowSeconds: 5400, scratch: scratch) + defer { ring.close() } + try ring.append(pts: 0, isKeyframe: true, isVideo: true, bytes: Data([1, 2, 3])) + + // Ninety minutes into a DVR session: the directory is as old as the session. + let past = Date().addingTimeInterval(-7200) + try FileManager.default.setAttributes([.creationDate: past], ofItemAtPath: scratch.path) + let created = try scratch.resourceValues(forKeys: [.creationDateKey]).creationDate + #expect(created.map { $0 < Date().addingTimeInterval(-3600) } == true) + + let cache = SegmentCache(baseDirectory: base) + defer { cache.close() } + + #expect(FileManager.default.fileExists(atPath: scratch.path), + "a live ring's directory must not be swept by age alone") + #expect(ring.packet(atSeq: 0)?.bytes == Data([1, 2, 3])) + } + + // MARK: - Index footprint + + @Test("The index is a flat record per packet, not a heap object per packet") + func indexIsCompact() throws { + let dirs = ScratchDirs() + #expect(MemoryLayout.stride == 24) + let ring = try PacketRingBuffer(windowSeconds: .infinity, scratch: makeScratch(dirs)) + defer { ring.close() } + let payload = Data(repeating: 1, count: 64) + let count = 100_000 + for i in 0...stride + #expect(ring.indexFootprintBytes < flatMinimum * 6, + "footprint \(ring.indexFootprintBytes) against a flat minimum of \(flatMinimum)") + } + + // MARK: - Byte budget + + @Test("A byte budget bounds the disk use, even with an infinite window, and the span opens on a keyframe") + func byteBudgetBoundsDiskAndKeepsKeyframeAlignment() throws { + let dirs = ScratchDirs() + let scratch = makeScratch(dirs) + let budget = 8 << 20 + let ring = try PacketRingBuffer(windowSeconds: .infinity, scratch: scratch, byteBudget: budget) + defer { ring.close() } + let size = 256 << 10 + func body(_ i: Int) -> Data { Data(repeating: UInt8(truncatingIfNeeded: i), count: size) } + var retainedBound = 0 + for i in 0..<200 { + try ring.append(pts: Double(i) * 0.5, isKeyframe: i % 8 == 0, isVideo: true, bytes: body(i)) + retainedBound = max(retainedBound, bytesOnDisk(in: scratch)) + } + // 512 KB chunks: the budget plus the one chunk the writer is filling. + #expect(retainedBound <= budget + (512 << 10), "on-disk bytes stayed within budget plus one chunk") + #expect(ring.diskBytes <= budget + (512 << 10)) + let bounds = ring.seqBounds + #expect(bounds.end == 200) + #expect(bounds.first > 0, "50 MB of packets cannot fit an 8 MB budget") + let first = try #require(ring.packet(atSeq: bounds.first)) + #expect(first.isKeyframe && first.isVideo) + #expect(first.bytes == body(bounds.first)) + #expect(ring.packet(atSeq: bounds.end - 1)?.bytes == body(199)) + #expect(ring.packet(atSeq: bounds.first - 1) == nil) + } + + @Test("The budget holds when the window is finite too, and the stricter bound wins") + func strictestBoundWins() throws { + let dirs = ScratchDirs() + let ring = try PacketRingBuffer(windowSeconds: 20, scratch: makeScratch(dirs), byteBudget: 64 << 20) + defer { ring.close() } + let payload = Data(repeating: 3, count: 4 << 10) + for i in 0..<400 { + try ring.append(pts: Double(i), isKeyframe: i % 5 == 0, isVideo: true, bytes: payload) + } + let oldest = try #require(ring.oldestPts) + #expect(oldest <= 399 - 20 && oldest > 399 - 20 - 5, "time still bounds it when bytes do not") + } + + @Test("A budget smaller than one chunk never evicts the only chunk") + func tinyBudgetKeepsTheTail() throws { + let dirs = ScratchDirs() + let ring = try PacketRingBuffer(windowSeconds: .infinity, scratch: makeScratch(dirs), byteBudget: 1) + defer { ring.close() } + for i in 0..<50 { + try ring.append(pts: Double(i), isKeyframe: i % 5 == 0, isVideo: true, bytes: Data(repeating: 9, count: 10_000)) + } + let bounds = ring.seqBounds + #expect(bounds.end == 50) + #expect(bounds.first > 0) + #expect(ring.diskBytes <= (64 << 10) + 10_000, "the chunk being written is what stays") + let newest = try #require(ring.packet(atSeq: 49)) + #expect(newest.bytes.count == 10_000) + #expect(ring.packet(atSeq: bounds.first) != nil) + } + + @Test("A GOP larger than the whole budget still cannot grow the directory without bound") + func hugeGopIsBoundedToo() throws { + let dirs = ScratchDirs() + let scratch = makeScratch(dirs) + let budget = 1 << 20 + let ring = try PacketRingBuffer(windowSeconds: .infinity, scratch: scratch, byteBudget: budget) + defer { ring.close() } + // One keyframe, then nothing but delta frames: no pivot exists. + try ring.append(pts: 0, isKeyframe: true, isVideo: true, bytes: Data(repeating: 0, count: 1000)) + for i in 1..<2000 { + try ring.append(pts: Double(i), isKeyframe: false, isVideo: true, bytes: Data(repeating: 1, count: 8 << 10)) + } + #expect(bytesOnDisk(in: scratch) <= 3 * budget) + #expect(ring.seqBounds.end == 2000) + } + + @Test("The resident floor is nil until eviction moves the span, then tracks the oldest keyframe") + func residentFloorFollowsEviction() throws { + let dirs = ScratchDirs() + let ring = try PacketRingBuffer(windowSeconds: .infinity, scratch: makeScratch(dirs), byteBudget: 4 << 20) + defer { ring.close() } + let size = 128 << 10 + try ring.append(pts: 100, isKeyframe: true, isVideo: true, bytes: Data(repeating: 0, count: size)) + #expect(ring.residentFloorSessionSeconds(sessionStartPts: 100) == nil) + for i in 1..<100 { + try ring.append(pts: 100 + Double(i), isKeyframe: i % 4 == 0, isVideo: true, + bytes: Data(repeating: 0, count: size)) + } + let floor = try #require(ring.residentFloorSessionSeconds(sessionStartPts: 100)) + let oldest = try #require(ring.oldestKeyframePts) + #expect(floor == oldest - 100) + #expect(floor > 0) + #expect(ring.residentFloorSessionSeconds(sessionStartPts: .nan) == nil) + } + + // MARK: - Writes that fail + + private final class FlakyDisk: @unchecked Sendable { + private let lock = NSLock() + private var failures = 0 + private var failureCode = ENOSPC + private var capacity: Int? + var ring: PacketRingBuffer? + private(set) var rejected = 0 + + func failNext(_ n: Int, code: Int32 = ENOSPC) { + lock.lock(); failures = n; failureCode = code; lock.unlock() + } + func setCapacity(_ bytes: Int?) { lock.lock(); capacity = bytes; lock.unlock() } + + var writer: PacketRingBuffer.WriteAll { + { [self] fd, bytes, offset in + lock.lock() + var fail = false + var code = ENOSPC + if failures > 0 { failures -= 1; fail = true; code = failureCode } + let cap = capacity + lock.unlock() + if !fail, let cap, let ring, ring.diskBytes + bytes.count > cap { fail = true } + if fail { + lock.lock(); rejected += 1; lock.unlock() + throw NSError(domain: NSPOSIXErrorDomain, code: Int(code)) + } + try PacketRingBuffer.pwriteAll(fd, bytes, offset) + } + } + } + + private func flakyRing(disk: FlakyDisk, dirs: ScratchDirs, byteBudget: Int = .max) throws -> PacketRingBuffer { + let ring = try PacketRingBuffer(windowSeconds: .infinity, scratch: makeScratch(dirs), + byteBudget: byteBudget, chunkTargetBytes: 64 << 10, + writeAll: disk.writer) + disk.ring = ring + return ring + } + + @Test("A write that fails once for want of space is retried after the oldest chunk is freed, and nothing is dropped", + arguments: [ENOSPC, EDQUOT]) + func failedWriteMakesRoomAndRetries(code: Int32) throws { + let dirs = ScratchDirs() + let disk = FlakyDisk() + let ring = try flakyRing(disk: disk, dirs: dirs) + defer { ring.close() } + let body = Data(repeating: 7, count: 32 << 10) + for i in 0..<10 { + try ring.append(pts: Double(i), isKeyframe: i % 2 == 0, isVideo: true, bytes: body) + } + #expect(ring.seqBounds.first == 0) + disk.failNext(1, code: code) + try ring.append(pts: 10, isKeyframe: true, isVideo: true, bytes: body) + + let bounds = ring.seqBounds + #expect(bounds.end == 11, "the packet that hit the full disk was stored, not dropped") + #expect(bounds.first > 0, "the window shrank to make the room") + let first = try #require(ring.packet(atSeq: bounds.first)) + #expect(first.isKeyframe) + #expect(ring.packet(atSeq: 10)?.bytes == body) + } + + @Test("A write that fails for any other reason drops that packet and keeps the whole history", + arguments: [EMFILE, EIO, ENOENT]) + func otherWriteFailuresKeepTheWindow(code: Int32) throws { + let dirs = ScratchDirs() + let disk = FlakyDisk() + let ring = try flakyRing(disk: disk, dirs: dirs) + defer { ring.close() } + let body = { (i: Int) in Data(repeating: UInt8(truncatingIfNeeded: i), count: 32 << 10) } + for i in 0..<10 { + try ring.append(pts: Double(i), isKeyframe: i % 2 == 0, isVideo: true, bytes: body(i)) + } + let before = ring.seqBounds + let diskBefore = ring.diskBytes + #expect(before.first == 0) + + for keyframe in [false, true] { + disk.failNext(1, code: code) + #expect(throws: (any Error).self) { + try ring.append(pts: 10, isKeyframe: keyframe, isVideo: true, bytes: body(10)) + } + let after = ring.seqBounds + #expect(after.first == before.first && after.end == before.end, + "errno \(code): nothing evicted, nothing indexed (keyframe \(keyframe))") + #expect(ring.diskBytes == diskBefore) + } + for i in 0..<10 { #expect(ring.packet(atSeq: i)?.bytes == body(i)) } + + try ring.append(pts: 10, isKeyframe: false, isVideo: true, bytes: body(10)) + #expect(ring.seqBounds.end == before.end + 1, "the next write goes through") + #expect(ring.seqBounds.first == 0) + } + + @Test("A full volume shrinks the rewind window and the live feed keeps flowing") + func fullVolumeKeepsFeedingWithAShrinkingWindow() throws { + let dirs = ScratchDirs() + let disk = FlakyDisk() + disk.setCapacity(1 << 20) + let ring = try flakyRing(disk: disk, dirs: dirs) + defer { ring.close() } + let body = { (i: Int) in Data(repeating: UInt8(truncatingIfNeeded: i), count: 16 << 10) } + for i in 0..<500 { + try ring.append(pts: Double(i) * 0.04, isKeyframe: i % 5 == 0, isVideo: true, bytes: body(i)) + #expect(ring.seqBounds.end == i + 1, "append \(i) must not freeze at the live edge") + } + #expect(disk.rejected > 0) + #expect(ring.diskBytes <= 1 << 20) + let bounds = ring.seqBounds + #expect(bounds.first > 0) + let first = try #require(ring.packet(atSeq: bounds.first)) + #expect(first.isKeyframe) + #expect(ring.packet(atSeq: 499)?.bytes == body(499)) + } + + @Test("With nothing reclaimable the append throws cleanly, and the ring recovers when space returns") + func nothingToFreeThrowsWithoutCorruption() throws { + let dirs = ScratchDirs() + let disk = FlakyDisk() + let ring = try flakyRing(disk: disk, dirs: dirs) + defer { ring.close() } + let body = { (i: Int) in Data(repeating: UInt8(truncatingIfNeeded: i), count: 4 << 10) } + try ring.append(pts: 0, isKeyframe: true, isVideo: true, bytes: body(0)) + try ring.append(pts: 1, isKeyframe: false, isVideo: true, bytes: body(1)) + + disk.failNext(1) + #expect(throws: (any Error).self) { + try ring.append(pts: 2, isKeyframe: false, isVideo: true, bytes: body(2)) + } + #expect(ring.seqBounds.end == 2, "a dropped packet leaves no entry behind") + + try ring.append(pts: 3, isKeyframe: false, isVideo: true, bytes: body(3)) + #expect(ring.seqBounds.end == 3) + #expect(ring.packet(atSeq: 0)?.bytes == body(0)) + #expect(ring.packet(atSeq: 1)?.bytes == body(1)) + #expect(ring.packet(atSeq: 2)?.bytes == body(3)) + } + + @Test("An incoming keyframe may replace a retained span that cannot otherwise make room") + func incomingKeyframeReplacesATailOnlySpan() throws { + let dirs = ScratchDirs() + let disk = FlakyDisk() + let ring = try flakyRing(disk: disk, dirs: dirs) + defer { ring.close() } + let body = Data(repeating: 5, count: 4 << 10) + for i in 0..<3 { + try ring.append(pts: Double(i), isKeyframe: i == 0, isVideo: true, bytes: body) + } + disk.failNext(1) + try ring.append(pts: 3, isKeyframe: true, isVideo: true, bytes: Data(repeating: 6, count: 4 << 10)) + + let bounds = ring.seqBounds + #expect(bounds.end == 4) + #expect(bounds.first == 3) + let only = try #require(ring.packet(atSeq: 3)) + #expect(only.isKeyframe && only.bytes == Data(repeating: 6, count: 4 << 10)) + } + + // MARK: - Reads beside the writer + + @Test("Readers racing eviction only ever see the bytes of the sequence they asked for") + func concurrentReadersSeeConsistentBytes() async throws { + let dirs = ScratchDirs() + let ring = try PacketRingBuffer(windowSeconds: .infinity, scratch: makeScratch(dirs), + byteBudget: 2 << 20, chunkTargetBytes: 128 << 10) + defer { ring.close() } + let total = 6000 + @Sendable func body(_ i: Int) -> Data { Data(repeating: UInt8(truncatingIfNeeded: i), count: 2000 + i % 500) } + + let finished = Counter() + let mismatches = Counter() + let writerDone = Counter() + let writer = Thread { + for i in 0.. bounds.first else { continue } + state = state &* 6364136223846793005 &+ 1442695040888963407 + let seq = bounds.first + Int(state >> 33) % (bounds.end - bounds.first) + if let p = ring.packet(atSeq: seq) { + if p.bytes != body(seq) || p.isVideo != (seq % 3 != 0) { mismatches.add(1) } + } + if let v = ring.isVideo(atSeq: seq), v != (seq % 3 != 0) { mismatches.add(1) } + } + finished.add(1) + }) + } + writer.start() + readers.forEach { $0.start() } + try await waitFor { finished.value == 4 } + #expect(mismatches.value == 0) + #expect(ring.seqBounds.end == total) + } + + @Test("isVideo(atSeq:) answers from the index: resident kinds, nil outside the span") + func isVideoAnswersFromTheIndex() throws { + let dirs = ScratchDirs() + let ring = try PacketRingBuffer(windowSeconds: .infinity, scratch: makeScratch(dirs), byteBudget: 1 << 20) + defer { ring.close() } + #expect(ring.isVideo(atSeq: 0) == nil) + for i in 0..<300 { + try ring.append(pts: Double(i), isKeyframe: i % 6 == 0, isVideo: i % 2 == 0, + bytes: Data(repeating: 1, count: 16 << 10)) + } + let bounds = ring.seqBounds + #expect(bounds.first > 0) + for seq in bounds.first.. URL { + let dir = FileManager.default.temporaryDirectory + .appendingPathComponent("\(prefix)-\(UUID().uuidString)", isDirectory: true) + try? FileManager.default.createDirectory(at: dir, withIntermediateDirectories: true) + lock.lock(); dirs.append(dir); lock.unlock() + return dir + } + deinit { for dir in dirs { try? FileManager.default.removeItem(at: dir) } } +} diff --git a/Tests/AetherEngineTests/PacketRingBufferTests.swift b/Tests/AetherEngineTests/PacketRingBufferTests.swift index cb2df5a70..a2b42d80d 100644 --- a/Tests/AetherEngineTests/PacketRingBufferTests.swift +++ b/Tests/AetherEngineTests/PacketRingBufferTests.swift @@ -7,6 +7,7 @@ final class PacketRingBufferTests: XCTestCase { let d = FileManager.default.temporaryDirectory .appendingPathComponent("prbtest-\(ProcessInfo.processInfo.globallyUniqueString)") try? FileManager.default.createDirectory(at: d, withIntermediateDirectories: true) + addTeardownBlock { try? FileManager.default.removeItem(at: d) } return d } func testAppendAndKeyframeSeek() throws { diff --git a/Tests/AetherEngineTests/PacketTimingDeltaTests.swift b/Tests/AetherEngineTests/PacketTimingDeltaTests.swift new file mode 100644 index 000000000..28add33c8 --- /dev/null +++ b/Tests/AetherEngineTests/PacketTimingDeltaTests.swift @@ -0,0 +1,20 @@ +import Testing +@testable import AetherEngine + +/// Audit SUB-112 item 4: `aetherctl pktdump` (the public `PacketTimingProbe.run`) subtracted packet +/// timestamps from the probed file unchecked, so dts on opposite sides near 2^62 trapped the probe. +@Suite("Packet timing probe dts deltas") +struct PacketTimingDeltaTests { + + @Test("A delta no Int64 can hold is nil instead of a trap") + func overflowingDelta() { + #expect(PacketTimingProbe.dtsDelta(Int64.max - 1, after: -(1 << 62)) == nil) + #expect(PacketTimingProbe.dtsDelta(-(1 << 62) - 10, after: Int64.max - 1) == nil) + } + + @Test("Ordinary deltas are unchanged, backward ones included") + func ordinaryDelta() { + #expect(PacketTimingProbe.dtsDelta(4004, after: 3003) == 1001) + #expect(PacketTimingProbe.dtsDelta(3003, after: 4004) == -1001) + } +} diff --git a/Tests/AetherEngineTests/PlaylistNamedHostCredentialTests.swift b/Tests/AetherEngineTests/PlaylistNamedHostCredentialTests.swift new file mode 100644 index 000000000..836524151 --- /dev/null +++ b/Tests/AetherEngineTests/PlaylistNamedHostCredentialTests.swift @@ -0,0 +1,287 @@ +import Foundation +import Testing +@testable import AetherEngine + +/// A playlist can name any host and any scheme for a variant, a rendition, a segment or a key. The +/// host's credential headers were set on every one of those fetches, including an http:// URI inside +/// an https:// playlist. They now follow the redirect rule (audit NET-7): the token only to the origin +/// the host gave it for, with no downgrade, and every other header wherever the playlist points. +@Suite("Credential headers stay with the host's origin", .serialized) +struct PlaylistNamedHostCredentialTests { + + private let headers = [ + "Authorization": "MediaBrowser Token=\"t0k3n\"", + "X-Emby-Token": "t0k3n", + "Cookie": "connect.sid=s1", + "Referer": "https://portal.example/", + "User-Agent": "UA/1", + ] + private let credentialNames = ["Authorization", "X-Emby-Token", "Cookie"] + private let origin = URL(string: "https://media.example/live/master.m3u8")! + + @Test("The live ingest keeps the token on its own origin and drops it elsewhere", arguments: [ + ("https://media.example/live/seg1.ts", true), + ("https://media.example:443/keys/k1", true), + ("https://cdn.other/seg1.ts", false), + ("http://media.example/live/seg1.ts", false), + ("https://media.example:8443/seg1.ts", false), + ]) + func liveIngest(target: String, keepsCredentials: Bool) throws { + let reader = HLSLiveIngestReader(playlistURL: origin, httpHeaders: headers) + let request = reader.makeRequest(try #require(URL(string: target))) + expect(request, keepsCredentials: keepsCredentials) + } + + @Test("A companion reader judges by its parent's origin, not by the rendition URL it was handed") + func companionInheritsTheHostOrigin() throws { + let rendition = try #require(URL(string: "https://cdn.other/audio/index.m3u8")) + let companion = HLSLiveIngestReader( + playlistURL: rendition, httpHeaders: headers, role: .companionAudio, credentialOrigin: origin) + expect(companion.makeRequest(try #require(URL(string: "https://cdn.other/audio/a1.aac"))), + keepsCredentials: false) + expect(companion.makeRequest(try #require(URL(string: "https://media.example/audio/a1.aac"))), + keepsCredentials: true) + } + + @Test("The VOD ingest applies the same rule") + func vodIngest() throws { + let reader = HLSVODIngestReader(playlistURL: origin, httpHeaders: headers) + defer { reader.close() } + expect(reader.makeRequest(try #require(URL(string: "https://media.example/v/seg0.ts"))), + keepsCredentials: true) + expect(reader.makeRequest(try #require(URL(string: "http://media.example/v/seg0.ts"))), + keepsCredentials: false) + } + + @Test("The relay sends credentials to the host's origins only, not to one a playlist revealed") + func relayHeaders() throws { + let scope = CredentialScope(headers: headers, anchors: [origin]) + let own = scope.headers(for: try #require(URL(string: "https://media.example/hls/seg.ts"))) + #expect(own == headers) + let discovered = scope.headers(for: try #require(URL(string: "http://cdn.other/seg.ts"))) + for name in credentialNames { #expect(discovered[name] == nil, "\(name) reached a discovered origin") } + #expect(discovered["Referer"] == "https://portal.example/") + #expect(discovered["User-Agent"] == "UA/1") + } + + /// Audit NET-109: allowing a fetch and granting the token were one call, so anything the relay + /// was told it may fetch also received the credentials. + @Test("An origin the relay is only allowed to fetch gets no credentials") + func relayAllowIsNotGrant() throws { + let relay = HLSOriginRelay() + defer { relay.stop() } + let edge = try #require(URL(string: "https://edge.cdn/v.m3u8")) + relay.grantCredentials(to: origin, httpHeaders: headers) + relay.allow(edge) + + #expect(relay.credentialAnchors == [origin]) + let onEdge = relay.upstreamHeaders(for: try #require(URL(string: "https://edge.cdn/seg.ts"))) + for name in credentialNames { #expect(onEdge[name] == nil, "\(name) reached an allowed origin") } + #expect(onEdge["Referer"] == "https://portal.example/") + let onOrigin = relay.upstreamHeaders(for: try #require(URL(string: "https://media.example/seg.ts"))) + #expect(onOrigin == headers) + } + + @Test("Asking the server for a relay address grants nothing") + func relayURLOnlyAllows() throws { + let relay = HLSOriginRelay() + let server = HLSLocalServer(relay: relay) + try server.start() + defer { server.stop(); relay.stop() } + relay.grantCredentials(to: origin, httpHeaders: headers) + let other = try #require(URL(string: "https://cdn.other/v/index.m3u8")) + + _ = try #require(server.relayURL(for: other)) + + #expect(relay.credentialAnchors == [origin]) + let onOther = relay.upstreamHeaders(for: other) + for name in credentialNames { #expect(onOther[name] == nil, "\(name) reached \(other)") } + } + + /// Audit Vcred-101: the live rendition fetch set every host header on the rendition playlist and + /// every WebVTT segment the master named. + @Test("The live subtitle rendition fetch judges by the ingest's origin", arguments: [ + ("https://media.example/subs/en.m3u8", true), + ("https://subs.other/en.m3u8", false), + ("http://media.example/subs/en.m3u8", false), + ]) + func liveSubtitleRendition(target: String, keepsCredentials: Bool) throws { + let reader = HLSLiveIngestReader(playlistURL: origin, httpHeaders: headers) + let scope = AetherEngine.liveSubtitleRenditionCredentials(headers: headers, source: reader) + let sent = scope.headers(for: try #require(URL(string: target))) + for name in credentialNames { + #expect((sent[name] != nil) == keepsCredentials, "\(name) on \(target)") + } + #expect(sent["Referer"] == "https://portal.example/") + } + + @Test("Without a live ingest reader the rendition fetch carries no credentials at all") + func liveSubtitleRenditionWithoutAnIngest() throws { + let scope = AetherEngine.liveSubtitleRenditionCredentials(headers: headers, source: nil) + let sent = scope.headers(for: origin) + for name in credentialNames { #expect(sent[name] == nil, "\(name) went out with no anchor") } + } + + @Test("The carriage probe does not carry the token to a variant on another host") + func carriageProbeVariant() async throws { + CredentialCaptureProtocol.reset() + let variant = "http://cdn.other/v/index.m3u8" + CredentialCaptureProtocol.bodies[origin.absoluteString] = Data(""" + #EXTM3U + #EXT-X-STREAM-INF:BANDWIDTH=1000000 + \(variant) + """.utf8) + CredentialCaptureProtocol.bodies[variant] = Data(""" + #EXTM3U + #EXT-X-TARGETDURATION:6 + #EXT-X-MAP:URI="init.mp4" + #EXTINF:6.0, + seg0.m4s + """.utf8) + let configuration = URLSessionConfiguration.ephemeral + configuration.protocolClasses = [CredentialCaptureProtocol.self] + let session = URLSession(configuration: configuration) + defer { session.invalidateAndCancel() } + + let evidence = await HLSCarriageProbe.classifyFromPlaylists( + playlistURL: origin, httpHeaders: headers, advertisesFragmentedMP4OnlyVideo: false, + session: session) + #expect(evidence == .settled(.otherCarriage)) + + let master = try #require(CredentialCaptureProtocol.seen[origin.absoluteString]) + let onVariant = try #require(CredentialCaptureProtocol.seen[variant]) + #expect(master["X-Emby-Token"] == "t0k3n") + for name in credentialNames { #expect(onVariant[name] == nil, "\(name) reached \(variant)") } + #expect(onVariant["Referer"] == "https://portal.example/") + } + + /// Audit NAT-105 / NET-109: the #316 build fetched the master's first variant with every host + /// header, whatever host the master named, and anchored the relay's grant on the redirect target. + @Test("The subtitle proxy keeps the token off an edge the master redirected to", + .timeLimit(.minutes(2))) + func subtitleProxyAfterACrossOriginRedirect() async throws { + let originMaybe = CannedHTTPOrigin() + let edgeMaybe = CannedHTTPOrigin() + let originServer = try #require(originMaybe) + let edge = try #require(edgeMaybe) + defer { originServer.stop(); edge.stop() } + originServer.route("/master.m3u8", .redirect(to: "\(edge.baseURL)/master.m3u8")) + edge.route("/master.m3u8", .body(""" + #EXTM3U + #EXT-X-STREAM-INF:BANDWIDTH=1000000 + v/index.m3u8 + """, contentType: "application/vnd.apple.mpegurl")) + edge.route("/v/index.m3u8", .body(""" + #EXTM3U + #EXT-X-TARGETDURATION:6 + #EXTINF:6.0, + seg0.ts + #EXT-X-ENDLIST + """, contentType: "application/vnd.apple.mpegurl")) + let originURL = try #require(URL(string: "\(originServer.baseURL)/master.m3u8")) + let sidecar = try #require(URL(string: "\(originServer.baseURL)/en.vtt")) + + let prepared = try #require(await RemoteHLSSubtitleProxy.prepare( + originURL: originURL, + tracks: [.init(externalID: 100_000, source: ExternalSubtitleTrack(url: sidecar, language: "en"))], + httpHeaders: headers, needsRelay: true)) + defer { prepared.tearDown() } + #expect(prepared.servesSubtitleRenditions) + let relay = try #require(prepared.server.relay) + #expect(relay.credentialAnchors == [originURL], "the grant moved to the redirect target") + let backHome = try #require(URL(string: "\(originServer.baseURL)/seg0.ts")) + #expect(relay.upstreamHeaders(for: backHome)["X-Emby-Token"] == "t0k3n", + "a relayed request to the host's own origin lost its token") + let atEdge = try #require(URL(string: "\(edge.baseURL)/v/seg0.ts")) + #expect(relay.upstreamHeaders(for: atEdge)["X-Emby-Token"] == nil) + + let onOrigin = try #require(originServer.requests(to: "/master.m3u8").first) + #expect(onOrigin.headers["x-emby-token"] == "t0k3n") + for path in ["/master.m3u8", "/v/index.m3u8"] { + let seen = try #require(edge.requests(to: path).first, "\(path) was never fetched") + for name in credentialNames { + #expect(seen.headers[name.lowercased()] == nil, "\(name) reached the edge on \(path)") + } + #expect(seen.headers["referer"] == "https://portal.example/") + } + } + + /// Audit DEC-103: the remote-HLS audio tap put every host header on every playlist, segment and + /// key a playlist named. + @Test("The audio tap keeps the token on the loaded origin", arguments: [ + ("https://media.example/live/seg1.ts", true), + ("https://cdn.other/seg1.ts", false), + ("http://media.example/live/seg1.ts", false), + ]) + func audioTapSegment(target: String, keepsCredentials: Bool) async throws { + CredentialCaptureProtocol.reset() + CredentialCaptureProtocol.bodies[target] = Data([0x47, 0x40, 0x11, 0x10]) + let configuration = URLSessionConfiguration.ephemeral + configuration.protocolClasses = [CredentialCaptureProtocol.self] + let session = URLSession(configuration: configuration) + defer { session.invalidateAndCancel() } + let fetcher = AudioTapHLSFetcher(session: session, httpHeaders: headers, credentialOrigin: origin) + let url = try #require(URL(string: target)) + + _ = try await fetcher.fetchSegment(url, crypt: nil, base: origin) + + let seen = try #require(CredentialCaptureProtocol.seen[target]) + for name in credentialNames { + #expect((seen[name] != nil) == keepsCredentials, "\(name) on \(target)") + } + #expect(seen["Referer"] == "https://portal.example/") + } + + private func expect(_ request: URLRequest, keepsCredentials: Bool, + sourceLocation: SourceLocation = #_sourceLocation) { + for name in credentialNames { + let value = request.value(forHTTPHeaderField: name) + #expect((value != nil) == keepsCredentials, "\(name) on \(request.url!)", + sourceLocation: sourceLocation) + } + #expect(request.value(forHTTPHeaderField: "Referer") == "https://portal.example/", + sourceLocation: sourceLocation) + #expect(request.value(forHTTPHeaderField: "User-Agent") == "UA/1", sourceLocation: sourceLocation) + } +} + +/// Answers from canned bodies and records the headers each URL was asked with. Its own class rather +/// than the #119 suite's, whose statics a parallel suite would reset under this one. +private final class CredentialCaptureProtocol: URLProtocol, @unchecked Sendable { + private static let lock = NSLock() + nonisolated(unsafe) private static var _bodies: [String: Data] = [:] + nonisolated(unsafe) private static var _seen: [String: [String: String]] = [:] + + static var bodies: [String: Data] { + get { lock.withLock { _bodies } } + set { lock.withLock { _bodies = newValue } } + } + + static var seen: [String: [String: String]] { lock.withLock { _seen } } + + static func reset() { + lock.withLock { _bodies = [:]; _seen = [:] } + } + + override class func canInit(with request: URLRequest) -> Bool { true } + override class func canonicalRequest(for request: URLRequest) -> URLRequest { request } + override func stopLoading() {} + + override func startLoading() { + guard let url = request.url else { + client?.urlProtocol(self, didFailWithError: URLError(.badURL)) + return + } + let fields = request.allHTTPHeaderFields ?? [:] + Self.lock.withLock { Self._seen[url.absoluteString] = fields } + guard let data = Self.bodies[url.absoluteString] else { + client?.urlProtocol(self, didFailWithError: URLError(.fileDoesNotExist)) + return + } + let response = HTTPURLResponse(url: url, statusCode: 200, httpVersion: "HTTP/1.1", + headerFields: ["Content-Length": String(data.count)])! + client?.urlProtocol(self, didReceive: response, cacheStoragePolicy: .notAllowed) + client?.urlProtocol(self, didLoad: data) + client?.urlProtocolDidFinishLoading(self) + } +} diff --git a/Tests/AetherEngineTests/PregateAudioBufferTests.swift b/Tests/AetherEngineTests/PregateAudioBufferTests.swift index e9f896004..96faf7570 100644 --- a/Tests/AetherEngineTests/PregateAudioBufferTests.swift +++ b/Tests/AetherEngineTests/PregateAudioBufferTests.swift @@ -74,4 +74,21 @@ struct PregateAudioBufferTests { isAudioPkt: true, audioWaitForVideo: true, isHeadOfStream: false, isLive: false, bufferedBytes: cap - 1024, packetSize: 1025, capBytes: cap) == false) } + + // Audit SEG-102: the byte cap counts payload only, so 1-byte (or empty) laced packets let the + // buffer pin about 8.4 million AVPackets, 1.4 GB of heap, while the VOD gate waits unbounded. + @Test("Tiny packets are refused once the entry cap is reached, however few bytes they carry") + func entryCapBindsTinyPackets() { + let entries = HLSSegmentProducer.maxPregateAudioBufferEntries + #expect(entries == 65_536) + #expect(HLSSegmentProducer.shouldBufferPregateAudio( + isAudioPkt: true, audioWaitForVideo: true, isHeadOfStream: true, isLive: false, + bufferedBytes: entries - 1, packetSize: 1, capBytes: cap, bufferedCount: entries - 1) == true) + #expect(HLSSegmentProducer.shouldBufferPregateAudio( + isAudioPkt: true, audioWaitForVideo: true, isHeadOfStream: true, isLive: false, + bufferedBytes: entries, packetSize: 1, capBytes: cap, bufferedCount: entries) == false) + #expect(HLSSegmentProducer.shouldBufferPregateAudio( + isAudioPkt: true, audioWaitForVideo: true, isHeadOfStream: false, isLive: false, + bufferedBytes: 0, packetSize: 0, capBytes: cap, bufferedCount: entries) == false) + } } diff --git a/Tests/AetherEngineTests/PrematureEndOfItemTests.swift b/Tests/AetherEngineTests/PrematureEndOfItemTests.swift index d82a082f4..048d680a9 100644 --- a/Tests/AetherEngineTests/PrematureEndOfItemTests.swift +++ b/Tests/AetherEngineTests/PrematureEndOfItemTests.swift @@ -152,4 +152,25 @@ struct PrematureEndOfItemTests { attemptsUsed: AetherEngine.prematureEndRecoveryMaxAttempts, lastAttemptPlayhead: nil)) } + + /// The recovery runs inside AVPlayer's end notification against a live item, so this reads its + /// body, as the #93 fallback test does: a pause made through the engine while the re-seek was in + /// flight clears the play intent, and the recovery must not restart the player over it. + @Test("The recovery resumes only if the viewer still wants playback after the re-seek") + func recoveryKeepsAPauseMadeDuringTheReseek() throws { + let source = URL(fileURLWithPath: #filePath) + .deletingLastPathComponent() + .deletingLastPathComponent() + .deletingLastPathComponent() + .appendingPathComponent("Sources/AetherEngine/Native/NativeAVPlayerHost.swift") + let text = try #require(try? String(contentsOf: source, encoding: .utf8)) + let fn = try #require(text.range(of: "func recoverFromPrematureEnd(")) + let end = try #require(text[fn.upperBound...].range(of: "\n }\n")) + let body = text[fn.lowerBound.. Bool { + lock.lock() + defer { lock.unlock() } + if claimed { return false } + claimed = true + return true + } + } + + private func expected(at offset: Int64, count: Int) -> [UInt8] { + (0.. (read: Int64, last: Int32, firstBadOffset: Int64?) { + let chunk = 256 * 1024 + let buf = UnsafeMutablePointer.allocate(capacity: chunk) + defer { buf.deallocate() } + var read: Int64 = 0 + var last: Int32 = 0 + var nextCheck: Int64 = 0 + var bad: Int64? + while read < total { + last = reader.read(into: buf, size: Int32(chunk)) + if last <= 0 { break } + if read + Int64(last) > nextCheck { + let start = max(0, nextCheck - read) + let n = Int(min(1024, Int64(last) - start)) + let got = Array(UnsafeBufferPointer(start: buf.advanced(by: Int(start)), count: n)) + if got != expected(at: read + start, count: n), bad == nil { bad = read + start } + nextCheck = (read + Int64(last) + 4 * Int64(mb) - 1) / (4 * Int64(mb)) * 4 * Int64(mb) + } + read += Int64(last) + } + return (read, last, bad) + } + + private func response(_ status: Int, _ headers: [String: String]) -> HTTPURLResponse { + HTTPURLResponse(url: URL(string: "http://example.invalid/movie.bin")!, statusCode: status, + httpVersion: "HTTP/1.1", headerFields: headers)! + } + + // MARK: - The classification every reader shares + + @Test("a response is classified by what it did with the range that was asked for") + func classification() { + func answer(_ http: HTTPURLResponse, start: Int64 = 0, end: Int64? = 99) -> AVIOReader.RangeAnswer { + AVIOReader.rangeAnswer(http, requestedStart: start, requestedEnd: end) + } + #expect(answer(response(206, ["Content-Range": "bytes 0-99/1000", "Content-Length": "100"])) == .honoured) + #expect(answer(response(206, ["Content-Range": "bytes 64-99/1000"])) == .misplaced(start: 64)) + #expect(answer(response(206, ["Content-Range": "bytes 0-999/1000", "Content-Length": "1000"])) == .overWide) + #expect(answer(response(206, ["Content-Range": "bytes 0-99/*"])) == .honoured) + #expect(answer(response(200, ["Content-Length": "1000"])) == .ignored) + #expect(answer(response(200, [:])) == .ignored, "a 200 that states no length cannot be the asked range") + #expect(answer(response(200, ["Content-Length": "100"])) == .wholeFile) + #expect(answer(response(200, ["Content-Length": "50"])) == .wholeFile) + #expect(answer(response(200, ["Content-Length": "100"]), start: 5, end: 99) == .ignored, + "a 200 cannot be a range that starts past byte 0") + #expect(answer(response(200, ["Content-Length": "5000"]), end: nil) == .wholeFile, + "an open-ended ask has no span a 200 could exceed") + #expect(answer(response(404, [:])) == .unjudged) + #expect(answer(response(416, [:])) == .unjudged) + } + + // MARK: - On the wire + + /// What a forward-only source owes: a non-seekable pb that still states its length, every byte of + /// the body once and in order, a clean EOF, a bounded buffer, and no ranged request past the open. + private func expectForwardOnlyPlayback(_ reader: AVIOReader, _ server: ThrottledOriginServer, + total: Int64, unrangedRequests: ClosedRange = 1...1) { + #expect(reader.originIgnoresRange) + #expect(!reader.isSeekable, "the pb must be non-seekable so FFmpeg never seeks a source that cannot") + #expect(reader.seek(offset: 0, whence: Self.avseekSize) == total, + "the Content-Length still answers AVSEEK_SIZE, which an MPEG-TS duration estimate needs") + + let result = drain(reader, total: total) + #expect(result.read == total, "read \(result.read / Int64(mb)) MB of \(total / Int64(mb)) MB") + #expect(result.firstBadOffset == nil, "bytes differ from the origin's at \(result.firstBadOffset ?? -1)") + let tailBuf = UnsafeMutablePointer.allocate(capacity: 4096) + defer { tailBuf.deallocate() } + #expect(reader.read(into: tailBuf, size: 4096) == FFmpegErr.eof, "a body read to its end is EOF") + + #expect(reader.streamPeakBufferBytesForTesting <= 64 * mb + 8 * mb, + "the streaming buffer held \(reader.streamPeakBufferBytesForTesting / mb) MB of a \(total / Int64(mb)) MB body") + + // One unranged GET carried the whole body. Everything else is the open's own: the bounded + // range that was answered with a 200, the one byte confirmation, the suffix tail prefetch. + #expect(unrangedRequests.contains(server.rangeHeaderPresence.filter { !$0 }.count), + "\(server.rangeHeaderPresence.filter { !$0 }.count) requests without a Range header") + let tail = total - Int64(AVIOReader.tailPrefetchBytes) + let strays = server.requestLog.enumerated().filter { index, request in + server.rangeHeaderPresence[index] && request.start > 1 && request.start != tail + } + #expect(strays.isEmpty, "ranged requests past the open: \(strays.map(\.element))") + } + + @Test("an origin that ignores Range is read forward-only past the 32 MB range boundary", + .timeLimit(.minutes(2))) + func forwardOnlyPastTheRangeBoundary() throws { + let total = Int64(72 * mb) + let maybe = ThrottledOriginServer(totalSize: total, throttleUs: 500, patternedBody: true, + respond: { _, _, _ in .serve200 }) + let server = try #require(maybe) + defer { server.stop() } + let reader = AVIOReader(url: URL(string: "http://127.0.0.1:\(server.port)/movie.ts")!) + defer { reader.markClosed(); reader.close() } + try reader.open() + + expectForwardOnlyPlayback(reader, server, total: total) + } + + @Test("a cold open that was refused once does not hide an origin that ignores Range", + .timeLimit(.minutes(2))) + func refusedColdOpenStillFindsTheIgnoredRange() throws { + let total = Int64(72 * mb) + let once = FirstClaim() + let respond: @Sendable (Int, Int64, String) -> ThrottledOriginServer.Directive = { _, offset, _ in + offset == 0 && once.take() ? .status(503) : .serve200 + } + let maybe = ThrottledOriginServer(totalSize: total, throttleUs: 500, patternedBody: true, + respond: respond) + let server = try #require(maybe) + defer { server.stop() } + let reader = AVIOReader(url: URL(string: "http://127.0.0.1:\(server.port)/movie.ts")!) + defer { reader.markClosed(); reader.close() } + try reader.open() + + // The size probe's HEAD fallback has no Range header either, and it fires when the probe + // behind a refusal is slow, which is the case here. + expectForwardOnlyPlayback(reader, server, total: total, unrangedRequests: 1...2) + } + + @Test("an origin that honours Range keeps the seekable path and its range refills", + .timeLimit(.minutes(2))) + func honouringOriginIsUnaffected() throws { + let total = Int64(40 * mb) + let server = try #require(ThrottledOriginServer(totalSize: total, throttleUs: 500)) + defer { server.stop() } + let reader = AVIOReader(url: URL(string: "http://127.0.0.1:\(server.port)/movie.bin")!) + defer { reader.markClosed(); reader.close() } + try reader.open() + + #expect(!reader.originIgnoresRange) + #expect(reader.isSeekable) + #expect(reader.seek(offset: 0, whence: Self.avseekSize) == total) + + let chunk = 256 * 1024 + let buf = UnsafeMutablePointer.allocate(capacity: chunk) + defer { buf.deallocate() } + var read: Int64 = 0 + while read < 36 * Int64(mb) { + let n = reader.read(into: buf, size: Int32(chunk)) + #expect(n > 0, "read failed at \(read)") + if n <= 0 { break } + read += Int64(n) + } + #expect(read >= 36 * Int64(mb)) + #expect(server.rangeHeaderPresence.allSatisfy { $0 }, "a ranged source asked for an unranged GET") + #expect(server.requestedRanges.contains { $0.start == 32 * Int64(mb) }, + "the refill at the 32 MB boundary never asked for its range") + #expect(!server.requestLog.contains { $0.start == 1 && $0.end == 1 }, + "an honoured range needs no confirmation") + } + + @Test("an origin that honours Range and refused the cold open once keeps the seekable path", + .timeLimit(.minutes(2))) + func refusedColdOpenOnHonouringOriginStaysSeekable() throws { + let total = Int64(40 * mb) + let once = FirstClaim() + let respond: @Sendable (Int, Int64, String) -> ThrottledOriginServer.Directive = { _, offset, _ in + offset == 0 && once.take() ? .status(503) : .serve206 + } + let server = try #require(ThrottledOriginServer(totalSize: total, throttleUs: 500, respond: respond)) + defer { server.stop() } + let reader = AVIOReader(url: URL(string: "http://127.0.0.1:\(server.port)/movie.bin")!) + defer { reader.markClosed(); reader.close() } + try reader.open() + + #expect(!reader.originIgnoresRange) + #expect(reader.isSeekable) + #expect(reader.resolvedByteSize == total) + + let chunk = 256 * 1024 + let buf = UnsafeMutablePointer.allocate(capacity: chunk) + defer { buf.deallocate() } + var read: Int64 = 0 + while read < 36 * Int64(mb) { + let n = reader.read(into: buf, size: Int32(chunk)) + #expect(n > 0, "read failed at \(read)") + if n <= 0 { break } + read += Int64(n) + } + #expect(read >= 36 * Int64(mb)) + #expect(server.requestedRanges.contains { $0.start == 32 * Int64(mb) }, + "the refill at the 32 MB boundary never asked for its range") + #expect(!server.requestLog.contains { $0.start == 1 && $0.end == 1 }, + "an honoured range needs no confirmation") + } + + @Test("a 200 on a cache miss at the open does not take the seekable path away", + .timeLimit(.minutes(2))) + func cacheMissAtTheOpenIsConfirmedAway() throws { + let total = Int64(40 * mb) + let once = FirstClaim() + let respond: @Sendable (Int, Int64, String) -> ThrottledOriginServer.Directive = { _, offset, _ in + offset == 0 && once.take() ? .serve200 : .serve206 + } + let maybe = ThrottledOriginServer(totalSize: total, throttleUs: 500, respond: respond) + let server = try #require(maybe) + defer { server.stop() } + let reader = AVIOReader(url: URL(string: "http://127.0.0.1:\(server.port)/movie.bin")!) + defer { reader.markClosed(); reader.close() } + try reader.open() + + #expect(!reader.originIgnoresRange) + #expect(reader.isSeekable) + #expect(reader.resolvedByteSize == total) + #expect(server.requestLog.contains { $0.start == 1 && $0.end == 1 }, + "the 200 was never confirmed with a one byte range") + let buf = UnsafeMutablePointer.allocate(capacity: 64 * 1024) + defer { buf.deallocate() } + #expect(reader.read(into: buf, size: 64 * 1024) > 0) + } + + @Test("a small file answered with a 200 that covers it stays sized and seekable", + .timeLimit(.minutes(1))) + func smallWholeFileStaysSeekable() throws { + let total = Int64(4 * mb) + let maybe = ThrottledOriginServer(totalSize: total, throttleUs: 0, patternedBody: true, + respond: { _, _, _ in .serve200 }) + let server = try #require(maybe) + defer { server.stop() } + let reader = AVIOReader(url: URL(string: "http://127.0.0.1:\(server.port)/small.mp4")!) + defer { reader.markClosed(); reader.close() } + try reader.open() + + #expect(!reader.originIgnoresRange) + #expect(reader.isSeekable) + #expect(reader.resolvedByteSize == total) + #expect(!server.requestLog.contains { $0.start == 1 && $0.end == 1 }, + "a whole small file needs no confirmation") + let buf = UnsafeMutablePointer.allocate(capacity: 4096) + defer { buf.deallocate() } + #expect(reader.read(into: buf, size: 4096) == 4096) + #expect(Array(UnsafeBufferPointer(start: buf, count: 4096)) == expected(at: 0, count: 4096)) + } +} diff --git a/Tests/AetherEngineTests/RangedResponseIntegrityTests.swift b/Tests/AetherEngineTests/RangedResponseIntegrityTests.swift new file mode 100644 index 000000000..f17a1cf6e --- /dev/null +++ b/Tests/AetherEngineTests/RangedResponseIntegrityTests.swift @@ -0,0 +1,110 @@ +import Testing +import Foundation +@testable import AetherEngine + +/// A ranged answer is only trusted as far as it matches what was asked: it has to start at the +/// requested offset (audit DMX-5), and nothing past the requested end is kept (audit DMX-1). +@Suite("Ranged response integrity") +struct RangedResponseIntegrityTests { + + private final class FirstRequestAt: @unchecked Sendable { + private let lock = NSLock() + private var seen = false + func claim() -> Bool { + lock.lock() + defer { lock.unlock() } + if seen { return false } + seen = true + return true + } + } + + private func readExact(_ reader: AVIOReader, _ size: Int) -> [UInt8]? { + let buf = UnsafeMutablePointer.allocate(capacity: size) + defer { buf.deallocate() } + var got = 0 + while got < size { + let n = reader.read(into: buf.advanced(by: got), size: Int32(size - got)) + guard n > 0 else { return nil } + got += Int(n) + } + return Array(UnsafeBufferPointer(start: buf, count: size)) + } + + private func expected(at offset: Int64, count: Int) -> [UInt8] { + (0.. ThrottledOriginServer.Directive = { _, offset, _ in + offset == target && once.claim() ? .serve206From(start: target - 4096) : .serve206 + } + let origin = ThrottledOriginServer(totalSize: 512 * 1024 * 1024, patternedBody: true, respond: respond) + let server = try #require(origin) + defer { server.stop() } + let reader = AVIOReader(url: URL(string: "http://127.0.0.1:\(server.port)/movie.bin")!) + defer { reader.markClosed(); reader.close() } + try reader.open() + #expect(readExact(reader, 4096) == expected(at: 0, count: 4096)) + + #expect(reader.seek(offset: target, whence: SEEK_SET) == target) + #expect(readExact(reader, 4096) == expected(at: target, count: 4096)) + } + + @Test("a chunk fetch 206 that starts elsewhere is refused", .timeLimit(.minutes(1))) + func chunkMisplacedRangeIsRefused() async throws { + let target: Int64 = 16 * 1024 * 1024 + let once = FirstRequestAt() + let respond: @Sendable (Int, Int64, String) -> ThrottledOriginServer.Directive = { _, offset, _ in + offset == target && once.claim() ? .serve206From(start: target - 4096) : .serve206 + } + let origin = ThrottledOriginServer(totalSize: 64 * 1024 * 1024, patternedBody: true, respond: respond) + let server = try #require(origin) + defer { server.stop() } + let reader = AVIOReader(url: URL(string: "http://127.0.0.1:\(server.port)/movie.bin")!, + chunkSize: 256 * 1024, prefetchEnabled: false) + defer { reader.markClosed(); reader.close() } + try reader.open() + + #expect(reader.seek(offset: target, whence: SEEK_SET) == target) + let got = readExact(reader, 4096) + // Refused and retried is fine, failed is fine; the misplaced bytes are not. + if let got { #expect(got == expected(at: target, count: 4096)) } + } + + @Test("Content-Range start is compared against the requested offset") + func misplacedStartIsNamed() { + let url = URL(string: "http://example.invalid/movie.bin")! + func response(_ status: Int, _ contentRange: String?) -> HTTPURLResponse { + HTTPURLResponse(url: url, statusCode: status, httpVersion: "HTTP/1.1", + headerFields: contentRange.map { ["Content-Range": $0] } ?? [:])! + } + #expect(AVIOReader.misplacedRangeStart(response(206, "bytes 100-199/1000"), requestedOffset: 100) == nil) + #expect(AVIOReader.misplacedRangeStart(response(206, "bytes 64-199/1000"), requestedOffset: 100) == 64) + #expect(AVIOReader.misplacedRangeStart(response(206, "bytes 100-199/*"), requestedOffset: 100) == nil) + #expect(AVIOReader.misplacedRangeStart(response(206, nil), requestedOffset: 100) == nil) + #expect(AVIOReader.misplacedRangeStart(response(200, "bytes 64-199/1000"), requestedOffset: 100) == nil) + } +} diff --git a/Tests/AetherEngineTests/RefreshableHLSAuthorizationTests.swift b/Tests/AetherEngineTests/RefreshableHLSAuthorizationTests.swift index 06800f903..8c40de0eb 100644 --- a/Tests/AetherEngineTests/RefreshableHLSAuthorizationTests.swift +++ b/Tests/AetherEngineTests/RefreshableHLSAuthorizationTests.swift @@ -149,7 +149,7 @@ struct RefreshableHLSAuthorizationTests { let server = HLSLocalServer(relay: relay) try server.start() defer { server.stop(); relay.stop() } - relay.admit(origin.url("/media.m3u8")) + relay.grantCredentials(to: origin.url("/media.m3u8")) let playlist = relay.rewritePlaylist("#EXTM3U\nhttp://localhost:\(origin.port)/foreign.ts\n", relativeTo: origin.url("/media.m3u8"), port: server.port, token: server.pathToken) let entry = try #require(URL(string: String(playlist.split(separator: "\n")[1]))) @@ -201,7 +201,7 @@ struct RefreshableHLSAuthorizationTests { let server = HLSLocalServer(relay: relay) try server.start() defer { server.stop(); relay.stop() } - relay.admit(origin.url("/media"), httpHeaders: ["X-Static-Secret": "never-forward"]) + relay.grantCredentials(to: origin.url("/media"), httpHeaders: ["X-Static-Secret": "never-forward"]) let entry = try #require(server.relayURL(for: origin.url("/media"))) _ = try await URLSession.shared.data(from: entry) await state.setToken("fresh") @@ -277,7 +277,7 @@ struct RefreshableHLSAuthorizationTests { "X-Emby-Authorization": "private", "X-MediaBrowser-Token": "private", "User-Agent": "SyntheticPlayer", "X-Transport": "preserved"] let relay = HLSOriginRelay() - relay.admit(origin.url(path), httpHeaders: headers) + relay.grantCredentials(to: origin.url(path), httpHeaders: headers) let server = HLSLocalServer(relay: relay) try server.start() defer { server.stop(); relay.stop() } @@ -306,7 +306,7 @@ struct RefreshableHLSAuthorizationTests { let (data, _) = try await URLSession.shared.data(from: entry) #expect(origin.requests.map { $0["authorization"] } == ["Bearer /redirect", "Bearer /final/media.m3u8"]) let line = try #require(String(decoding: data, as: UTF8.self).split(separator: "\n").first { !$0.hasPrefix("#") }) - #expect(HLSOriginRelay.originURL(fromQuery: URL(string: String(line))!.query!) == origin.url("/final/segment.ts")) + #expect(relay.originURL(fromQuery: URL(string: String(line))!.query!) == origin.url("/final/segment.ts")) } @Test("Stopping wakes a pending resolver and prevents its late result from sending") @@ -381,7 +381,7 @@ struct RefreshableHLSAuthorizationTests { let text = String(decoding: data, as: UTF8.self) let line = try #require(text.split(separator: "\n").first { !$0.hasPrefix("#") }) let segment = try #require(URL(string: String(line))) - #expect(HLSOriginRelay.originURL(fromQuery: segment.query ?? "") == origin.url("/final/segment.ts")) + #expect(relay.originURL(fromQuery: segment.query ?? "") == origin.url("/final/segment.ts")) } } diff --git a/Tests/AetherEngineTests/RemoteDiscSessionReleaseTests.swift b/Tests/AetherEngineTests/RemoteDiscSessionReleaseTests.swift new file mode 100644 index 000000000..3bdfe4f75 --- /dev/null +++ b/Tests/AetherEngineTests/RemoteDiscSessionReleaseTests.swift @@ -0,0 +1,164 @@ +import Testing +import Foundation +@testable import AetherEngine + +/// Audit NET-110 / DMX-110: the Demuxer builds the `HTTPDiscIOReader` for a remote disc image +/// itself, so it is the only party that can end that reader's `URLSession`. The disc adapter's +/// `ConcatIOReader.close()` is deliberately a no-op and the bridge does not own its reader, so a +/// demuxer that forgot the reader left one uninvalidated session (and its pooled keep-alive +/// socket) behind per open: 1000 opens measured as 1020 open sockets. +/// +/// The origin counts sockets that are still open, which is the one place the leak shows. +/// `.serialized`: the warm case goes through the process-wide prewarm store. +@Suite("Remote disc image sessions are released (audit NET-110)", .serialized) +struct RemoteDiscSessionReleaseTests { + + /// 0.5 s of 64x64 MPEG-2 video in an MPEG-PS pack stream (ffmpeg `-f vob`), 4096 bytes: the + /// smallest title the DVD adapter can hand to `mpegps` and have it open. + private static let vob = Data(base64Encoded: """ + AAABukQABAAEAYZmz/gAAAG7AAnDM2cAIf/g4OYAAAHgB92AwQ4xAAO3dREAA1+REGDm/wAAAbMEAEAS///gEAAAAbUUigAB + AAIAAAG4AAgAQAAAAQAAD//4AAABtY//80GAAAABARv4ffSQDMAzAoAPgKJAqBVIxGVtiSWkxCMangd7bXiwAvAC8AeABAAP + CaAPwB6TQMEwMCEIQ3JoFSb9xhCIRX+JYaW3bPttulPQhH+vvoBmAZgVAH4FUgVAqkanq2xIKQYhGMRgO9/4SANQQQEyYWMK + JYaTWxTkhAzd08ez5Ro+AKAgAfgg/jgGIDAAtAdgB8AaAFw1IDElLIZNALUlDCYTDEDcGFjQ3uSlBHe/LEIAbExGAKeBnjs+ + RgpfEk68OTAQAKwAwIQYUGIAGAA6+KQhG5YCYhBpfyM2GlhpSSy0j0pT0/r/e9TfPr9HACsAJQBSAYlhhZNAHoBqE8vjAwhh + pCDMczFExKEZnHXqggAUgCUAcgBaQwB+QwB2AxLATpQAVAYQWwGCYSwG43EwM2xRW4xDjMts9/pQEH9kAQgi/0ADcEn/MBiC + b/5eGJgA3AqVwLsG4c2Rw77Efe+nXz6/m4ALwAsAFgBqGBgYTQB0AZdHwbgLAUDAxOclEgsMQgtHUfvfJggAWACgAdACMmAD + 8mADsBATADVCMBTFcm5ywwlgNxvJgbviwz8Yh+V9+2vYAOwBsAxAGxNwYBXYJzlFoflFjxn+3dB332+EfX06/NIFQKEwBiTC + yYTSsWUXmSjmkXfH75e69fcAQPyAQAMADUEADIAZk0BMTQwvgJiaWGlgFwDv/F4CoDrJCMkvFFIAKwzdCAKlDEDEr2JSUf36 + QGgDRAA4DU8ML6QwmsM3JKBjvvzcE5uePvcAMgQPwSGCB+QAZAUQAhAdIIQCEAfDUAVDQxRCArgKjCUGlFHhiQChJLKWg7o6 + 20AAAAECG+qCB8cAE5NAMQBWGACorllAUAHBRQaTUd9iaAnAdhiA3Ftg3lFoDAKJKQGJ5SC0JLIeWj66t6IED8ghk0CoIH6w + AjSTQ0sNTgBuAZhmSWnp+KJgFSwwMK3T/xhQYhBRSDkIRtla87emAMQRABgSP/QAZXw4IAIgIn94JABIJn/V/P759fQb59fp + QAxBEAGBI/9ABlfrwIAIgIn94JABIJn/V/qm+fX0G+fX8igDEEQAYEj/0AGV+ZggAiAif3gkAEgmf9X8/qAQPpwAmJgAYgBE + AnIbFIAYgDYoNAMQKBqWwFiYV+A7JnSBgssYGkLf4pHRkkMomdKW1feBIZCJgDEmBpCIYYUTSYTSkhgaGBqMWUGlF5klFoTu + hKE7/JR83yvfpQAxBEAGBI/9ABlfrwIAIgIn94JABIJn/UAAAAEDG/qAIH5oIf7AAhBH/qAHoJf+pNvM2UED5IATAVAHwASg + AjTxoFABqTAwsM7IxKJoCYmBgFCaTGJaU9kYmBpL6CtwG4zOM/tV6QAz3BCAGVwEpgCq+IBABD+BC/vXgRACTQR/+r+f3z6+ + g3z6/SgBijghADK4CUzgPL9cBABD3BC/vVwRACTOCN/1f6pvn19Bvn1/I4AxSgEIAYJQAlNwDy/NAQAQ1AAjdQIYBJ6gRf+r + +f3z6+g3OBA+xAE4FABMCABYAO0k3DSwDEmFgDwChTpw0mgIQwsr8agovYDBMAyTQkZkgYJQ3dOv0kAZ/cAsBI/9ABlfrwIA + IagARuoEMAkEz/qAAAABBBusAJwEoffRgRP9k35aDQBYANiGATgFwBkGgj/+gMCEkjle+lAGYAUADgA1ISSkAJgHQFC+tBDJ + hCDEFoTjnGbDUD0frFdvfFgDAAJyYGAUJqMAmDQ1JSM2JYZhv7q3+UcbfWAIA0mAgAYgVALigDEh8sMAdlpQkhkxCfiEgrlb + lpKcaXsS3ZC23a/bgGADgOiBW82A6DQB8XlEwChM5u5RSMYd2bkcVfPgwAZACwB2GFE0MAMAB4TQwNQgsotADoB0WUWhOdCU + JxXyUfL3yvr7sED8wED8TEIEIAcBiAKAC8BMX+nAFPDMh8OTsMS5gz+/JwBUCB+akhJIYCABCAKADEAwJoDAlb4h8M2DRo1Y + YnYYnYYv7/L1/RQAmASB186BEAKRfo4MAHIBeQgCYA1AThgI4AYFQ1BGTr5qTCGALyaEAB6AalIwBKQyGhGDulYdvagBOCAC + uA7AMhgDsBiAIQB4AwDAlLk1BDQpDOzpdJJytv7sALwBmAYABaQgDUhhoDECgAyDCFkflgIQ1GycWNJGR90jGO2/dOvzwBiA + 5D4AUvcADkAMQEIA/JuShKQGBSMyCgwvu5vLLwzp45W2x2P/vouAC8AV44AbgD3Akf6gOwTf/L0gAsAHwAJgKl8ArQBVAZ0B + OQlA13XsleONv0EARggAnADMBAGAJgHYCcB2AwAYE0A1wangMAkNSBTlDALFhoZwMJSxeclPt3z8+AAAAQAAV//7gAAAAbWB + H/NBgAAAAQESgcgB5+6svDu2fIQjJCe5ZaISYoxz17Y5ZO2dKfyuWX4ghx36svj+2fpSnpCcxZaYGD/80/q2Y18hlwGzFmMe + rKyRIwICSRUSErdh7Djl7vuNP6N0czwUo47Cl/i9jdxq3JeT0+Hr7CsPzZWUphuxrfv+8PcdxCiI4dwg8vlb1EieFgdgGZh4 + 5R7OIbDHeDws7iVMa3zdCcwmJxPEiV9ZJzDAn/I2qBoAAAG6RAAEAAwBhmbP+AAAAeAHXYDACzEABWc9EQAFD1n/DgrqZRqj + /1jwhIw8suEdeFnMfl4XsbjkBKC0b6IhOJzGDs/6UJbhCfD+FtzGZQ7ncdj9soamPDx++WpYs8WEfgS3gAAAAQISjAjHB+3U + avnOrfYZzu6enwk8L/PJzc3GcYMQ6EaA0RarA0RZoyHCTHAmztx34HnZk5CCk+AWMzdBwhtvxpJ/26P9AaIsAAABAxKIZBhJ + 748KUpfZnW+HYbmgFsBodVYGiLU0GP/+Bi//qPkdXFOQGYax3MXng77iFwDNQ+AAAAEEEymAU1hgDjAI4A1AHAYUWAOwB4GF + hhRZRYBiAgQkpKEJKLRwHYZuhKE75O5333cdAMQGBMwCHBoA0UTOAOQQgDCYAxDEEoB2AXAOyG5CAdAIQHReGFpwYWGjQKgM + E9BQCdGGAKAKgBsAZkKTiwxKeGJ+IWyMXy07krAOTz2AcEk6DQHYaTADQBMGAD8B2WA2AMgA/LKAQgVb8ANAEADrAMCwDUAv + SUxLJoYAmIQDvgO0gUITkMNQQg1BMQWQ0lI9TAHYDsAmARkSJmPD7d8yXOJCQ91qzP2xIdv1amwePw5XfGZxubvz8lH+8AmD + QHXT3G5D4YPHc8cyXc77fwFQ0m4N7slCsSRhrjj9nUMcknrXZTAKwGSTw6AMgBqQyyEANgB2Q8TUkzkwAuANS8QxqSuTCUA6 + DeS8XuEDGGjTGeBCAILALy8APiEGpAoQwDVgEyAE+cEIAksB25MDAHY1HQlIDZPGjCYWS3YkO8GlAOi+MJvbBn4aNJpYFgwr + EwBtw1PT22GANynyP+vbIdKN4mFcmAFpDAD8m4AyANADQB0TADUAzAHpN34DoBAAmGkMAPwHaQMYBunAICiEA3JoYTCWTQ0l + hhRMK5ZaEepgDoB0ATgIiLE3nB0rO6MFPj2WrZxqca7KwDnsiXx/DkwBqUAxyMxK/X2ZW+UBMx3Y7/N2x0QiYTOUhaRfGOpe + Pc85z2HK2QmAAAABAACX//uAAAABtYEf80GAAAABARKsH0AGokYDn//HAw//wMoAMuqxFTiIDR//E8G0AGAAAAECEnTZOgAA + AQMSjwP//89gA/qauVVYGiLXQYv/4AAAAQQTKYBSSg7CCJAGoA2DAwsAdgD4ospBZRYBiAmKSUnDUJRwKhmShO6vknffNx0A + nAdEwANwMgDbgKAQf7WAQEwBCjAhf8AFwDEmjSEA6AMwKBqCUWCEAOWTXIYCYllBgCcmBDkIBigBAAhATW5SS+UWXiYhCMXy + al+VnJZ4xYCQ9mHqiwHYaGAGgCAmAF4DsNxMAQAB+WGAGpDX+AYAJgHQAeFgIQC9KEBJDKATAUAd8B2khEwEL/kNQBQNQTMk + mpKKTUwNATgJBBFsBgCH/yTAFenCT1sc65ldSmOZXwfjYAySAmSksaS8yjBQVFkMsDPL5z5icxA4iymAVkJJKRPgDAAak3kI + AcADsm4N5M5CALgDUNwbi+UBjlAOg1IGcW+GdDDRrqZ4oAPA0AvLwA8JgaWBQhgGYIQBRQDHI/AD4sBOSyYGAVcaGdPJha2J + ieTd+VwLZJLiGGAOi+jBr5AYApyQ1PGBgZiYA24aWktLIZGJpSiStCnSh4MKAwA3IeAYk3AGQBoAgAoGAIQDUAdk1BfAdAJg + EA0hgB+BUslbll4A0DCYA3DSiYNJqcWGFEwrhpaMmpgYAmASiSJQDEEMAcmgKvWCCQEDzHBg//nXmSE4dzgqANUAMchAwZ+L + fCzIohFANsVvxozucLcDkAAAAQAA1//7gAAAAbWBH/NBgAAAAQESepB///m0AbQAAAECEnTZPgAAAQMSlgD6AXVWeBuaaeOi + PAAAAQQTKYQkEnB0GwBmAOCiiwB2APiiyiklJAMQECEoShHQlCQKhmShO+b7nfdXgEIDoMADXFgDbpAwCD/awCAmAGpWYBiA + XAMSaNJgDAA1AolAwN+AoWTRpDAYEtAYAnJgQNIQDEoBAAagJrQkosslFlgNshBLSGp36AkaeMf7AWPWziYNAqWGAIAEAYAP + wHZeJgBoAH4aUAhAq3/AMgEADoAPCwEIBelG3SQygEwFAHfAd8hEwEL/oNKIRaCFkk3hhSamAB+CH/34CoBcCH/yCQAGkj6d + /1rFnkc6GflBKAjrXn3J3cnnk+ANAA8SN6RuJLgfeSQ/iaBlsxLc8XZTCGk7iEwYAbQBgANSakmADYAdk0BvwwDABcAahoDf + FjOGEoB0GpJZLV0DGca+ZnkAB4WAXpwA+ISUgUJoBnwA6KAY5HYBgWAnJZMDAKvuUNAwWsYTCwM7krsjJJcTQwCiegDJwZ0h + o0NSEBhWJmDOGlpL7ZkYhlPt3WjN+lDxMDOGAFpDAD0m4AyANAEADooA1AMwB2TUJ4DABAAgGk0APwHZZKAunAIAwhANw0MJ + mSTUksMKJhXDS0b1MAD4EMAXQFAC8EMAcEj/1BG8yhQrE86fZKvt8O+cDkAa4BOglDCV1IaQTMBgDDtxjOLYcJgAAAG+AIn/ + //////////////////////////////////////////////////////////////////////////////////////////////// + /////////////////////////////////////////////////////////////////////////////////////w== + """, options: .ignoreUnknownCharacters)! + + private static func isoImage(vob: Data) -> Data { + let sector = ISO9660Fixture.sectorSize + var image = ISO9660Fixture.make(files: [ + .init(name: "VTS_01_1.VOB", length: vob.count, content: [UInt8](vob.prefix(sector))), + ]) + image.append(vob.dropFirst(sector)) + return image + } + + private func url(_ origin: KeepAliveRangeOrigin, _ name: String = "disc.iso") -> URL { + URL(string: "http://127.0.0.1:\(origin.port)/\(name)")! + } + + private func drained(_ origin: KeepAliveRangeOrigin) async throws -> Bool { + try await waitFor(upTo: .seconds(10)) { origin.openConnectionCount == 0 } + } + + @Test("a disc image that opens releases its session when the demuxer closes") + func openedDiscReleasesOnClose() async throws { + let origin = try #require(KeepAliveRangeOrigin(data: Self.isoImage(vob: Self.vob))) + defer { origin.stop() } + + let demuxer = Demuxer() + try demuxer.open(url: url(origin)) + #expect(demuxer.isDiscSource, "the fixture was not taken for a disc") + #expect(origin.openConnectionCount > 0, "the open never reached the origin") + demuxer.close() + + #expect(try await drained(origin), "sockets left open: \(origin.openConnectionCount)") + } + + @Test("a disc image whose open throws releases its session") + func failedOpenReleasesItsSession() async throws { + // A pack-start-code-only title: recognised as a disc, and mpegps then finds no stream. + let image = ISO9660Fixture.make(files: [.init(name: "VTS_01_1.VOB", length: 2048)]) + let origin = try #require(KeepAliveRangeOrigin(data: image)) + defer { origin.stop() } + + for _ in 0..<3 { + let demuxer = Demuxer() + #expect(throws: (any Error).self) { try demuxer.open(url: url(origin)) } + demuxer.close() + } + #expect(origin.acceptedConnectionCount > 0, "the open never reached the origin") + + #expect(try await drained(origin), "sockets left open: \(origin.openConnectionCount)") + } + + /// The root directory's length, past the reader's extent cap: `wrap` throws `malformed` instead of + /// reporting "not a disc". + private static func unparseableImage() -> Data { + var image = ISO9660Fixture.make(files: [.init(name: "VTS_01_1.VOB", length: 2048)]) + let lengthOffset = 16 * ISO9660Fixture.sectorSize + 156 + 10 + image.replaceSubrange(lengthOffset..<(lengthOffset + 4), with: ISO9660Fixture.le32(16 * 1024 * 1024)) + return image + } + + @Test("a disc image whose structure cannot be parsed releases its session") + func throwingWrapReleasesItsSession() async throws { + let origin = try #require(KeepAliveRangeOrigin(data: Self.unparseableImage())) + defer { origin.stop() } + + let demuxer = Demuxer() + #expect(throws: (any Error).self) { try demuxer.open(url: url(origin)) } + demuxer.close() + + #expect(origin.acceptedConnectionCount > 0, "the open never reached the origin") + #expect(try await drained(origin), "sockets left open: \(origin.openConnectionCount)") + } + + @Test("a disc image whose structure cannot be parsed hands the warm back") + func throwingWrapReturnsTheWarm() async throws { + let origin = try #require(KeepAliveRangeOrigin(data: Self.unparseableImage())) + defer { origin.stop() } + let source = url(origin, "corrupt.iso") + _ = await SourcePrewarmFetcher.warm(url: source, extraHeaders: [:], byteBudget: 1 << 20, into: .shared) + #expect(SourcePrewarmStore.shared.isWarm(for: source), "the warm was not taken") + defer { _ = SourcePrewarmStore.shared.take(for: source) } + + let demuxer = Demuxer() + defer { demuxer.close() } + #expect(throws: (any Error).self) { try demuxer.open(url: source) } + + #expect(SourcePrewarmStore.shared.isWarm(for: source), + "a throwing wrap dropped the warm instead of putting it back") + } +} diff --git a/Tests/AetherEngineTests/RemoteHLSCueClockTests.swift b/Tests/AetherEngineTests/RemoteHLSCueClockTests.swift new file mode 100644 index 000000000..3505b149b --- /dev/null +++ b/Tests/AetherEngineTests/RemoteHLSCueClockTests.swift @@ -0,0 +1,230 @@ +import Testing +import Foundation +@testable import AetherEngine + +/// AE#616: on `nativeRemoteHLS`, a Jellyfin transcode restarted at the keyframe before a slot makes +/// AVPlayer's item time lead the picture. The injected #316 renditions are placed by media timestamp, +/// so a line presented at item time T whose cue starts at S measures the lead T - S. Values are the +/// reporter's measurements (seg 494: slot 1483.482, keyframe 1482.272, lead 1.209). +struct RemoteHLSCueClockTests { + + private func clock(_ cues: [(Double, String)]) -> RemoteHLSCueClock { + var clock = RemoteHLSCueClock() + clock.setCues(cues.map { (start: $0.0, text: $0.1) }) + clock.noteTimeJump() + // The landing delivery after the jump teaches nothing by design. + _ = clock.observe(strings: [], itemTime: 0) + return clock + } + + @Test("A presented line measures item time minus its cue start") + func measuresLead() throws { + var c = clock([(1490.0, "Where were you?"), (1493.5, "Out.")]) + let measured = c.observe(strings: ["Where were you?"], itemTime: 1491.209) + let lead = try #require(measured) + #expect(abs(lead - 1.209) < 1e-9) + #expect(c.offset == lead) + } + + @Test("Each new line re-measures, so a seek that moved the anchor is corrected on the next line") + func remeasuresAfterSeek() throws { + var c = clock([(1490.0, "Where were you?"), (1800.0, "Run.")]) + _ = c.observe(strings: ["Where were you?"], itemTime: 1491.209) + c.noteTimeJump() + _ = c.observe(strings: [], itemTime: 1790.0) + let measured = c.observe(strings: ["Run."], itemTime: 1803.086) + let lead = try #require(measured) + #expect(abs(lead - 3.086) < 1e-9) + } + + @Test("The delivery at a seek landing is skipped: a line active there is stamped with the landing, not its start") + func skipsLandingDelivery() { + var c = RemoteHLSCueClock() + c.setCues([(start: 100.0, text: "Long line")]) + c.noteTimeJump() + let measured = c.observe(strings: ["Long line"], itemTime: 103.5) + #expect(measured == nil) + #expect(c.offset == nil) + } + + @Test("A line ending measures nothing; the survivor was stamped when it appeared") + func endingLineMeasuresNothing() { + var c = clock([(10.0, "A"), (11.0, "B")]) + _ = c.observe(strings: ["A"], itemTime: 12.0) + _ = c.observe(strings: ["A", "B"], itemTime: 13.0) + let measured = c.observe(strings: ["B"], itemTime: 14.0) + #expect(measured == nil) + #expect(c.offset == 2.0) + } + + @Test("Repeated text resolves to the start nearest the current lead") + func repeatedTextPicksNearestToCurrentLead() throws { + var c = clock([(20.0, "First"), (100.0, "Yeah."), (104.0, "Yeah.")]) + _ = c.observe(strings: ["First"], itemTime: 23.0) // lead 3 + _ = c.observe(strings: [], itemTime: 24.0) + // Candidates 107 - 100 = 7 and 107 - 104 = 3; the current lead is 3. + let measured = c.observe(strings: ["Yeah."], itemTime: 107.0) + let lead = try #require(measured) + #expect(lead == 3.0) + } + + @Test("A text with one plausible start wins over an ambiguous sibling in the same delivery") + func uniqueCandidateWins() throws { + var c = clock([(50.0, "Yeah."), (56.5, "Yeah."), (52.0, "Only once")]) + let measured = c.observe(strings: ["Yeah.", "Only once"], itemTime: 57.5) + let lead = try #require(measured) + #expect(lead == 5.5) + } + + @Test("A match outside the plausible band is a different line, not a measurement") + func implausibleMatchIgnored() { + var c = clock([(10.0, "Hello")]) + let measured = c.observe(strings: ["Hello"], itemTime: 500.0) + #expect(measured == nil) + #expect(c.offset == nil) + } + + @Test("Text AVPlayer hands back matches the sanitized cue the .vtt carried") + func normalizationMatchesServedText() throws { + var c = clock([(30.0, "{\\an8}Line one\\Nline two")]) + let measured = c.observe(strings: ["Line one\nline two"], itemTime: 31.25) + let lead = try #require(measured) + #expect(lead == 1.25) + } + + @Test("Unknown text measures nothing") + func unknownText() { + var c = clock([(30.0, "Known")]) + let measured = c.observe(strings: ["Origin's own rendition"], itemTime: 31.0) + #expect(measured == nil) + } + + @Test("A time jump marks the kept offset as unmeasured until the next matched line") + func timeJumpClearsMeasuredUntilNextLine() { + var c = clock([(1490.0, "Where were you?"), (1800.0, "Run.")]) + #expect(!c.isMeasuredSinceJump) + _ = c.observe(strings: ["Where were you?"], itemTime: 1491.209) + #expect(c.isMeasuredSinceJump) + c.noteTimeJump() + #expect(!c.isMeasuredSinceJump) + #expect(c.offset.map { abs($0 - 1.209) < 1e-9 } == true) + _ = c.observe(strings: [], itemTime: 1790.0) + #expect(!c.isMeasuredSinceJump) + _ = c.observe(strings: ["Run."], itemTime: 1803.086) + #expect(c.isMeasuredSinceJump) + } + + // MARK: - Markup stripping + + /// The `normalize` that removed one tag at a time, searching again from the start, kept as the oracle. + private func legacyNormalize(_ text: String) -> String { + var s = MovTextSampleBuilder.sanitize(text) + while let open = s.firstIndex(of: "<"), let close = s[open...].firstIndex(of: ">") { + s.removeSubrange(open...close) + } + s = s.replacingOccurrences(of: "<", with: "<") + .replacingOccurrences(of: ">", with: ">") + .replacingOccurrences(of: " ", with: " ") + .replacingOccurrences(of: "&", with: "&") + return s.split(whereSeparator: { $0.isWhitespace || $0.isNewline }).joined(separator: " ") + } + + private struct SplitMix64: RandomNumberGenerator { + var state: UInt64 + mutating func next() -> UInt64 { + state &+= 0x9E37_79B9_7F4A_7C15 + var z = state + z = (z ^ (z >> 30)) &* 0xBF58_476D_1CE4_E5B9 + z = (z ^ (z >> 27)) &* 0x94D0_49BB_1331_11EB + return z ^ (z >> 31) + } + } + + private static let trickyLines: [String] = [ + "", + "plain", + "Line one\nline two", + "<>x", + "<><>", + "a<>b<>c", + "c>", + "<>", + "ab", + "><", + "acx<", + "<", + ">", + "open only", + "close only", + "nested", + "<i>escaped</i>", + "<>", + "&lt;", + "&lt;", + "a  b", + " ;", + "Hi & bye", + "{\\an8}top\\Nnext", + "{}x", + "<{b}>x", + "<\u{0301}b>x", + "x>", + "\u{0600}x", + "e\u{0301}\u{0301}x", + "x\u{0301}y>", + "\u{1F1FA}\u{1F1F8}\u{1F1EC}\u{1F1E7}", + "\u{1F468}\u{200D}\u{1F469}", + "\u{1100}\u{1161}", + "\r\nx", + "\u{226E}b>x", + "a\u{00A0}c", + " spaced ", + ] + + @Test("The forward strip matches the old tag-at-a-time strip on tricky lines") + func stripMatchesLegacyOnTrickyLines() { + for line in Self.trickyLines { + let got = RemoteHLSCueClock.normalize(line) + let want = legacyNormalize(line) + #expect(got == want, "line: \(line.debugDescription)") + #expect(Array(got.unicodeScalars) == Array(want.unicodeScalars), "line: \(line.debugDescription)") + } + } + + @Test("The forward strip matches the old tag-at-a-time strip on generated lines full of markup") + func stripMatchesLegacyOnGeneratedLines() { + let alphabet: [String] = ["<", ">", "<", ">", "", "", "<>", "<", ">", "&", " ", + "&", "lt;", "a", " ", "\n", "\r", "{", "}", "\\N", "\u{0301}", "\u{0600}", + "\u{200D}", "\u{1F468}", "\u{1F1FA}", "\u{1F1F8}", "\u{1100}", "\u{1161}", + "\u{226E}", "\u{00E9}"] + var rng = SplitMix64(state: 0x0616_C0E5) + for _ in 0..<6000 { + let length = Int.random(in: 0...30, using: &rng) + var line = "" + for _ in 0..a", count: 8 * 1024) + #expect(hostile.utf8.count == 64 * 1024) + // The fastest of a few runs, so one preemption on a loaded machine is not read as the cost. + // About 50 ms in a debug build here against 1.1 s for the rescanning strip; the bound leaves a + // loaded CI runner room without letting a quadratic strip back in. + var fastest = Duration.seconds(3600) + for _ in 0..<3 where fastest >= .milliseconds(400) { + let started = ContinuousClock.now + let normalized = RemoteHLSCueClock.normalize(hostile) + fastest = min(fastest, ContinuousClock.now - started) + #expect(normalized == String(repeating: "a", count: 8 * 1024)) + } + #expect(fastest < .milliseconds(400), "took \(fastest)") + } +} diff --git a/Tests/AetherEngineTests/RemoteHLSMasterRewriteTests.swift b/Tests/AetherEngineTests/RemoteHLSMasterRewriteTests.swift index b15b847b7..b48fd5ad1 100644 --- a/Tests/AetherEngineTests/RemoteHLSMasterRewriteTests.swift +++ b/Tests/AetherEngineTests/RemoteHLSMasterRewriteTests.swift @@ -245,4 +245,62 @@ struct RemoteHLSMasterRewriteTests { originPlaylist: master, originURL: Self.origin, renditions: [Self.english]) #expect(Self.lines(out).contains("https://cdn.test/other/main.m3u8")) } + + // MARK: - Declared names (audit NAT-2) + + private static func declaredNames(in master: String, uri: String) -> [String] { + lines(master).filter { $0.contains("URI=\"\(uri)\"") } + .compactMap { HLSPlaylistParser.attribute("NAME", in: $0) } + } + + /// The engine selects an injected rendition by NAME and hides it from the legible list by NAME. + /// It used to key on the name the track asked for, so a disambiguated sidecar selected the + /// origin's rendition, and the origin's own track disappeared from the list. + @Test("The names handed back are exactly the ones the master declares, one per rendition in every group") + func declaredNamesMatchTheMaster() throws { + let out = try RemoteHLSMasterRewrite.rewriteDeclaringNames( + originPlaylist: Self.masterMixedGroups, originURL: Self.origin, + renditions: [Self.english, + RemoteHLSMasterRewrite.Rendition(ordinal: 1, name: "English", language: "en"), + RemoteHLSMasterRewrite.Rendition(ordinal: 2, name: "The \"Cut\"")]) + + #expect(out.renditionNames == ["English 2", "English 3", "The 'Cut'"]) + for (ordinal, name) in out.renditionNames.enumerated() { + let declared = Self.declaredNames(in: out.master, uri: "subs_\(ordinal).m3u8") + #expect(declared == [name, name], "one NAME for the rendition in both groups") + } + } + + @Test("A wrapped media playlist hands back its names too") + func wrappedPlaylistDeclaresNames() throws { + let media = "#EXTM3U\n#EXT-X-TARGETDURATION:6\n#EXTINF:6,\nseg0.ts\n#EXT-X-ENDLIST\n" + let out = try RemoteHLSMasterRewrite.rewriteDeclaringNames( + originPlaylist: media, originURL: Self.origin, + renditions: [Self.english, + RemoteHLSMasterRewrite.Rendition(ordinal: 1, name: "English", language: "en")]) + #expect(out.renditionNames == ["English", "English 2"]) + #expect(Self.declaredNames(in: out.master, uri: "subs_1.m3u8") == ["English 2"]) + } + + // MARK: - Control characters (audit NAT-6) + + /// A sidecar name is host input, typically a file name or server metadata. A line break in it + /// ended the tag, and the rest of the name wrote master lines of its own, a variant included. + @Test("A line break in a name or language cannot write a line into the master") + func lineBreaksCannotInjectLines() throws { + let hostile = "x\r\n#EXT-X-STREAM-INF:BANDWIDTH=1\r\nhttp://attacker.test/v.m3u8\n#" + let out = try RemoteHLSMasterRewrite.rewriteDeclaringNames( + originPlaylist: Self.master, originURL: Self.origin, + renditions: [RemoteHLSMasterRewrite.Rendition( + ordinal: 0, name: hostile, language: "en\u{2028}#EXT-X-ENDLIST\u{85}x")]) + + let all = Self.lines(out.master) + #expect(all.filter { $0.hasPrefix("#EXT-X-STREAM-INF:") }.count == 1) + #expect(!all.contains { $0.hasPrefix("http://attacker.test") }) + #expect(!all.contains("#EXT-X-ENDLIST")) + #expect(all.filter { $0.hasPrefix("#EXT-X-MEDIA:") }.count == 1) + let name = try #require(out.renditionNames.first) + #expect(!name.unicodeScalars.contains { $0.properties.generalCategory == .control }) + #expect(Self.declaredNames(in: out.master, uri: "subs_0.m3u8") == [name]) + } } diff --git a/Tests/AetherEngineTests/RemoteHLSStreamDescriptionTests.swift b/Tests/AetherEngineTests/RemoteHLSStreamDescriptionTests.swift new file mode 100644 index 000000000..b4e03a103 --- /dev/null +++ b/Tests/AetherEngineTests/RemoteHLSStreamDescriptionTests.swift @@ -0,0 +1,284 @@ +import Foundation +import Testing +import CoreMedia +import AudioToolbox +@testable import AetherEngine + +/// The `nativeRemoteHLS` bypass runs no libav probe, so a stats panel had nothing but the host's own +/// metadata of the ORIGINAL file to show: a capped 1280x720 H.264 transcode read "3840x2160, Main 10". +/// AVPlayer's parsed item tracks already carry the delivered stream, so these pin the mapping from its +/// format descriptions onto the fields the probe path publishes, in the probe path's vocabulary. +@Suite("RemoteHLSStreamDescription") +struct RemoteHLSStreamDescriptionTests { + + // MARK: - Fixtures + + private static let avc1: FourCharCode = 0x61766331 + private static let hvc1: FourCharCode = 0x68766331 + + /// avcC with no SPS/PPS: version, profile_idc, constraint flags, level, lengthSizeMinusOne, 0 SPS, 0 PPS. + private static func avcC(profileIDC: UInt8, compatibility: UInt8 = 0, highExtension: (chroma: UInt8, lumaDepth: UInt8)? = nil) -> Data { + var bytes: [UInt8] = [1, profileIDC, compatibility, 31, 0xFF, 0xE0, 0x00] + if let ext = highExtension { + bytes += [0xFC | ext.chroma, 0xF8 | (ext.lumaDepth - 8), 0xF8 | (ext.lumaDepth - 8), 0x00] + } + return Data(bytes) + } + + /// hvcC header (23 bytes) with no parameter-set arrays. + private static func hvcC(profileIDC: UInt8, chroma: UInt8 = 1, lumaDepth: UInt8 = 8) -> Data { + var bytes = [UInt8](repeating: 0, count: 23) + bytes[0] = 1 + bytes[1] = profileIDC & 0x1F + bytes[12] = 150 + bytes[16] = 0xFC | chroma + bytes[17] = 0xF8 | (lumaDepth - 8) + bytes[18] = 0xF8 | (lumaDepth - 8) + return Data(bytes) + } + + private static func videoDescription( + subType: FourCharCode, width: Int32, height: Int32, + extensions: [CFString: Any] + ) throws -> CMFormatDescription { + var desc: CMFormatDescription? + let status = CMVideoFormatDescriptionCreate( + allocator: kCFAllocatorDefault, codecType: subType, width: width, height: height, + extensions: extensions as CFDictionary, formatDescriptionOut: &desc) + try #require(status == noErr) + return try #require(desc) + } + + private static func audioDescription(formatID: AudioFormatID, sampleRate: Double, channels: UInt32, + bitsPerChannel: UInt32 = 0, cookie: Data? = nil) throws -> CMFormatDescription { + var asbd = AudioStreamBasicDescription( + mSampleRate: sampleRate, mFormatID: formatID, mFormatFlags: 0, mBytesPerPacket: 0, + mFramesPerPacket: 1536, mBytesPerFrame: 0, mChannelsPerFrame: channels, + mBitsPerChannel: bitsPerChannel, mReserved: 0) + var desc: CMFormatDescription? + let status: OSStatus + if let cookie { + status = cookie.withUnsafeBytes { raw in + CMAudioFormatDescriptionCreate( + allocator: kCFAllocatorDefault, asbd: &asbd, layoutSize: 0, layout: nil, + magicCookieSize: cookie.count, magicCookie: raw.baseAddress, extensions: nil, + formatDescriptionOut: &desc) + } + } else { + status = CMAudioFormatDescriptionCreate( + allocator: kCFAllocatorDefault, asbd: &asbd, layoutSize: 0, layout: nil, + magicCookieSize: 0, magicCookie: nil, extensions: nil, formatDescriptionOut: &desc) + } + try #require(status == noErr) + return try #require(desc) + } + + /// dec3 payload for one independent substream without dependents, JOC flag as given. + private static func dec3(joc: Bool) -> Data { + // data_rate(13)=640 num_ind_sub(3)=0 | fscod bsid reserved asvc bsmod acmod lfeon reserved + // num_dep_sub(4)=0 reserved(1) | reserved(7) flag_ec3_extension_type_a(1) | complexity_index(8) + Data([0x50, 0x00, 0x20, 0x0F, 0x00, joc ? 0x01 : 0x00, joc ? 16 : 0]) + } + + // MARK: - Delivered video + + @Test("the device case: a 720p H.264 High transcode reads as what was delivered") + func deliveredH264Transcode() throws { + let desc = try Self.videoDescription( + subType: Self.avc1, width: 1280, height: 720, + extensions: [ + kCMFormatDescriptionExtension_ColorPrimaries: kCMFormatDescriptionColorPrimaries_ITU_R_709_2, + kCMFormatDescriptionExtension_TransferFunction: kCMFormatDescriptionTransferFunction_ITU_R_709_2, + kCMFormatDescriptionExtension_YCbCrMatrix: kCMFormatDescriptionYCbCrMatrix_ITU_R_709_2, + kCMFormatDescriptionExtension_FullRangeVideo: false, + kCMFormatDescriptionExtension_SampleDescriptionExtensionAtoms: ["avcC": Self.avcC(profileIDC: 100)], + ]) + let video = try #require(RemoteHLSStreamDescription.video(from: desc)) + #expect(video.width == 1280) + #expect(video.height == 720) + #expect(video.codecName == "h264") + #expect(video.format == VideoStreamFormat( + pixelFormat: "yuv420p", bitDepth: 8, colorPrimaries: "bt709", transfer: "bt709", + matrix: "bt709", range: "tv", profile: "High")) + } + + @Test("HEVC Main 10 PQ reads its profile, depth and BT.2020 description from hvcC and the extensions") + func hevcMain10PQ() throws { + let desc = try Self.videoDescription( + subType: Self.hvc1, width: 3840, height: 2160, + extensions: [ + kCMFormatDescriptionExtension_ColorPrimaries: kCMFormatDescriptionColorPrimaries_ITU_R_2020, + kCMFormatDescriptionExtension_TransferFunction: kCMFormatDescriptionTransferFunction_SMPTE_ST_2084_PQ, + kCMFormatDescriptionExtension_YCbCrMatrix: kCMFormatDescriptionYCbCrMatrix_ITU_R_2020, + kCMFormatDescriptionExtension_SampleDescriptionExtensionAtoms: ["hvcC": Self.hvcC(profileIDC: 2, lumaDepth: 10)], + ]) + let video = try #require(RemoteHLSStreamDescription.video(from: desc)) + #expect(video.codecName == "hevc") + #expect(video.format.pixelFormat == "yuv420p10le") + #expect(video.format.bitDepth == 10) + #expect(video.format.profile == "Main 10") + #expect(video.format.colorPrimaries == "bt2020") + #expect(video.format.transfer == "smpte2084") + #expect(video.format.matrix == "bt2020nc") + #expect(video.format.range == nil) + } + + /// Same rule as AE#658 on the probe path: an untagged stream is not reported as BT.709. + @Test("an untagged stream keeps its gaps") + func untaggedStaysNil() throws { + let desc = try Self.videoDescription(subType: Self.avc1, width: 640, height: 360, extensions: [:]) + let video = try #require(RemoteHLSStreamDescription.video(from: desc)) + #expect(video.format.colorPrimaries == nil) + #expect(video.format.transfer == nil) + #expect(video.format.matrix == nil) + #expect(video.format.range == nil) + #expect(video.format.profile == nil) + #expect(video.format.pixelFormat == nil) + #expect(video.format.bitDepth == nil) + } + + @Test("CoreMedia colour names map onto libav's") + func colourNames() { + #expect(RemoteHLSStreamDescription.primariesName(kCMFormatDescriptionColorPrimaries_P3_D65 as String) == "smpte432") + #expect(RemoteHLSStreamDescription.primariesName(kCMFormatDescriptionColorPrimaries_DCI_P3 as String) == "smpte431") + #expect(RemoteHLSStreamDescription.primariesName(kCMFormatDescriptionColorPrimaries_EBU_3213 as String) == "bt470bg") + #expect(RemoteHLSStreamDescription.primariesName(kCMFormatDescriptionColorPrimaries_SMPTE_C as String) == "smpte170m") + #expect(RemoteHLSStreamDescription.transferName(kCMFormatDescriptionTransferFunction_ITU_R_2100_HLG as String) == "arib-std-b67") + #expect(RemoteHLSStreamDescription.transferName(kCMFormatDescriptionTransferFunction_ITU_R_2020 as String) == "bt2020-10") + #expect(RemoteHLSStreamDescription.transferName(kCMFormatDescriptionTransferFunction_Linear as String) == "linear") + #expect(RemoteHLSStreamDescription.transferName(kCMFormatDescriptionTransferFunction_sRGB as String) == "iec61966-2-1") + #expect(RemoteHLSStreamDescription.matrixName(kCMFormatDescriptionYCbCrMatrix_ITU_R_601_4 as String) == "smpte170m") + #expect(RemoteHLSStreamDescription.matrixName(kCMFormatDescriptionYCbCrMatrix_SMPTE_240M_1995 as String) == "smpte240m") + // A name CoreMedia adds later passes through rather than vanishing. + #expect(RemoteHLSStreamDescription.primariesName("Future_Primaries") == "Future_Primaries") + } + + @Test("constraint flags in avcC reach the profile name") + func constrainedBaseline() { + let config = RemoteHLSStreamDescription.avcConfiguration(Self.avcC(profileIDC: 66, compatibility: 0x40)) + #expect(config?.profileName == "Constrained Baseline") + let high10 = RemoteHLSStreamDescription.avcConfiguration( + Self.avcC(profileIDC: 110, highExtension: (chroma: 1, lumaDepth: 10))) + #expect(high10?.profileName == "High 10") + #expect(high10?.pixelFormat == "yuv420p10le") + #expect(high10?.bitDepth == 10) + } + + @Test("a truncated configuration record yields nothing rather than a guess") + func truncatedRecords() { + #expect(RemoteHLSStreamDescription.avcConfiguration(Data([1, 100])) == nil) + #expect(RemoteHLSStreamDescription.hevcConfiguration(Data([1, 2, 0, 0])) == nil) + } + + @Test("an audio description is not a video one") + func audioIsNotVideo() throws { + let desc = try Self.audioDescription(formatID: kAudioFormatEnhancedAC3, sampleRate: 48_000, channels: 6) + #expect(RemoteHLSStreamDescription.video(from: desc) == nil) + } + + // MARK: - Audio tracks + + @Test("the device case: E-AC-3 5.1 at 48 kHz becomes an eac3 TrackInfo") + func eac3Track() throws { + let desc = try Self.audioDescription(formatID: kAudioFormatEnhancedAC3, sampleRate: 48_000, channels: 6) + let reading = try #require(RemoteHLSStreamDescription.audioReading(from: desc, isEnabled: true, language: "eng")) + let (tracks, active) = RemoteHLSStreamDescription.audioTracks([reading]) + let track = try #require(tracks.first) + #expect(tracks.count == 1) + #expect(track.id == RemoteHLSStreamDescription.audioTrackIDBase) + #expect(active == track.id) + #expect(track.codec == "eac3") + #expect(track.channels == 6) + #expect(track.sampleRate == 48_000) + #expect(track.bitsPerSample == 0) + #expect(track.language == "eng") + #expect(track.name == "ENG (eac3)") + #expect(track.isAtmos == false) + #expect(track.isDefault) + } + + @Test("a dec3 cookie announcing JOC marks the track Atmos, as a box or as its payload") + func eac3JOC() throws { + let payload = Self.dec3(joc: true) + var box = Data([0, 0, 0, UInt8(8 + payload.count)]) + box.append(contentsOf: Array("dec3".utf8)) + box.append(payload) + for cookie in [payload, box] { + let desc = try Self.audioDescription( + formatID: kAudioFormatEnhancedAC3, sampleRate: 48_000, channels: 6, cookie: cookie) + let reading = try #require(RemoteHLSStreamDescription.audioReading(from: desc, isEnabled: true, language: nil)) + let track = try #require(RemoteHLSStreamDescription.audioTracks([reading]).tracks.first) + #expect(track.isAtmos) + #expect(track.profile == "Dolby Digital Plus + Dolby Atmos") + } + #expect(RemoteHLSStreamDescription.eac3CarriesJOC(dec3: Self.dec3(joc: false)) == false) + #expect(RemoteHLSStreamDescription.eac3CarriesJOC(dec3: Data([0x50])) == false) + } + + @Test("audio format IDs map onto libavcodec names, AAC with its profile") + func audioCodecNames() { + #expect(RemoteHLSStreamDescription.audioCodec(formatID: kAudioFormatMPEG4AAC).name == "aac") + #expect(RemoteHLSStreamDescription.audioCodec(formatID: kAudioFormatMPEG4AAC).profile == "LC") + #expect(RemoteHLSStreamDescription.audioCodec(formatID: kAudioFormatMPEG4AAC_HE).profile == "HE-AAC") + #expect(RemoteHLSStreamDescription.audioCodec(formatID: kAudioFormatMPEG4AAC_HE_V2).profile == "HE-AACv2") + #expect(RemoteHLSStreamDescription.audioCodec(formatID: kAudioFormatAC3).name == "ac3") + #expect(RemoteHLSStreamDescription.audioCodec(formatID: kAudioFormatEnhancedAC3).name == "eac3") + #expect(RemoteHLSStreamDescription.audioCodec(formatID: kAudioFormatMPEGLayer3).name == "mp3") + #expect(RemoteHLSStreamDescription.audioCodec(formatID: kAudioFormatFLAC).name == "flac") + #expect(RemoteHLSStreamDescription.audioCodec(formatID: kAudioFormatOpus).name == "opus") + #expect(RemoteHLSStreamDescription.audioCodec(formatID: kAudioFormatAppleLossless).name == "alac") + // An unmapped format still names itself, like the video side. + #expect(RemoteHLSStreamDescription.audioCodec(formatID: 0x7A7A7A7A).name == "zzzz") + } + + @Test("ids are synthetic and ordinal, the enabled track is the active one, und is no language") + func multipleTracks() throws { + let a = try Self.audioDescription(formatID: kAudioFormatMPEG4AAC, sampleRate: 44_100, channels: 2) + let b = try Self.audioDescription(formatID: kAudioFormatAC3, sampleRate: 48_000, channels: 6) + let readings = [ + try #require(RemoteHLSStreamDescription.audioReading(from: a, isEnabled: false, language: "und")), + try #require(RemoteHLSStreamDescription.audioReading(from: b, isEnabled: true, language: "deu")), + ] + let (tracks, active) = RemoteHLSStreamDescription.audioTracks(readings) + #expect(tracks.map(\.id) == [RemoteHLSStreamDescription.audioTrackIDBase, RemoteHLSStreamDescription.audioTrackIDBase + 1]) + #expect(active == RemoteHLSStreamDescription.audioTrackIDBase + 1) + #expect(tracks[0].language == nil) + #expect(tracks[0].name == "Track 1 (aac)") + #expect(tracks[0].isDefault == false) + #expect(tracks[1].name == "DEU (ac3)") + } + + @Test("no enabled track means no active index") + func nothingEnabled() throws { + let a = try Self.audioDescription(formatID: kAudioFormatMPEG4AAC, sampleRate: 48_000, channels: 2) + let reading = try #require(RemoteHLSStreamDescription.audioReading(from: a, isEnabled: false, language: nil)) + #expect(RemoteHLSStreamDescription.audioTracks([reading]).activeID == nil) + #expect(RemoteHLSStreamDescription.audioTracks([]).tracks.isEmpty) + } + + @Test("a video description is not an audio one") + func videoIsNotAudio() throws { + let desc = try Self.videoDescription(subType: Self.avc1, width: 16, height: 16, extensions: [:]) + #expect(RemoteHLSStreamDescription.audioReading(from: desc, isEnabled: true, language: nil) == nil) + } + + // Audit NAT-106: a QuickTime SoundDescriptionV2 carries its rate as a Float64 verbatim, so on the + // bypass the origin controls it, and `Int(_:)` trapped past `Int.max` (the log site on NaN too). + @Test("a non-finite, negative or absurd sample rate reads as unknown", arguments: [ + Double.nan, .infinity, -48_000, 1e300, 1e9, + ]) + func hostileSampleRate(rate: Double) { + let reading = RemoteHLSStreamDescription.AudioReading( + formatID: kAudioFormatLinearPCM, sampleRate: rate, channels: 2, bitsPerChannel: 24, + carriesJOC: false, isEnabled: true, language: nil) + #expect(RemoteHLSStreamDescription.audioTracks([reading]).tracks.first?.sampleRate == 0) + #expect(RemoteHLSStreamDescription.wholeSampleRate(rate) == 0) + } + + @Test("an ordinary sample rate is kept") + func ordinarySampleRate() { + #expect(RemoteHLSStreamDescription.wholeSampleRate(48_000) == 48_000) + #expect(RemoteHLSStreamDescription.wholeSampleRate(44_100.0) == 44_100) + #expect(RemoteHLSStreamDescription.wholeSampleRate(768_000) == 768_000) + } +} diff --git a/Tests/AetherEngineTests/RemoteHLSSubtitleProxyTests.swift b/Tests/AetherEngineTests/RemoteHLSSubtitleProxyTests.swift index 1a6f714cb..6365f6be8 100644 --- a/Tests/AetherEngineTests/RemoteHLSSubtitleProxyTests.swift +++ b/Tests/AetherEngineTests/RemoteHLSSubtitleProxyTests.swift @@ -266,6 +266,50 @@ struct RemoteHLSSubtitleProxyTests { #expect(merged.isEmpty) } + // MARK: - NAT-1: a hostile EXTINF never reaches Int(Double) + + private static func seg(_ duration: Double) -> HLSMediaSegment { + HLSMediaSegment(uri: "s.ts", duration: duration, discontinuityBefore: false) + } + + @Test("An infinite, negative or absurd EXTINF is refused before it becomes a program duration") + func rejectsHostileSegmentDurations() { + #expect(throws: RemoteHLSSubtitleProxy.Refusal.self) { + _ = try RemoteHLSSubtitleProxy.sumSegmentDurations([Self.seg(.infinity)]) + } + #expect(throws: RemoteHLSSubtitleProxy.Refusal.self) { + _ = try RemoteHLSSubtitleProxy.sumSegmentDurations([Self.seg(.nan)]) + } + #expect(throws: RemoteHLSSubtitleProxy.Refusal.self) { + _ = try RemoteHLSSubtitleProxy.sumSegmentDurations([Self.seg(-1)]) + } + #expect(throws: RemoteHLSSubtitleProxy.Refusal.self) { + // Finite, but a sum this large is still out of range (audit NAT-1's "finite but huge" case). + _ = try RemoteHLSSubtitleProxy.sumSegmentDurations([Self.seg(5e18), Self.seg(5e18)]) + } + } + + @Test("An ordinary EXTINF sum is unaffected") + func sumsOrdinaryDurations() throws { + let total = try RemoteHLSSubtitleProxy.sumSegmentDurations([Self.seg(5), Self.seg(6.5)]) + #expect(total == 11.5) + } + + @Test("A non-finite or absurd program duration is clamped before it reaches the provider's segment") + func providerClampsAHostileProgramDuration() { + let providerInf = RemoteHLSSubtitleProvider( + tracks: [], masterBody: Self.master, programDuration: .infinity, defaultHeaders: [:]) + #expect(providerInf.segmentDuration(at: 0).isFinite) + + let providerHuge = RemoteHLSSubtitleProvider( + tracks: [], masterBody: Self.master, programDuration: 5e18, defaultHeaders: [:]) + #expect(providerHuge.segmentDuration(at: 0) <= RemoteHLSSubtitleProxy.maxProgramDurationSeconds) + + let providerNaN = RemoteHLSSubtitleProvider( + tracks: [], masterBody: Self.master, programDuration: .nan, defaultHeaders: [:]) + #expect(providerNaN.segmentDuration(at: 0) == 1) + } + @Test("Without an m3u8/NAME the display name is still the key") func injectionKeyFallsBackToDisplayName() { let bare = RemoteHLSMediaSelection.LegibleOption( diff --git a/Tests/AetherEngineTests/RemoteHLSTelemetryTests.swift b/Tests/AetherEngineTests/RemoteHLSTelemetryTests.swift new file mode 100644 index 000000000..308a80eb5 --- /dev/null +++ b/Tests/AetherEngineTests/RemoteHLSTelemetryTests.swift @@ -0,0 +1,53 @@ +import Foundation +import Testing +@testable import AetherEngine + +/// On the `nativeRemoteHLS` bypass the sampler never ran, because its bitrate halves read the loopback's +/// demuxer counter, and so `liveTelemetry` stayed nil for the whole session. AVPlayer's own access log is +/// there on this route too; these pin how it fills the same two bitrate fields. +/// +/// The fields name the stream's bitrate, so on the bypass they come from what the variant declares, not +/// from what AVPlayer transferred: a player filling its buffer after a start or a seek pulls at link +/// speed, and a transfer-based figure read 22.6 Mbps for a 3.7 Mbps Jellyfin transcode (device, 2026-09-28). +/// The transfer stays in the network fields, where it belongs. +@Suite("RemoteHLSTelemetry") +struct RemoteHLSTelemetryTests { + + @Test("the bypass reads the variant's declared bitrate, every other route the demuxer") + func counterPerRoute() { + #expect(LiveTelemetrySampler.bitrateCounter(for: .remoteBypass) == .declaredVariant) + #expect(LiveTelemetrySampler.bitrateCounter(for: .loopback) == .demuxer) + #expect(LiveTelemetrySampler.bitrateCounter(for: .software) == .demuxer) + #expect(LiveTelemetrySampler.bitrateCounter(for: .none) == .demuxer) + } + + @Test("the loopback-only readings stay nil on the bypass") + func loopbackOnlyReadings() { + #expect(LiveTelemetrySampler.readsLoopbackPipeline(.remoteBypass) == false) + #expect(LiveTelemetrySampler.readsLoopbackPipeline(.loopback)) + } + + @Test("instant is BANDWIDTH, average is AVERAGE-BANDWIDTH") + func bothDeclared() { + let rates = LiveTelemetrySampler.declaredBitrates(indicated: 4_000_000, indicatedAverage: 3_700_000) + #expect(rates.instant == 4.0) + #expect(rates.average == 3.7) + } + + /// AVERAGE-BANDWIDTH is optional in a master; without it the peak is the only declaration there is. + @Test("a master without AVERAGE-BANDWIDTH reports the peak for both") + func averageFallsBackToPeak() { + let rates = LiveTelemetrySampler.declaredBitrates(indicated: 4_000_000, indicatedAverage: 0) + #expect(rates.instant == 4.0) + #expect(rates.average == 4.0) + } + + /// The access log reports an unknown bitrate as a negative number or zero. + @Test("nothing declared is nil, not a measured zero") + func nothingDeclared() { + let rates = LiveTelemetrySampler.declaredBitrates(indicated: -1, indicatedAverage: -1) + #expect(rates.instant == nil) + #expect(rates.average == nil) + #expect(LiveTelemetrySampler.declaredBitrates(indicated: .nan, indicatedAverage: .infinity).instant == nil) + } +} diff --git a/Tests/AetherEngineTests/RestartReopenStopTests.swift b/Tests/AetherEngineTests/RestartReopenStopTests.swift new file mode 100644 index 000000000..fad8dfe8c --- /dev/null +++ b/Tests/AetherEngineTests/RestartReopenStopTests.swift @@ -0,0 +1,35 @@ +import Testing +import Foundation +@testable import AetherEngine + +/// The #79 / #169 restart opens a replacement demuxer while the old pump is wedged. `stop()` has to reach +/// that open: an unregistered one ran its connect and stream probe to the end after the session was gone. +@Suite("Restart reopen: stop() reaches the replacement demuxer") +struct RestartReopenStopTests { + + private func makeEngine() -> HLSVideoEngine { + HLSVideoEngine(url: URL(fileURLWithPath: "/nonexistent/restart-reopen.mkv"), dvModeAvailable: false) + } + + @Test("A stop after registration aborts the replacement's open before it reaches the origin") + func stopAbortsTheRegisteredOpen() throws { + let origin = try ProbeHTTPTestOrigin(data: try ProbeTestFixtures.hdr10Plus()) + defer { origin.stop() } + let engine = makeEngine() + let fresh = try #require(engine.registerRestartReopenDemuxer(epoch: 0)) + defer { fresh.close() } + + engine.stop() + + let url = try #require(URL(string: "http://127.0.0.1:\(origin.port)/restart-reopen.mp4")) + #expect(throws: (any Error).self) { try fresh.open(url: url) } + #expect(origin.requests.isEmpty) + } + + @Test("A restart that a stop already superseded opens no replacement at all") + func supersededRestartRegistersNothing() { + let engine = makeEngine() + engine.stop() + #expect(engine.registerRestartReopenDemuxer(epoch: 0) == nil) + } +} diff --git a/Tests/AetherEngineTests/RingAudioFlushEpochTests.swift b/Tests/AetherEngineTests/RingAudioFlushEpochTests.swift new file mode 100644 index 000000000..8120093e6 --- /dev/null +++ b/Tests/AetherEngineTests/RingAudioFlushEpochTests.swift @@ -0,0 +1,87 @@ +import Foundation +import Testing +import AetherLibavcodec +import AetherLibavutil +@testable import AetherEngine + +/// Audit DEC-106, the DVR ring's audio path. `seekLiveDVR` flushes the audio output and only then moves +/// the ring cursor, so the feeder or the look-ahead pump can be holding a packet read at the old cursor. +/// Its decoded buffers used to be enqueued unconditionally, and after a rewind a buffer from before the +/// seek carries a HIGHER stamp than the new clock: it parks at the head of the fresh queue and mutes the +/// audio for the length of the rewind. +@Suite("DVR ring audio follows the flush epoch (DEC-106)") +struct RingAudioFlushEpochTests { + + private final class Counter: @unchecked Sendable { + private let lock = NSLock() + private var count = 0 + func increment() { lock.withLock { count += 1 } } + var value: Int { lock.withLock { count } } + } + + private func makeWAV(seconds: Double) -> Data { + let sampleRate = 48_000, channels = 2 + let pcm = Data(count: Int(Double(sampleRate) * seconds) * channels * 2) + var d = Data() + func str(_ s: String) { d.append(s.data(using: .ascii)!) } + func u32(_ v: UInt32) { withUnsafeBytes(of: v.littleEndian) { d.append(contentsOf: $0) } } + func u16(_ v: UInt16) { withUnsafeBytes(of: v.littleEndian) { d.append(contentsOf: $0) } } + str("RIFF"); u32(UInt32(36 + pcm.count)); str("WAVE") + str("fmt "); u32(16); u16(1); u16(UInt16(channels)); u32(UInt32(sampleRate)) + u32(UInt32(sampleRate * channels * 2)); u16(UInt16(channels * 2)); u16(16) + str("data"); u32(UInt32(pcm.count)); d.append(pcm) + return d + } + + @Test("a ring audio packet decided on before a flush yields no buffer after it, one decided after does") + func ringPacketsFollowTheEpoch() throws { + let demuxer = Demuxer() + try demuxer.open(reader: DataIOReader(data: makeWAV(seconds: 4))) + defer { demuxer.close() } + let audioIndex = demuxer.audioStreamIndex + let stream = try #require(demuxer.stream(at: audioIndex)) + let decoder = AudioDecoder() + try decoder.open(stream: stream) + defer { decoder.close() } + + var packets: [PacketRingBuffer.Packet] = [] + while let pkt = try demuxer.readPacket() { + if pkt.pointee.stream_index == audioIndex, let data = pkt.pointee.data { + packets.append(PacketRingBuffer.Packet( + pts: Double(packets.count) * 1024.0 / 48_000.0, isKeyframe: true, isVideo: false, + bytes: Data(bytes: data, count: Int(pkt.pointee.size)))) + } + var p: UnsafeMutablePointer? = pkt + trackedPacketFree(&p) + } + try #require(packets.count >= 40) + + let output = AudioOutput() + let tapped = Counter() + func feed(_ packet: PacketRingBuffer.Packet, epoch: UInt64) -> Bool { + SoftwarePlaybackHost.feedRingPacket( + packet, + videoDecoder: SoftwareVideoDecoder(), + audioDecoder: decoder, + audioOutput: output, + videoStreamIndex: 0, + audioStreamIndex: audioIndex, + videoTimeBaseSeconds: 1.0 / 90_000.0, + audioTimeBaseSeconds: 1.0 / 48_000.0, + audioTapSink: { _ in tapped.increment() }, + audioEpoch: epoch, + noteDecodeGeneration: {}) + } + + let stale = output.epoch + output.flush() + let half = packets.count / 2 + let staleProduced = packets[.. 0) + } +} diff --git a/Tests/AetherEngineTests/SWDemuxReadGateTests.swift b/Tests/AetherEngineTests/SWDemuxReadGateTests.swift index f0c5b8c02..b743dfa80 100644 --- a/Tests/AetherEngineTests/SWDemuxReadGateTests.swift +++ b/Tests/AetherEngineTests/SWDemuxReadGateTests.swift @@ -57,4 +57,57 @@ struct SWDemuxReadGateTests { parkedCount: 4, parkedCap: cap, clockArmed: true, lastAudioPts: 104, clockSeconds: .nan) == false) } + + // MARK: - Parked wait (audit PERF-110) + + @Test("a lead just over the target wakes when the gate opens, not before the floor") + func waitFollowsTheGateOpening() { + let clock = 100.0 + let target = AudioLookaheadPolicy.targetLeadSeconds + #expect(abs(SoftwarePlaybackHost.parkedRendererWaitSeconds( + clockArmed: true, lastAudioPts: clock + target + 0.012, clockSeconds: clock, rate: 1) - 0.012) < 1e-9) + #expect(SoftwarePlaybackHost.parkedRendererWaitSeconds( + clockArmed: true, lastAudioPts: clock + target + 0.001, clockSeconds: clock, rate: 1) == 0.005) + } + + @Test("a lead far over the target is capped at 20 ms so parked video keeps reaching the decoder") + func waitIsCapped() { + #expect(SoftwarePlaybackHost.parkedRendererWaitSeconds( + clockArmed: true, lastAudioPts: 110, clockSeconds: 100, rate: 1) == 0.020) + } + + @Test("a gate that is already open, an unarmed clock and unreadable inputs poll at the 5 ms floor") + func waitFloor() { + #expect(SoftwarePlaybackHost.parkedRendererWaitSeconds( + clockArmed: true, lastAudioPts: 101, clockSeconds: 100, rate: 1) == 0.005) + #expect(SoftwarePlaybackHost.parkedRendererWaitSeconds( + clockArmed: false, lastAudioPts: 110, clockSeconds: 100, rate: 1) == 0.005) + #expect(SoftwarePlaybackHost.parkedRendererWaitSeconds( + clockArmed: true, lastAudioPts: .nan, clockSeconds: 100, rate: 1) == 0.005) + #expect(SoftwarePlaybackHost.parkedRendererWaitSeconds( + clockArmed: true, lastAudioPts: 110, clockSeconds: .nan, rate: 1) == 0.005) + #expect(SoftwarePlaybackHost.parkedRendererWaitSeconds( + clockArmed: true, lastAudioPts: 110, clockSeconds: 100, rate: 0) == 0.005) + #expect(SoftwarePlaybackHost.parkedRendererWaitSeconds( + clockArmed: true, lastAudioPts: .infinity, clockSeconds: 100, rate: 1) == 0.005) + } + + @Test("the rate shortens the wait the same way it shortens the time to the gate") + func waitScalesWithRate() { + let target = AudioLookaheadPolicy.targetLeadSeconds + let excess = 0.016 + #expect(abs(SoftwarePlaybackHost.parkedRendererWaitSeconds( + clockArmed: true, lastAudioPts: 100 + target + excess, clockSeconds: 100, rate: 2) - 0.008) < 1e-9) + } + + @Test("whatever the inputs, the wait stays between 5 and 20 ms") + func waitIsAlwaysBounded() { + for lead in stride(from: -2.0, through: 12.0, by: 0.37) { + for rate: Float in [0.25, 0.5, 1, 1.5, 2, 4, 8] { + let w = SoftwarePlaybackHost.parkedRendererWaitSeconds( + clockArmed: true, lastAudioPts: 100 + lead, clockSeconds: 100, rate: rate) + #expect(w >= 0.005 && w <= 0.020) + } + } + } } diff --git a/Tests/AetherEngineTests/SWPlaybackDiagStateTests.swift b/Tests/AetherEngineTests/SWPlaybackDiagStateTests.swift index 0eaac5b16..21d4b782c 100644 --- a/Tests/AetherEngineTests/SWPlaybackDiagStateTests.swift +++ b/Tests/AetherEngineTests/SWPlaybackDiagStateTests.swift @@ -64,3 +64,22 @@ struct SWPlaybackDiagStateTests { #expect(diag.snapshot.lastAudioPts == 4.0) } } + +/// AE#395: the DVR feeder arm delivers audio through the ring pump and never writes `demuxDiag`. +@Suite("SWDiag audio marker source") +struct SWDiagAudioMarkerSourceTests { + + @Test("A DVR session reads the pump's fed PTS, not the demux loop's empty marker") + func dvrSessionReadsThePump() { + let marker = SoftwarePlaybackHost.diagAudioMarker( + demuxLoopPts: .nan, dvrPumpPts: 1234.5, isDVRSession: true) + #expect(marker == 1234.5) + } + + @Test("Every other session keeps the demux loop's marker") + func otherSessionsKeepTheDemuxMarker() { + let marker = SoftwarePlaybackHost.diagAudioMarker( + demuxLoopPts: 16.11, dvrPumpPts: .nan, isDVRSession: false) + #expect(marker == 16.11) + } +} diff --git a/Tests/AetherEngineTests/ScrubThumbnailSegmentMappingTests.swift b/Tests/AetherEngineTests/ScrubThumbnailSegmentMappingTests.swift new file mode 100644 index 000000000..9df683e5b --- /dev/null +++ b/Tests/AetherEngineTests/ScrubThumbnailSegmentMappingTests.swift @@ -0,0 +1,61 @@ +import Testing +import Foundation +@testable import AetherEngine + +/// Audit SEG-2: scrub stills read init.mp4 plus a mapped segment file as one stream instead of +/// concatenating two heap copies. The reader must present exactly the bytes the concatenation did. +@Suite("Scrub thumbnail segment mapping") +struct ScrubThumbnailSegmentMappingTests { + + private func readAll(_ reader: DataIOReader, chunk: Int32) -> [UInt8] { + var out: [UInt8] = [] + var buffer = [UInt8](repeating: 0, count: Int(chunk)) + while true { + let n = buffer.withUnsafeMutableBufferPointer { reader.read($0.baseAddress, size: chunk) } + if n <= 0 { break } + out += buffer[0.. Data { + Data(base64Encoded: AtmosDetectionProbeIntegrationTests.videoOnlyBase64, + options: .ignoreUnknownCharacters) ?? Data() + } + + // MARK: - The conversion + + @Test("Non-finite and out-of-range seconds convert to nil", arguments: [ + Double.nan, .infinity, -.infinity, 1e300, 4.7e12, -4.7e12, + ]) + func unconvertible(seconds: Double) { + #expect(Demuxer.ticks(forSeconds: seconds, timeBase: Self.microseconds) == nil) + } + + @Test("Ordinary seconds convert exactly as before") + func convertible() { + #expect(Demuxer.ticks(forSeconds: 1.5, timeBase: Self.microseconds) == 1_500_000) + #expect(Demuxer.ticks(forSeconds: -2, timeBase: Self.microseconds) == -2_000_000) + #expect(Demuxer.ticks(forSeconds: 7508.9, timeBase: AVRational(num: 1, den: 90_000)) + == Int64(7508.9 * 90_000.0 / 1.0)) + #expect(Demuxer.ticks(forSeconds: 1, timeBase: AVRational(num: 0, den: 1)) == nil) + } + + // MARK: - The Demuxer's seeks + + @Test("seek(to:) refuses NaN, infinity and a target past Int64", arguments: [ + Double.nan, .infinity, 1e300, + ]) + func demuxerSeekRefuses(seconds: Double) throws { + let demuxer = Demuxer() + defer { demuxer.close() } + try demuxer.open(reader: DataIOReader(data: Self.fixture()), formatHint: "mp4") + #expect(demuxer.seek(to: seconds) == false) + #expect(demuxer.seek(to: 0)) + } + + @Test("seekBounded refuses NaN, infinity and a target past Int64, stream-anchored or not", arguments: [ + Double.nan, .infinity, 1e300, + ]) + func demuxerSeekBoundedRefuses(seconds: Double) throws { + let demuxer = Demuxer() + defer { demuxer.close() } + try demuxer.open(reader: DataIOReader(data: Self.fixture()), formatHint: "mp4") + #expect(demuxer.seekBounded(to: seconds, timeout: 2) == false) + #expect(demuxer.seekBounded(to: seconds, anchorStreamIndex: demuxer.videoStreamIndex, timeout: 2) == false) + } + + // MARK: - Byte estimate + + @Test("A byte estimate against an inflated total saturates at the total") + func byteEstimateSaturates() { + #expect(Demuxer.byteEstimateTarget(fileSize: .max, duration: 10, target: 100) == .max) + #expect(Demuxer.byteEstimateCorrection( + landed: 1.5, target: 3600, startOrigin: 0, duration: 3600, + fileSize: 1_000_000_000_000_000_000, currentByte: 997_000_000_000_000_000, attempt: 0) + == .probe(1_000_000_000_000_000_000)) + } + + // MARK: - FrameExtractor boundary + + @Test("A thumbnail at a non-finite or negative time is nil", arguments: [ + Double.infinity, .nan, -1, + ]) + func thumbnailRefuses(seconds: Double) async { + let extractor = FrameExtractor(reader: DataIOReader(data: Self.fixture()), formatHint: "mp4") + #expect(await extractor.thumbnail(at: seconds) == nil) + #expect(await extractor.snapshot(at: seconds) == nil) + await extractor.shutdown() + } + + @Test("A thumbnail far past the end is clamped to the source, not a trap") + func thumbnailFarPastTheEnd() async { + let extractor = FrameExtractor(reader: DataIOReader(data: Self.fixture()), formatHint: "mp4") + _ = await extractor.thumbnail(at: 1e300) + _ = await extractor.snapshot(at: 1e300) + await extractor.shutdown() + } +} diff --git a/Tests/AetherEngineTests/SegmentCacheTests.swift b/Tests/AetherEngineTests/SegmentCacheTests.swift index e30386e4e..74011795c 100644 --- a/Tests/AetherEngineTests/SegmentCacheTests.swift +++ b/Tests/AetherEngineTests/SegmentCacheTests.swift @@ -31,6 +31,34 @@ struct SegmentCacheTests { #expect(c.totalBytes == 250) } + @Test("A re-adopted index gets a fresh file and retires the previous one (audit SEG-4)") + func readoptionNeverReusesAPath() throws { + let c = SegmentCache(forwardWindow: 5, backwardWindow: 5) + defer { c.close() } + func staged(_ n: Int, fill: UInt8) throws -> URL { + let url = c.sessionDir.appendingPathComponent("staging-\(UUID().uuidString).tmp") + try makeData(n, fill: fill).write(to: url) + return url + } + c.adopt(index: 3, stagingPath: try staged(100, fill: 0x01), byteCount: 100) + let first = try #require(c.peekURL(index: 3)) + c.adopt(index: 3, stagingPath: try staged(250, fill: 0x02), byteCount: 250) + let second = try #require(c.peekURL(index: 3)) + + // A reader that resolved the first URL and finds it gone must not be able to drop the + // second adoption, which it could while both generations shared one path. + #expect(first != second) + #expect(!FileManager.default.fileExists(atPath: first.path)) + #expect(c.fetch(index: 3, timeout: 0.1)?.first == 0x02) + #expect(c.count == 1) + #expect(c.totalBytes == 250) + + c.store(index: 3, data: makeData(40, fill: 0x03)) + #expect(!FileManager.default.fileExists(atPath: second.path)) + #expect(c.peek(index: 3)?.first == 0x03) + #expect(c.totalBytes == 40) + } + @Test("Backward refetch does not evict already-produced forward segments") func backwardRefetchKeepsForward() { // forwardWindow small, backwardWindow wide so production never prunes the trailing end. diff --git a/Tests/AetherEngineTests/SegmentFetchWaitTests.swift b/Tests/AetherEngineTests/SegmentFetchWaitTests.swift index e6975e08e..c7af72054 100644 --- a/Tests/AetherEngineTests/SegmentFetchWaitTests.swift +++ b/Tests/AetherEngineTests/SegmentFetchWaitTests.swift @@ -120,6 +120,28 @@ struct SegmentFetchWaitTests { #expect(elapsed >= 0.3) } + @Test("a fetch riding a restart that outlives close() returns instead of spinning") + func rideStopsWhenCacheCloses() { + let cache = SegmentCache(forwardWindow: 10, backwardWindow: 10) + storeAbove(cache, range: 45...50) + cache.close() + let recorder = Recorder() + let polls = PollCounter() + let provider = VideoSegmentProvider( + cache: cache, segments: segments(60), codecsString: "hvc1", supplementalCodecs: nil, + resolution: (1920, 1080), videoRange: .sdr, frameRate: 24.0, hdcpLevel: nil, + sourceBitrate: 8_000_000, + restartHandler: { idx in recorder.record(idx) }, + restartActivity: { _ = polls.increment(); return true }, + initialRestartIndex: 0, + repositionWaitSlice: 0.05, + repositionRideCapSeconds: 30.0 + ) + let served = provider.mediaSegment(at: 40) + #expect(served == nil) + #expect(polls.increment() <= 3) + } + @Test("resume-anchored provider cold-waits at the anchor instead of restarting the producer") func resumeAnchorColdStart() { let cache = SegmentCache(forwardWindow: 10, backwardWindow: 10) diff --git a/Tests/AetherEngineTests/SegmentPlanHostileInputTests.swift b/Tests/AetherEngineTests/SegmentPlanHostileInputTests.swift new file mode 100644 index 000000000..a413c7349 --- /dev/null +++ b/Tests/AetherEngineTests/SegmentPlanHostileInputTests.swift @@ -0,0 +1,131 @@ +import Foundation +import Testing +import AetherLibavutil +@testable import AetherEngine + +/// Audit HLS-102: the segment-plan builders converted durations and index timestamps to `Int64` / `Int` +/// with trapping conversions and sized arrays from the duration. A container header can state about +/// 9.2e12 s, and a Matroska index can carry entries near both `Int64` extremes (libavformat rejects only +/// the relative-timestamp band), so nothing upstream bounded either. +@Suite("Segment-plan inputs from a hostile header or index") +struct SegmentPlanHostileInputTests { + + private let ts90k = AVRational(num: 1, den: 90_000) + private let nanos = AVRational(num: 1, den: 1_000_000_000) + + // MARK: - Container duration + + @Test("A container duration past a week is clamped to a week, not zeroed") + func containerDurationClamps() { + #expect(Demuxer.effectiveDurationSeconds( + declared: nil, readerDuration: nil, discTitle: nil, container: 9.2e12) == 604_800) + #expect(Demuxer.effectiveDurationSeconds( + declared: 1e13, readerDuration: nil, discTitle: nil, container: 7508) == 604_800) + #expect(Demuxer.effectiveDurationSeconds( + declared: nil, readerDuration: 1e13, discTitle: nil, container: 7508) == 604_800) + } + + @Test("A non-finite duration resolves to zero") + func nonFiniteDurationIsZero() { + #expect(Demuxer.effectiveDurationSeconds( + declared: .infinity, readerDuration: nil, discTitle: nil, container: 7508) == 0) + #expect(Demuxer.effectiveDurationSeconds( + declared: nil, readerDuration: nil, discTitle: .infinity, container: 7508) == 0) + } + + @Test("An ordinary duration keeps the precedence chain") + func ordinaryDurationUnchanged() { + #expect(Demuxer.effectiveDurationSeconds( + declared: nil, readerDuration: nil, discTitle: 42, container: 91_843.97) == 42) + #expect(Demuxer.effectiveDurationSeconds( + declared: nil, readerDuration: nil, discTitle: nil, container: 7508.93) == 7508.93) + } + + // MARK: - Index entries + + @Test("Index entries past 2^62 ticks or 4e9 seconds are implausible") + func implausibleIndexEntries() { + #expect(Demuxer.isPlausibleIndexTimestamp(Int64.max - (1 << 49), timeBase: ts90k) == false) + #expect(Demuxer.isPlausibleIndexTimestamp(Int64.min + 1, timeBase: ts90k) == false) + #expect(Demuxer.isPlausibleIndexTimestamp(Int64.min, timeBase: ts90k) == false) + // 2^61 ticks at 90 kHz is 2.56e13 s, far past the seconds limit. + #expect(Demuxer.isPlausibleIndexTimestamp(1 << 61, timeBase: ts90k) == false) + #expect(Demuxer.isPlausibleIndexTimestamp(1 << 62, timeBase: nanos) == false) + } + + @Test("Real index entries, epoch-anchored ones included, stay plausible") + func plausibleIndexEntries() { + #expect(Demuxer.isPlausibleIndexTimestamp(0, timeBase: ts90k)) + #expect(Demuxer.isPlausibleIndexTimestamp(-3003, timeBase: ts90k)) + #expect(Demuxer.isPlausibleIndexTimestamp(Int64(7508.9 * 90_000), timeBase: ts90k)) + // Epoch nanoseconds in 2026 (1.79e18) and epoch at 10 MHz. + #expect(Demuxer.isPlausibleIndexTimestamp(1_790_000_000_000_000_000, timeBase: nanos)) + #expect(Demuxer.isPlausibleIndexTimestamp(17_900_000_000_000_000, timeBase: AVRational(num: 1, den: 10_000_000))) + } + + @Test("The multi-clip fold and the composition offset drop an entry that would overflow") + func foldAndOffsetAreChecked() { + // A disc playlist stating a shift of 2e14 s is 1.8e19 ticks at 90 kHz. + #expect(Demuxer.foldedIndexTimestamp(90_000, subtractSeconds: 2e14, timeBase: ts90k) == nil) + #expect(Demuxer.foldedIndexTimestamp(90_000, subtractSeconds: .nan, timeBase: ts90k) == nil) + #expect(Demuxer.foldedIndexTimestamp(Int64.min + 5, subtractSeconds: 1, timeBase: ts90k) == nil) + #expect(Demuxer.foldedIndexTimestamp(900_000, subtractSeconds: 1.5, timeBase: ts90k) == 765_000) + #expect(Demuxer.foldedIndexTimestamp(900_000, subtractSeconds: 0, timeBase: ts90k) == 900_000) + #expect(Demuxer.offsetIndexTimestamp(Int64.max - 10, by: 20) == nil) + #expect(Demuxer.offsetIndexTimestamp(1000, by: -3003) == -2003) + #expect(Demuxer.offsetIndexTimestamp(1000, by: nil) == 1000) + } + + // MARK: - Builders + + @Test("The trust check survives an index spanning both Int64 extremes") + func trustCheckSpread() { + #expect(HLSVideoEngine.keyframeIndexIsTrustworthy( + keyframes: [Int64.max - (1 << 49), Int64.min + 1], + videoTimeBase: ts90k, sourceDurationSeconds: 7200) == false) + } + + @Test("A segmented plan over a manifest sum past Int64 ticks is empty instead of trapping") + func segmentedPlanHugeDuration() { + let plan = HLSVideoEngine.buildSegmentedSourcePlan( + segmentStartsSeconds: [0, 4, 8], videoTimeBase: ts90k, + sourceDurationSeconds: 2e14, startPts0: 0) + #expect(plan.isEmpty) + let hugeStart = HLSVideoEngine.buildSegmentedSourcePlan( + segmentStartsSeconds: [0, 4, 2e14], videoTimeBase: ts90k, + sourceDurationSeconds: 2e14 + 4, startPts0: 0) + #expect(hugeStart.isEmpty) + } + + @Test("A uniform plan over a huge duration stays bounded") + func uniformPlanBounded() { + let plan = HLSVideoEngine.buildUniformSegmentPlan( + videoTimeBase: ts90k, sourceDurationSeconds: 1e12) + #expect(plan.count <= HLSVideoEngine.maxPlanSegments) + #expect(plan.last.map { $0.startSeconds + $0.durationSeconds } == 1e12) + #expect(HLSVideoEngine.buildUniformSegmentPlan( + videoTimeBase: ts90k, sourceDurationSeconds: .infinity).isEmpty) + } + + @Test("A uniform plan whose ticks leave Int64 is empty instead of trapping") + func uniformPlanTickOverflow() { + #expect(HLSVideoEngine.buildUniformSegmentPlan( + videoTimeBase: ts90k, sourceDurationSeconds: 1e14).isEmpty) + } + + @Test("A keyframe plan whose final end leaves Int64 at a fine time base is empty instead of trapping") + func keyframePlanTickOverflow() { + #expect(HLSVideoEngine.buildKeyframeSegmentPlan( + keyframes: [0, 4_000_000_000, 8_000_000_000], videoTimeBase: nanos, + sourceDurationSeconds: 1e10).isEmpty) + } + + @Test("A week-long uniform plan at the target stride is unchanged by the cap") + func weekLongPlanUnchanged() { + let plan = HLSVideoEngine.buildUniformSegmentPlan( + videoTimeBase: ts90k, sourceDurationSeconds: 604_800) + #expect(plan.count == 151_200) + #expect(plan[1].startSeconds == 4) + #expect(plan.last.map { $0.startSeconds + $0.durationSeconds } == 604_800) + } +} diff --git a/Tests/AetherEngineTests/SegmentPlanSparseIndexTests.swift b/Tests/AetherEngineTests/SegmentPlanSparseIndexTests.swift index 1a43011bd..84a4cfd2c 100644 --- a/Tests/AetherEngineTests/SegmentPlanSparseIndexTests.swift +++ b/Tests/AetherEngineTests/SegmentPlanSparseIndexTests.swift @@ -88,14 +88,17 @@ struct SegmentPlanSparseIndexTests { @Test("The minimum-coverage threshold is one targetSegmentDuration (boundary)") func minimumCoverageBoundary() { // Span exactly one segment is trustworthy (seg 0 can be cut); span just under is not (seg 0 - // degenerates to the whole file). Both have a single sub-cap inter-keyframe gap, so only the - // coverage check decides. Pins the threshold to targetSegmentDuration (4.0 s). + // degenerates to the whole file). Both have a single sub-cap inter-keyframe gap and the tail + // witness is lifted, so only the coverage check decides. Pins the threshold to + // targetSegmentDuration (4.0 s). let exactlyOneSegment: [Int64] = [0, 4_000] // ms: 4.000 s span let justUnderOneSegment: [Int64] = [0, 3_999] // ms: 3.999 s span #expect(HLSVideoEngine.keyframeIndexIsTrustworthy( - keyframes: exactlyOneSegment, videoTimeBase: mkvMs, sourceDurationSeconds: 6843.872) == true) + keyframes: exactlyOneSegment, videoTimeBase: mkvMs, sourceDurationSeconds: 6843.872, + maxTrailingGapSeconds: .infinity) == true) #expect(HLSVideoEngine.keyframeIndexIsTrustworthy( - keyframes: justUnderOneSegment, videoTimeBase: mkvMs, sourceDurationSeconds: 6843.872) == false) + keyframes: justUnderOneSegment, videoTimeBase: mkvMs, sourceDurationSeconds: 6843.872, + maxTrailingGapSeconds: .infinity) == false) } @Test("A dense 4 s-GOP index across the whole title is trustworthy") @@ -114,26 +117,54 @@ struct SegmentPlanSparseIndexTests { @Test("The trusted-gap cap is pinned at 30 s (boundary)") func gapCapBoundary() { // 30.000 s gap is trusted, 30.001 s is not. Pins the cap so a future targetSegmentDuration - // change becomes a visible test break. + // change becomes a visible test break. The tail witness is lifted so only the gap decides. let exactly30: [Int64] = [0, Int64(30.0 * 90_000)] let justOver30: [Int64] = [0, Int64(30.001 * 90_000)] #expect(HLSVideoEngine.keyframeIndexIsTrustworthy( - keyframes: exactly30, videoTimeBase: ts90k, sourceDurationSeconds: 6599) == true) + keyframes: exactly30, videoTimeBase: ts90k, sourceDurationSeconds: 6599, + maxTrailingGapSeconds: .infinity) == true) #expect(HLSVideoEngine.keyframeIndexIsTrustworthy( - keyframes: justOver30, videoTimeBase: ts90k, sourceDurationSeconds: 6599) == false) + keyframes: justOver30, videoTimeBase: ts90k, sourceDurationSeconds: 6599, + maxTrailingGapSeconds: .infinity) == false) } - @Test("A trailing gap from the last keyframe to EOF is not counted") - func tailGapNotCounted() { - // Keyframes spaced 4 s up to 60% of the duration, then nothing. The last-keyframe-to-EOF span - // is not an inter-keyframe gap and must not demote an otherwise-dense index. + /// A dense 4 s index from 0 up to `lastIndexedSeconds`, then nothing. + private func denseIndex(upTo lastIndexedSeconds: Double) -> [Int64] { let stride: Int64 = 4 * 90_000 var kfs: [Int64] = [] var t: Int64 = 0 - let lastIndexed = Int64(0.6 * 6599 * 90_000) - while t <= lastIndexed { kfs.append(t); t += stride } + while t <= Int64(lastIndexedSeconds * 90_000) { kfs.append(t); t += stride } + return kfs + } + + @Test("A dense index that stops minutes before EOF is a partial scan, not trustworthy (PR #703)") + func partialScanRejected() { + // Keyframes spaced 4 s up to 60% of the duration, then nothing: what a capped prewarm leaves + // on an MKV whose Cues are missing or point past EOF. Every inter-keyframe gap is fine; the + // index still cannot plan the last 40% of the title. + #expect(HLSVideoEngine.keyframeIndexIsTrustworthy( + keyframes: denseIndex(upTo: 0.6 * 6599), videoTimeBase: ts90k, + sourceDurationSeconds: 6599) == false) + } + + @Test("Trusting the partial scan would have produced a final segment reaching to EOF (PR #703)") + func partialScanProducesGiantFinalSegment() { + // The device report: the last planned segment ran from the final scanned keyframe at 205 s to + // the end of the title at 5933 s, a segment the producer can never finish. + let plan = HLSVideoEngine.buildKeyframeSegmentPlan( + keyframes: denseIndex(upTo: 205), videoTimeBase: ts90k, sourceDurationSeconds: 5933) + #expect((plan.last?.durationSeconds ?? 0) > 5700) + } + + @Test("The trailing-gap tolerance is pinned at 60 s (boundary)") + func trailingGapBoundary() { + // Coverage 0 to 1200 s: a duration 60 s past it is trusted (a long final GOP, a padded audio + // or subtitle tail), 60.5 s past it is not. + let kfs = denseIndex(upTo: 1200) + #expect(HLSVideoEngine.keyframeIndexIsTrustworthy( + keyframes: kfs, videoTimeBase: ts90k, sourceDurationSeconds: 1260) == true) #expect(HLSVideoEngine.keyframeIndexIsTrustworthy( - keyframes: kfs, videoTimeBase: ts90k, sourceDurationSeconds: 6599) == true) + keyframes: kfs, videoTimeBase: ts90k, sourceDurationSeconds: 1260.5) == false) } @Test("Degenerate inputs are never trustworthy") diff --git a/Tests/AetherEngineTests/SequentialBridgedTailPlaylistTests.swift b/Tests/AetherEngineTests/SequentialBridgedTailPlaylistTests.swift new file mode 100644 index 000000000..116a6ce6c --- /dev/null +++ b/Tests/AetherEngineTests/SequentialBridgedTailPlaylistTests.swift @@ -0,0 +1,210 @@ +import Foundation +import Testing +@testable import AetherEngine + +/// Audit SEG-103: on a sequential origin the bridged audio follows the video cutter (7.22), but the +/// end-of-file flush of the bridge still routed its tail by plan time. When the last GOP crossed the +/// last plan boundary, that tail opened the plan index the cutter never reached: the real final +/// segment was closed as not-final, its duration never reported, and ENDLIST landed one segment early, +/// cutting the end of the archive off. +@Suite("A sequential archive whose last GOP crosses the last plan boundary lists its whole tail") +struct SequentialBridgedTailPlaylistTests { + + /// 16 s, H.264 16x16 at 10 fps with keyframes at 0, 9 and 11 s, and silent mono Opus (bridged to + /// FLAC) in 120 ms frames. The first GOP outlasts every packet the open samples, so a forward-only + /// session knows one keyframe and plans the uniform 4 s stride (boundaries 0, 4, 8, 12). The cutter + /// ends in seg2 ([9, 16)), while the flushed audio tail sits in plan seg3. Regenerate with: + /// + /// ffmpeg -f lavfi -i "color=c=black:s=16x16:r=10:d=16" -f lavfi -i "anullsrc=r=48000:cl=mono" \ + /// -map 0:v -map 1:a -c:v libx264 -preset ultrafast -profile:v baseline -crf 51 -g 1000 \ + /// -sc_threshold 0 -force_key_frames "0,9,11" -x264-params bframes=0 -pix_fmt yuv420p \ + /// -c:a libopus -b:a 6k -frame_duration 120 -ac 1 -t 16 -f matroska - > tail.mkv + /// + /// Written to a pipe so the file carries no Cues: the whole file fits the first read, and with its + /// Cues in the buffer the plan would come out keyframe-aligned and hide the defect. + private static let fixtureBase64 = [ + "GkXfo6NChoEBQveBAULygQRC84EIQoKIbWF0cm9za2FCh4EEQoWBAhhTgGcB/////////xFNm3Sxv4S8aizJTbuLU6uEFUmp", + "ZlOsgaFNu4tTq4QWVK5rU6yB8U27jFOrhBJUw2dTrIIB3OwBAAAAAAAAYgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA", + "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAFUmp", + "Zsu/hLjH+Nwq17GDD0JATYCNTGF2ZjYyLjEyLjEwMVdBjUxhdmY2Mi4xMi4xMDFzpJA0kss7xod4ugMjeGfAY/KcRImIQM9A", + "AAAAAAAWVK5rQOW/hJ6CS2SuAQAAAAAAAHHXgQFzxYiAoAqYJZs2m5yBACK1nIN1bmSIgQCGj1ZfTVBFRzQvSVNPL0FWQ4OB", + "ASPjg4QF9eEA4JCwgRC6gRCagQJVsIRVuYEBY6KlAULACv/hABVnQsAK2nsBEAAAAwAQAAADAUDxImoBAAVozgGXIK4BAAAA", + "AAAAXNeBAnPFiM7OMF8gqYhDnIEAIrWcg3VuZIiBAIaGQV9PUFVTVqqDYy6gVruEBMS0AIOBAuGRn4EBtYhA53AAAAAAAGJk", + "gRBjopNPcHVzSGVhZAEBOAGAuwAAAAAAElTDZ0CVv4Qb4HYic3OgY8CAZ8iaRaOHRU5DT0RFUkSHjUxhdmY2Mi4xMi4xMDFz", + "c7NjwItjxYiAoAqYJZs2m2fIokWjh0VOQ09ERVJEh5VMYXZjNjIuMjguMTAxIGxpYngyNjRzc7NjwItjxYjOzjBfIKmIQ2fI", + "okWjh0VOQ09ERVJEh5VMYXZjNjIuMjguMTAxIGxpYm9wdXMfQ7Z1RDe/hAXdYNDngQCjoYIAAIAaDgL5jsjhz5S+lx76gm6w", + "Aissw4zYC9cFtr0Q0qNCaoEAAIAAAAJVBgX//1HcRem95tlIt5Ys2CDZI+7veDI2NCAtIGNvcmUgMTY1IHIzMjIyIGIzNTYw", + "NWEgLSBILjI2NC9NUEVHLTQgQVZDIGNvZGVjIC0gQ29weWxlZnQgMjAwMy0yMDI1IC0gaHR0cDovL3d3dy52aWRlb2xhbi5v", + "cmcveDI2NC5odG1sIC0gb3B0aW9uczogY2FiYWM9MCByZWY9MSBkZWJsb2NrPTA6MDowIGFuYWx5c2U9MDowIG1lPWRpYSBz", + "dWJtZT0wIHBzeT0xIHBzeV9yZD0xLjAwOjAuMDAgbWl4ZWRfcmVmPTAgbWVfcmFuZ2U9MTYgY2hyb21hX21lPTEgdHJlbGxp", + "cz0wIDh4OGRjdD0wIGNxbT0wIGRlYWR6b25lPTIxLDExIGZhc3RfcHNraXA9MSBjaHJvbWFfcXBfb2Zmc2V0PTAgdGhyZWFk", + "cz0xIGxvb2thaGVhZF90aHJlYWRzPTEgc2xpY2VkX3RocmVhZHM9MCBucj0wIGRlY2ltYXRlPTEgaW50ZXJsYWNlZD0wIGJs", + "dXJheV9jb21wYXQ9MCBjb25zdHJhaW5lZF9pbnRyYT0wIGJmcmFtZXM9MCB3ZWlnaHRwPTAga2V5aW50PTEwMDAga2V5aW50", + "X21pbj0xMCBzY2VuZWN1dD0wIGludHJhX3JlZnJlc2g9MCByYz1jcmYgbWJ0cmVlPTAgY3JmPTUxLjAgcWNvbXA9MC42MCBx", + "cG1pbj0wIHFwbWF4PTY5IHFwc3RlcD00IGlwX3JhdGlvPTEuNDAgYXE9MACAAAAACWWIhDomKAAVwKONgQBkAAAAAAVBmiAy", + "lKOfggB5gBkCKyzDjNgL1wW2vRDSAissw4zYC9cFtr0Q0qONgQDIAAAAAAVBmkA2lKOfggDxgBkCKyzDjNgL1wW2vRDSAiss", + "w4zYC9cFtr0Q0qONgQEsAAAAAAVBmmA2lKOfggFpgBkCKyzDjNgL1wW2vRDSAissw4zYC9cFtr0Q0qONgQGQAAAAAAVBmoA2", + "lKOfggHhgBkCKyzDjNgL1wW2vRDSAissw4zYC9cFtr0Q0qONgQH0AAAAAAVBmqA2lKOfggJZgBkCKyzDjNgL1wW2vRDSAiss", + "w4zYC9cFtr0Q0qONgQJYAAAAAAVBmsA6lKONgQK8AAAAAAVBmuA6lKOfggLRgBkCKyzDjNgL1wW2vRDSAissw4zYC9cFtr0Q", + "0qONgQMgAAAAAAVBmwA6lKOfggNJgBkCKyzDjNgL1wW2vRDSAissw4zYC9cFtr0Q0qONgQOEAAAAAAVBmyA6lKOfggPBgBkC", + "KyzDjNgL1wW2vRDSAissw4zYC9cFtr0Q0qONgQPoAAAAAAVBm0A6lB9DtnVBqL+EaVCv8ueCBDmjn4IAAIAZAissw4zYC9cF", + "tr0Q0gIrLMOM2AvXBba9ENKjjYEAEwAAAAAFQZtgOpSjn4IAeIAZAissw4zYC9cFtr0Q0gIrLMOM2AvXBba9ENKjjYEAdwAA", + "AAAFQZuAOpSjjYEA2wAAAAAFQZugOpSjn4IA8IAZAissw4zYC9cFtr0Q0gIrLMOM2AvXBba9ENKjjYEBPwAAAAAFQZvAOpSj", + "n4IBaIAZAissw4zYC9cFtr0Q0gIrLMOM2AvXBba9ENKjjYEBowAAAAAFQZvgOpSjn4IB4IAZAissw4zYC9cFtr0Q0gIrLMOM", + "2AvXBba9ENKjjYECBwAAAAAFQZoAOpSjn4ICWIAZAissw4zYC9cFtr0Q0gIrLMOM2AvXBba9ENKjjYECawAAAAAFQZogOpSj", + "n4IC0IAZAissw4zYC9cFtr0Q0gIrLMOM2AvXBba9ENKjjYECzwAAAAAFQZpAOpSjjYEDMwAAAAAFQZpgOpSjn4IDSIAZAiss", + "w4zYC9cFtr0Q0gIrLMOM2AvXBba9ENKjjYEDlwAAAAAFQZqAOpQfQ7Z1Qcm/hGOleDnnggf5o5+CAACAGQIrLMOM2AvXBba9", + "ENICKyzDjNgL1wW2vRDSo42BADsAAAAABUGaoDqUo5+CAHiAGQIrLMOM2AvXBba9ENICKyzDjNgL1wW2vRDSo42BAJ8AAAAA", + "BUGawDqUo5+CAPCAGQIrLMOM2AvXBba9ENICKyzDjNgL1wW2vRDSo42BAQMAAAAABUGa4DqUo5+CAWiAGQIrLMOM2AvXBba9", + "ENICKyzDjNgL1wW2vRDSo42BAWcAAAAABUGbADqUo42BAcsAAAAABUGbIDqUo5+CAeCAGQIrLMOM2AvXBba9ENICKyzDjNgL", + "1wW2vRDSo42BAi8AAAAABUGbQDqUo5+CAliAGQIrLMOM2AvXBba9ENICKyzDjNgL1wW2vRDSo42BApMAAAAABUGbYDqUo5+C", + "AtCAGQIrLMOM2AvXBba9ENICKyzDjNgL1wW2vRDSo42BAvcAAAAABUGbgDqUo5+CA0iAGQIrLMOM2AvXBba9ENICKyzDjNgL", + "1wW2vRDSo42BA1sAAAAABUGboDqUo5+CA8CAGQIrLMOM2AvXBba9ENICKyzDjNgL1wW2vRDSo42BA78AAAAABUGbwDqUH0O2", + "dUHYv4RN4H6V54IMHKONgQAAAAAAAAVBm+A6lKOfggAVgBkCKyzDjNgL1wW2vRDSAissw4zYC9cFtr0Q0qONgQBkAAAAAAVB", + "mgA6lKOfggCNgBkCKyzDjNgL1wW2vRDSAissw4zYC9cFtr0Q0qONgQDIAAAAAAVBmiA6lKOfggEFgBkCKyzDjNgL1wW2vRDS", + "Aissw4zYC9cFtr0Q0qONgQEsAAAAAAVBmkA6lKOfggF9gBkCKyzDjNgL1wW2vRDSAissw4zYC9cFtr0Q0qONgQGQAAAAAAVB", + "mmA6lKOfggH1gBkCKyzDjNgL1wW2vRDSAissw4zYC9cFtr0Q0qONgQH0AAAAAAVBmoA6lKONgQJYAAAAAAVBmqA6lKOfggJt", + "gBkCKyzDjNgL1wW2vRDSAissw4zYC9cFtr0Q0qONgQK8AAAAAAVBmsA6lKOfggLlgBkCKyzDjNgL1wW2vRDSAissw4zYC9cF", + "tr0Q0qONgQMgAAAAAAVBmuA6lKOfggNdgBkCKyzDjNgL1wW2vRDSAissw4zYC9cFtr0Q0qONgQOEAAAAAAVBmwA6lKOfggPV", + "gBkCKyzDjNgL1wW2vRDSAissw4zYC9cFtr0Q0qONgQPoAAAAAAVBmyA6lB9DtnVB2L+EPAUI9+eCEGmjn4IAAIAZAissw4zY", + "C9cFtr0Q0gIrLMOM2AvXBba9ENKjjYH//wAAAAAFQZtAOpSjjYEAYwAAAAAFQZtgOpSjn4IAeIAZAissw4zYC9cFtr0Q0gIr", + "LMOM2AvXBba9ENKjjYEAxwAAAAAFQZuAOpSjn4IA8IAZAissw4zYC9cFtr0Q0gIrLMOM2AvXBba9ENKjjYEBKwAAAAAFQZug", + "OpSjn4IBaIAZAissw4zYC9cFtr0Q0gIrLMOM2AvXBba9ENKjjYEBjwAAAAAFQZvAOpSjn4IB4IAZAissw4zYC9cFtr0Q0gIr", + "LMOM2AvXBba9ENKjjYEB8wAAAAAFQZvgOpSjn4ICWIAZAissw4zYC9cFtr0Q0gIrLMOM2AvXBba9ENKjjYECVwAAAAAFQZoA", + "OpSjjYECuwAAAAAFQZogOpSjn4IC0IAZAissw4zYC9cFtr0Q0gIrLMOM2AvXBba9ENKjjYEDHwAAAAAFQZpAOpSjn4IDSIAZ", + "Aissw4zYC9cFtr0Q0gIrLMOM2AvXBba9ENKjjYEDgwAAAAAFQZpgOpSjn4IDwIAZAissw4zYC9cFtr0Q0gIrLMOM2AvXBba9", + "ENKjjYED5wAAAAAFQZqAOpQfQ7Z1Qai/hGM6jEvnghSho5+CAACAGQIrLMOM2AvXBba9ENICKyzDjNgL1wW2vRDSo42BABMA", + "AAAABUGaoDqUo5+CAHiAGQIrLMOM2AvXBba9ENICKyzDjNgL1wW2vRDSo42BAHcAAAAABUGawDqUo42BANsAAAAABUGa4DqU", + "o5+CAPCAGQIrLMOM2AvXBba9ENICKyzDjNgL1wW2vRDSo42BAT8AAAAABUGbADqUo5+CAWiAGQIrLMOM2AvXBba9ENICKyzD", + "jNgL1wW2vRDSo42BAaMAAAAABUGbIDqUo5+CAeCAGQIrLMOM2AvXBba9ENICKyzDjNgL1wW2vRDSo42BAgcAAAAABUGbQDqU", + "o5+CAliAGQIrLMOM2AvXBba9ENICKyzDjNgL1wW2vRDSo42BAmsAAAAABUGbYDqUo5+CAtCAGQIrLMOM2AvXBba9ENICKyzD", + "jNgL1wW2vRDSo42BAs8AAAAABUGbgDqUo42BAzMAAAAABUGboDqUo5+CA0iAGQIrLMOM2AvXBba9ENICKyzDjNgL1wW2vRDS", + "o42BA5cAAAAABUGbwDqUH0O2dUHJv4S/7m/C54IYYaOfggAAgBkCKyzDjNgL1wW2vRDSAissw4zYC9cFtr0Q0qONgQA7AAAA", + "AAVBm+A6lKOfggB4gBkCKyzDjNgL1wW2vRDSAissw4zYC9cFtr0Q0qONgQCfAAAAAAVBmgA6lKOfggDwgBkCKyzDjNgL1wW2", + "vRDSAissw4zYC9cFtr0Q0qONgQEDAAAAAAVBmiA6lKOfggFogBkCKyzDjNgL1wW2vRDSAissw4zYC9cFtr0Q0qONgQFnAAAA", + "AAVBmkA6lKONgQHLAAAAAAVBmmA6lKOfggHggBkCKyzDjNgL1wW2vRDSAissw4zYC9cFtr0Q0qONgQIvAAAAAAVBmoA6lKOf", + "ggJYgBkCKyzDjNgL1wW2vRDSAissw4zYC9cFtr0Q0qONgQKTAAAAAAVBmqA6lKOfggLQgBkCKyzDjNgL1wW2vRDSAissw4zY", + "C9cFtr0Q0qONgQL3AAAAAAVBmsA6lKOfggNIgBkCKyzDjNgL1wW2vRDSAissw4zYC9cFtr0Q0qONgQNbAAAAAAVBmuA6lKOf", + "ggPAgBkCKyzDjNgL1wW2vRDSAissw4zYC9cFtr0Q0qONgQO/AAAAAAVBmwA6lB9DtnVB2L+EcCOFlOeCHISjjYEAAAAAAAAF", + "QZsgOpSjn4IAFYAZAissw4zYC9cFtr0Q0gIrLMOM2AvXBba9ENKjjYEAZAAAAAAFQZtAOpSjn4IAjYAZAissw4zYC9cFtr0Q", + "0gIrLMOM2AvXBba9ENKjjYEAyAAAAAAFQZtgOpSjn4IBBYAZAissw4zYC9cFtr0Q0gIrLMOM2AvXBba9ENKjjYEBLAAAAAAF", + "QZuAOpSjn4IBfYAZAissw4zYC9cFtr0Q0gIrLMOM2AvXBba9ENKjjYEBkAAAAAAFQZugOpSjn4IB9YAZAissw4zYC9cFtr0Q", + "0gIrLMOM2AvXBba9ENKjjYEB9AAAAAAFQZvAOpSjjYECWAAAAAAFQZvgOpSjn4ICbYAZAissw4zYC9cFtr0Q0gIrLMOM2AvX", + "Bba9ENKjjYECvAAAAAAFQZoAOpSjn4IC5YAZAissw4zYC9cFtr0Q0gIrLMOM2AvXBba9ENKjjYEDIAAAAAAFQZogOpSjn4ID", + "XYAZAissw4zYC9cFtr0Q0gIrLMOM2AvXBba9ENKjjYEDhAAAAAAFQZpAOpSjn4ID1YAZAissw4zYC9cFtr0Q0gIrLMOM2AvX", + "Bba9ENKjjYED6AAAAAAFQZpgOpQfQ7Z1Qd6/hO+e4KzngiDRo5+CAACAGQIrLMOM2AvXBba9ENICKyzDjNgL1wW2vRDSo42B", + "//8AAAAABUGagDqUo42BAGMAAAAABUGaoDqUo5+CAHiAGQIrLMOM2AvXBba9ENICKyzDjNgL1wW2vRDSo42BAMcAAAAABUGa", + "wDqUo5+CAPCAGQIrLMOM2AvXBba9ENICKyzDjNgL1wW2vRDSo42BASsAAAAABUGa4DqUo5+CAWiAGQIrLMOM2AvXBba9ENIC", + "KyzDjNgL1wW2vRDSo42BAY8AAAAABUGbADqUo5+CAeCAGQIrLMOM2AvXBba9ENICKyzDjNgL1wW2vRDSo42BAfMAAAAABUGb", + "IDqUo5+CAliAGQIrLMOM2AvXBba9ENICKyzDjNgL1wW2vRDSo5OBAleAAAAAC2WIggEKJigACCDgo42BArsAAAAABUGaIDaU", + "o5+CAtCAGQIrLMOM2AvXBba9ENICKyzDjNgL1wW2vRDSo42BAx8AAAAABUGaQDaUo5+CA0iAGQIrLMOM2AvXBba9ENICKyzD", + "jNgL1wW2vRDSo42BA4MAAAAABUGaYDaUo5+CA8CAGQIrLMOM2AvXBba9ENICKyzDjNgL1wW2vRDSo42BA+cAAAAABUGagDaU", + "H0O2dUGov4R5ya9c54IlCaOfggAAgBkCKyzDjNgL1wW2vRDSAissw4zYC9cFtr0Q0qONgQATAAAAAAVBmqA2lKOfggB4gBkC", + "KyzDjNgL1wW2vRDSAissw4zYC9cFtr0Q0qONgQB3AAAAAAVBmsA6lKONgQDbAAAAAAVBmuA6lKOfggDwgBkCKyzDjNgL1wW2", + "vRDSAissw4zYC9cFtr0Q0qONgQE/AAAAAAVBmwA6lKOfggFogBkCKyzDjNgL1wW2vRDSAissw4zYC9cFtr0Q0qONgQGjAAAA", + "AAVBmyA6lKOfggHggBkCKyzDjNgL1wW2vRDSAissw4zYC9cFtr0Q0qONgQIHAAAAAAVBm0A6lKOfggJYgBkCKyzDjNgL1wW2", + "vRDSAissw4zYC9cFtr0Q0qONgQJrAAAAAAVBm2A6lKOfggLQgBkCKyzDjNgL1wW2vRDSAissw4zYC9cFtr0Q0qONgQLPAAAA", + "AAVBm4A6lKONgQMzAAAAAAVBm6A6lKOfggNIgBkCKyzDjNgL1wW2vRDSAissw4zYC9cFtr0Q0qONgQOXAAAAAAVBm8A6lB9D", + "tnVBz7+E81n7sOeCKMmjn4IAAIAZAissw4zYC9cFtr0Q0gIrLMOM2AvXBba9ENKjjYEAOwAAAAAFQZvgOpSjn4IAeIAZAiss", + "w4zYC9cFtr0Q0gIrLMOM2AvXBba9ENKjjYEAnwAAAAAFQZoAOpSjn4IA8IAZAissw4zYC9cFtr0Q0gIrLMOM2AvXBba9ENKj", + "jYEBAwAAAAAFQZogOpSjn4IBaIAZAissw4zYC9cFtr0Q0gIrLMOM2AvXBba9ENKjjYEBZwAAAAAFQZpAOpSjjYEBywAAAAAF", + "QZpgOpSjn4IB4IAZAissw4zYC9cFtr0Q0gIrLMOM2AvXBba9ENKjk4ECL4AAAAALZYiEBCiYoAAgg4Cjn4ICWIAZAissw4zY", + "C9cFtr0Q0gIrLMOM2AvXBba9ENKjjYECkwAAAAAFQZogNpSjn4IC0IAZAissw4zYC9cFtr0Q0gIrLMOM2AvXBba9ENKjjYEC", + "9wAAAAAFQZpANpSjn4IDSIAZAissw4zYC9cFtr0Q0gIrLMOM2AvXBba9ENKjjYEDWwAAAAAFQZpgNpSjn4IDwIAZAissw4zY", + "C9cFtr0Q0gIrLMOM2AvXBba9ENKjjYEDvwAAAAAFQZqANpQfQ7Z1Qdi/hGw/LR/ngizso42BAAAAAAAABUGaoDaUo5+CABWA", + "GQIrLMOM2AvXBba9ENICKyzDjNgL1wW2vRDSo42BAGQAAAAABUGawDqUo5+CAI2AGQIrLMOM2AvXBba9ENICKyzDjNgL1wW2", + "vRDSo42BAMgAAAAABUGa4DqUo5+CAQWAGQIrLMOM2AvXBba9ENICKyzDjNgL1wW2vRDSo42BASwAAAAABUGbADqUo5+CAX2A", + "GQIrLMOM2AvXBba9ENICKyzDjNgL1wW2vRDSo42BAZAAAAAABUGbIDqUo5+CAfWAGQIrLMOM2AvXBba9ENICKyzDjNgL1wW2", + "vRDSo42BAfQAAAAABUGbQDqUo42BAlgAAAAABUGbYDqUo5+CAm2AGQIrLMOM2AvXBba9ENICKyzDjNgL1wW2vRDSo42BArwA", + "AAAABUGbgDqUo5+CAuWAGQIrLMOM2AvXBba9ENICKyzDjNgL1wW2vRDSo42BAyAAAAAABUGboDqUo5+CA12AGQIrLMOM2AvX", + "Bba9ENICKyzDjNgL1wW2vRDSo42BA4QAAAAABUGbwDqUo5+CA9WAGQIrLMOM2AvXBba9ENICKyzDjNgL1wW2vRDSo42BA+gA", + "AAAABUGb4DqUH0O2dUHYv4RyMcKR54IxOaOfggAAgBkCKyzDjNgL1wW2vRDSAissw4zYC9cFtr0Q0qONgf//AAAAAAVBmgA6", + "lKONgQBjAAAAAAVBmiA6lKOfggB4gBkCKyzDjNgL1wW2vRDSAissw4zYC9cFtr0Q0qONgQDHAAAAAAVBmkA6lKOfggDwgBkC", + "KyzDjNgL1wW2vRDSAissw4zYC9cFtr0Q0qONgQErAAAAAAVBmmA6lKOfggFogBkCKyzDjNgL1wW2vRDSAissw4zYC9cFtr0Q", + "0qONgQGPAAAAAAVBmoA6lKOfggHggBkCKyzDjNgL1wW2vRDSAissw4zYC9cFtr0Q0qONgQHzAAAAAAVBmqA6lKOfggJYgBkC", + "KyzDjNgL1wW2vRDSAissw4zYC9cFtr0Q0qONgQJXAAAAAAVBmsA6lKONgQK7AAAAAAVBmuA6lKOfggLQgBkCKyzDjNgL1wW2", + "vRDSAissw4zYC9cFtr0Q0qONgQMfAAAAAAVBmwA6lKOfggNIgBkCKyzDjNgL1wW2vRDSAissw4zYC9cFtr0Q0qONgQODAAAA", + "AAVBmyA6lKOfggPAgBkCKyzDjNgL1wW2vRDSAissw4zYC9cFtr0Q0qONgQPnAAAAAAVBm0A6lB9DtnVBqL+EMUMKneeCNXGj", + "n4IAAIAZAissw4zYC9cFtr0Q0gIrLMOM2AvXBba9ENKjjYEAEwAAAAAFQZtgOpSjn4IAeIAZAissw4zYC9cFtr0Q0gIrLMOM", + "2AvXBba9ENKjjYEAdwAAAAAFQZuAOpSjjYEA2wAAAAAFQZugOpSjn4IA8IAZAissw4zYC9cFtr0Q0gIrLMOM2AvXBba9ENKj", + "jYEBPwAAAAAFQZvAOpSjn4IBaIAZAissw4zYC9cFtr0Q0gIrLMOM2AvXBba9ENKjjYEBowAAAAAFQZvgOpSjn4IB4IAZAiss", + "w4zYC9cFtr0Q0gIrLMOM2AvXBba9ENKjjYECBwAAAAAFQZoAOpSjn4ICWIAZAissw4zYC9cFtr0Q0gIrLMOM2AvXBba9ENKj", + "jYECawAAAAAFQZogOpSjn4IC0IAZAissw4zYC9cFtr0Q0gIrLMOM2AvXBba9ENKjjYECzwAAAAAFQZpAOpSjjYEDMwAAAAAF", + "QZpgOpSjn4IDSIAZAissw4zYC9cFtr0Q0gIrLMOM2AvXBba9ENKjjYEDlwAAAAAFQZqAOpQfQ7Z1Qcm/hI1CLejngjkxo5+C", + "AACAGQIrLMOM2AvXBba9ENICKyzDjNgL1wW2vRDSo42BADsAAAAABUGaoDqUo5+CAHiAGQIrLMOM2AvXBba9ENICKyzDjNgL", + "1wW2vRDSo42BAJ8AAAAABUGawDqUo5+CAPCAGQIrLMOM2AvXBba9ENICKyzDjNgL1wW2vRDSo42BAQMAAAAABUGa4DqUo5+C", + "AWiAGQIrLMOM2AvXBba9ENICKyzDjNgL1wW2vRDSo42BAWcAAAAABUGbADqUo42BAcsAAAAABUGbIDqUo5+CAeCAGQIrLMOM", + "2AvXBba9ENICKyzDjNgL1wW2vRDSo42BAi8AAAAABUGbQDqUo5+CAliAGQIrLMOM2AvXBba9ENICKyzDjNgL1wW2vRDSo42B", + "ApMAAAAABUGbYDqUo5+CAtCAGQIrLMOM2AvXBba9ENICKyzDjNgL1wW2vRDSo42BAvcAAAAABUGbgDqUo5+CA0iAGQIrLMOM", + "2AvXBba9ENICKyzDjNgL1wW2vRDSo42BA1sAAAAABUGboDqUo5+CA8CAGQIrLMOM2AvXBba9ENICKyzDjNgL1wW2vRDSo42B", + "A78AAAAABUGbwDqUH0O2dUCmv4R3b3PO54I9VKONgQAAAAAAAAVBm+A6lKOfggAVgBkCKyzDjNgL1wW2vRDSAissw4zYC9cF", + "tr0Q0qONgQBkAAAAAAVBmgA6lKOfggCNgBkCKyzDjNgL1wW2vRDSAissw4zYC9cFtr0Q0qONgQDIAAAAAAVBmiA6lKCroZ+C", + "AQUAGQIrLMOM2AvXBba9ENICKyzDjNgL1wW2vRDSm4EvdaKEBGGFYA==", + ] + private static let videoFrames = 160 + + /// Video samples (track 1) across every fragment of a segment, from the `trun` sample counts. + private static func videoSampleCount(_ segment: Data) -> Int { + let bytes = [UInt8](segment) + func u32(_ off: Int) -> Int { + Int(bytes[off]) << 24 | Int(bytes[off + 1]) << 16 | Int(bytes[off + 2]) << 8 | Int(bytes[off + 3]) + } + func boxes(_ range: Range) -> [(String, Range)] { + var out: [(String, Range)] = [] + var off = range.lowerBound + while off + 8 <= range.upperBound { + let size = u32(off) + guard size >= 8, off + size <= range.upperBound else { break } + out.append((String(decoding: bytes[off + 4.. String { (try? String(contentsOf: mediaURL, encoding: .utf8)) ?? "" } + try await waitFor { playlistText().contains("#EXT-X-ENDLIST") } + let playlist = playlistText() + + let lines = playlist.split(whereSeparator: \.isNewline).map(String.init) + let durations = lines.filter { $0.hasPrefix("#EXTINF:") } + .compactMap { Double($0.dropFirst("#EXTINF:".count).split(separator: ",").first ?? "") } + #expect(abs(durations.reduce(0, +) - 16) < 0.6, + "EXTINF sums to \(durations.reduce(0, +)) s for a 16 s source:\n\(playlist)") + + var frames = 0 + for uri in lines where uri.hasSuffix(".mp4") && !uri.hasPrefix("#") { + let data = try Data(contentsOf: mediaURL.deletingLastPathComponent().appendingPathComponent(uri)) + frames += Self.videoSampleCount(data) + } + #expect(frames == Self.videoFrames, + "the listed segments carry \(frames) of \(Self.videoFrames) video frames:\n\(playlist)") + } +} diff --git a/Tests/AetherEngineTests/SequentialLongGOPPlaylistTests.swift b/Tests/AetherEngineTests/SequentialLongGOPPlaylistTests.swift new file mode 100644 index 000000000..b75514f40 --- /dev/null +++ b/Tests/AetherEngineTests/SequentialLongGOPPlaylistTests.swift @@ -0,0 +1,145 @@ +// A sequential origin advertises its segments from the VIDEO cut ledger: a segment's EXTINF is the +// distance between two keyframe-gated opens, and a plan index no keyframe opened is a zero-duration +// hole with no URI. Audio used to be routed by time against the plan instead, so on a source whose +// GOP is longer than the 4 s stride it opened that hole on its own, the video after it went into a +// file the playlist never listed, and AVPlayer stalled at the end of seg0 (a remote MKV with an +// 11.3 s first GOP lost 4 to 11 s). The same hole arrived before seg0's capture had, and the report +// funnel anchored on it, so every later report was refused as out of order and the playlist stayed +// empty. Both are witnessed here by counting what the finished playlist actually carries. +import Foundation +import Testing +@testable import AetherEngine + +private func fixtureURL(_ name: String) -> URL { + URL(fileURLWithPath: #filePath) + .deletingLastPathComponent() + .deletingLastPathComponent() + .deletingLastPathComponent() + .appendingPathComponent("Fixtures") + .appendingPathComponent(name) +} + +private func fixtureExists(_ name: String) -> Bool { + FileManager.default.fileExists(atPath: fixtureURL(name).path) +} + +/// Video samples (track 1) across every fragment of a segment, from the `trun` sample counts. +private func videoSampleCount(_ segment: Data) -> Int { + func u32(_ off: Int) -> UInt32 { + segment.withUnsafeBytes { UInt32(bigEndian: $0.loadUnaligned(fromByteOffset: off, as: UInt32.self)) } + } + func boxes(_ range: Range) -> [(String, Range)] { + var out: [(String, Range)] = [] + var off = range.lowerBound + while off + 8 <= range.upperBound { + let size = Int(u32(off)) + guard size >= 8, off + size <= range.upperBound else { break } + let type = String(bytes: segment[off + 4.. String { (try? String(contentsOf: mediaURL, encoding: .utf8)) ?? "" } + try await waitFor { playlistText().contains("#EXT-X-ENDLIST") } + let playlist = playlistText() + + let lines = playlist.split(whereSeparator: \.isNewline).map(String.init) + let durations = lines.filter { $0.hasPrefix("#EXTINF:") } + .compactMap { Double($0.dropFirst("#EXTINF:".count).split(separator: ",").first ?? "") } + let uris = lines.filter { $0.hasSuffix(".mp4") && !$0.hasPrefix("#") } + #expect(!uris.isEmpty) + #expect(abs(durations.reduce(0, +) - 30) < 0.5, + "EXTINF sums to \(durations.reduce(0, +)) s for a 30 s source:\n\(playlist)") + + var frames = 0 + for uri in uris { + let data = try Data(contentsOf: mediaURL.deletingLastPathComponent().appendingPathComponent(uri)) + frames += videoSampleCount(data) + } + #expect(frames == 720, "the listed segments carry \(frames) of 720 video frames:\n\(playlist)") + } +} + +/// Device log 2026-09-27: an AirPlay receiver had prefetched to seg20, the hop back swapped in a +/// loopback item at 22.8 s, and its first fetch was seg5. The contiguity scan from there read the +/// hole the cutter left at seg7 as a gap, asked for a restart, and the sequential origin's refusal +/// published "Source cannot be repositioned" over a session that had every segment it needed. +@Suite("A sequential origin serves a resident backward target from the cache") +struct SequentialBackwardTargetTests { + + private final class Recorder: @unchecked Sendable { + private let lock = NSLock() + private var _indices: [Int] = [] + func record(_ i: Int) { lock.lock(); _indices.append(i); lock.unlock() } + var indices: [Int] { lock.lock(); defer { lock.unlock() }; return _indices } + } + + @Test("a backward jump across a cutter hole restarts nothing") + func backwardAcrossHoleKeepsProducer() throws { + let cache = SegmentCache(forwardWindow: 60, backwardWindow: 60) + defer { cache.close() } + let recorder = Recorder() + let plan = (0..<30).map { i in + HLSVideoEngine.Segment(startPts: Int64(i) * 4000, endPts: Int64(i + 1) * 4000, + startSeconds: Double(i) * 4.0, durationSeconds: 4.0) + } + let provider = VideoSegmentProvider( + cache: cache, segments: plan, codecsString: "avc1.64001F,fLaC", supplementalCodecs: nil, + resolution: (1280, 720), videoRange: .sdr, frameRate: 23.976, hdcpLevel: nil, + sourceBitrate: 3_000_000, sequentialAppendPlaylist: true, + restartHandler: { recorder.record($0) }) + for i in 0...20 { + let hole = i == 1 || i == 7 + provider.appendSequentialSegmentDuration(index: i, durationSeconds: hole ? 0 : 4) + if !hole { cache.store(index: i, data: Data(repeating: 0x5A, count: 8)) } + } + + _ = provider.mediaSegmentURL(at: 20) + let url = provider.mediaSegmentURL(at: 5) + #expect(url != nil) + #expect(recorder.indices.isEmpty, "restart requested at \(recorder.indices)") + } +} diff --git a/Tests/AetherEngineTests/SharedOutputCoordinatorTests.swift b/Tests/AetherEngineTests/SharedOutputCoordinatorTests.swift new file mode 100644 index 000000000..ec23bf332 --- /dev/null +++ b/Tests/AetherEngineTests/SharedOutputCoordinatorTests.swift @@ -0,0 +1,202 @@ +import Testing +import Foundation +@testable import AetherEngine + +/// Sodalite#175 spike: every stop that released the audio session paused the OTHER engine within +/// 10 ms (4 of 4 on an Apple TV). The coordinator is the one place that knows whether anyone is left. +@Suite("Shared output coordinator (Sodalite#175)") +@MainActor +struct SharedOutputCoordinatorTests { + + private final class Token {} + // Held, not temporary: an ObjectIdentifier of a freed object can be reused by the next allocation. + private let tokenA = Token() + private let tokenB = Token() + private var a: ObjectIdentifier { ObjectIdentifier(tokenA) } + private var b: ObjectIdentifier { ObjectIdentifier(tokenB) } + + @Test("a lone engine leaving is last out and releases as it did before") + func loneLeaveReleases() { + let c = SharedOutputCoordinator() + c.join(a, role: .primary, tag: nil) + #expect(c.leave(a, releasesSession: true) == .lastOut(releaseSession: true)) + #expect(c.isEmpty) + } + + @Test("leaving while another engine plays keeps the session") + func leaveWithOthersRemainingKeepsSession() { + let c = SharedOutputCoordinator() + c.join(a, role: .primary, tag: nil) + c.join(b, role: .secondary, tag: "tile2") + #expect(c.leave(b, releasesSession: true) == .othersRemain(1)) + #expect(!c.isEmpty) + } + + @Test("an opt-in from an earlier leaver is owed to the last one out") + func releaseOwedToLastOut() { + let c = SharedOutputCoordinator() + c.join(a, role: .primary, tag: nil) + c.join(b, role: .secondary, tag: "tile2") + _ = c.leave(a, releasesSession: true) + #expect(c.leave(b, releasesSession: false) == .lastOut(releaseSession: true)) + } + + @Test("the flag at leave time counts, not the one at join") + func releaseFlagAtLeaveCounts() { + let c = SharedOutputCoordinator() + c.join(a, role: .primary, tag: nil) + #expect(c.leave(a, releasesSession: false) == .lastOut(releaseSession: false)) + } + + @Test("the owed release does not survive into the next round") + func owedReleaseResetsWhenEmpty() { + let c = SharedOutputCoordinator() + c.join(a, role: .primary, tag: nil) + _ = c.leave(a, releasesSession: true) + c.join(b, role: .primary, tag: nil) + #expect(c.leave(b, releasesSession: false) == .lastOut(releaseSession: false)) + } + + @Test("leaving twice, or without joining, does nothing") + func leaveOfNonMemberIsInert() { + let c = SharedOutputCoordinator() + #expect(c.leave(a, releasesSession: true) == .notMember) + c.join(a, role: .primary, tag: nil) + _ = c.leave(a, releasesSession: true) + #expect(c.leave(a, releasesSession: true) == .notMember) + } + + @Test("joining twice is one membership") + func joinIsIdempotent() { + let c = SharedOutputCoordinator() + c.join(a, role: .primary, tag: nil) + c.join(a, role: .primary, tag: nil) + #expect(c.leave(a, releasesSession: true) == .lastOut(releaseSession: true)) + } + + @Test("the preferred channel count is the widest source, so a stereo tile cannot downmix 5.1") + func channelsAreMax() { + let c = SharedOutputCoordinator() + c.join(a, role: .primary, tag: nil) + c.join(b, role: .secondary, tag: "tile2") + #expect(c.preferredSourceChannels == nil) + c.noteSourceChannels(6, for: a) + c.noteSourceChannels(2, for: b) + #expect(c.preferredSourceChannels == 6) + _ = c.leave(a, releasesSession: true) + #expect(c.preferredSourceChannels == 2) + } + + @Test("channels noted for a non-member are ignored") + func channelsOfNonMemberIgnored() { + let c = SharedOutputCoordinator() + c.noteSourceChannels(8, for: a) + #expect(c.preferredSourceChannels == nil) + } + + @Test("a deferred criteria reset runs when the last engine leaves") + func deferredResetRunsOnLastOut() { + let c = SharedOutputCoordinator() + var resets = 0 + c.join(a, role: .primary, tag: nil) + c.join(b, role: .secondary, tag: "tile2") + #expect(c.othersActive(besides: a)) + _ = c.leave(a, releasesSession: true) + c.deferCriteriaReset(for: a) { resets += 1 } + #expect(resets == 0) + _ = c.leave(b, releasesSession: true) + #expect(resets == 1) + } + + @Test("an engine that loads again cancels its own deferred reset") + func rejoinCancelsDeferredReset() { + let c = SharedOutputCoordinator() + var resets = 0 + c.join(a, role: .primary, tag: nil) + c.join(b, role: .secondary, tag: "tile2") + _ = c.leave(a, releasesSession: true) + c.deferCriteriaReset(for: a) { resets += 1 } + c.join(a, role: .primary, tag: nil) + _ = c.leave(b, releasesSession: true) + #expect(resets == 0) + } + + @Test("a secondary rejoin keeps the deferred reset") + func secondaryRejoinKeepsDeferredReset() { + let c = SharedOutputCoordinator() + var resets = 0 + c.join(a, role: .primary, tag: nil) + c.join(b, role: .primary, tag: nil) + _ = c.leave(a, releasesSession: true) + c.deferCriteriaReset(for: a) { resets += 1 } + c.join(a, role: .secondary, tag: nil) + _ = c.leave(b, releasesSession: true) + #expect(resets == 0) + _ = c.leave(a, releasesSession: true) + #expect(resets == 1) + } + + @Test("a reset deferred when nobody else is playing runs at once instead of leaking into a later round") + func deferWithNobodyLeftRunsImmediately() { + let c = SharedOutputCoordinator() + var resets = 0 + c.join(a, role: .primary, tag: nil) + _ = c.leave(a, releasesSession: true) + c.deferCriteriaReset(for: a) { resets += 1 } + #expect(resets == 1) + c.join(b, role: .primary, tag: nil) + _ = c.leave(b, releasesSession: true) + #expect(resets == 1) + } + + private actor Journal { + private(set) var entries: [String] = [] + func note(_ entry: String) { entries.append(entry) } + } + + @Test("transitions from any engine run in the order they were asked for") + func transitionsAreOrderedAcrossEngines() async { + let c = SharedOutputCoordinator() + let journal = Journal() + _ = c.enqueueTransition { + try? await Task.sleep(for: .milliseconds(100)) + await journal.note("activation by A") + } + let release = c.enqueueTransition { await journal.note("release by B") } + await release.value + #expect(await journal.entries == ["activation by A", "release by B"]) + } + + // Nonisolated probe: LeaveOutcome must be usable from a nonisolated context (Task 3 needs this) + @Test("LeaveOutcome is usable from nonisolated context") + func leaveOutcomeIsUsableNonisolated() { + #expect(Self.isRelease(.lastOut(releaseSession: true))) + #expect(!Self.isRelease(.lastOut(releaseSession: false))) + #expect(!Self.isRelease(.notMember)) + #expect(!Self.isRelease(.othersRemain(2))) + } + + nonisolated static func isRelease(_ o: SharedOutputCoordinator.LeaveOutcome) -> Bool { + o == .lastOut(releaseSession: true) + } + + @Test("an engine released without stop() no longer holds the session for everyone else") + func releasedOwnerIsPruned() { + let c = SharedOutputCoordinator() + var dropped: Token? = Token() + let droppedID = ObjectIdentifier(dropped!) + c.join(droppedID, role: .secondary, tag: "tile2", owner: dropped) + c.join(a, role: .primary, tag: nil, owner: tokenA) + dropped = nil + #expect(!c.othersActive(besides: a)) + #expect(c.leave(a, releasesSession: true) == .lastOut(releaseSession: true)) + } + + @Test("a member joined without an owner is never pruned") + func ownerlessMemberStays() { + let c = SharedOutputCoordinator() + c.join(a, role: .primary, tag: nil) + c.join(b, role: .secondary, tag: "tile2") + #expect(c.leave(a, releasesSession: true) == .othersRemain(1)) + } +} diff --git a/Tests/AetherEngineTests/SharedOutputEngineWiringTests.swift b/Tests/AetherEngineTests/SharedOutputEngineWiringTests.swift new file mode 100644 index 000000000..f30f17fde --- /dev/null +++ b/Tests/AetherEngineTests/SharedOutputEngineWiringTests.swift @@ -0,0 +1,36 @@ +import Testing +import Foundation +@testable import AetherEngine + +@Suite("Engine wiring to the shared output coordinator (Sodalite#175)") +@MainActor +struct SharedOutputEngineWiringTests { + + @Test("only the last engine out, with a release owed, deactivates the session") + func deactivationFollowsOnlyLastOut() { + #expect(AetherEngine.shouldDeactivateAudioSession(after: .lastOut(releaseSession: true))) + #expect(!AetherEngine.shouldDeactivateAudioSession(after: .lastOut(releaseSession: false))) + #expect(!AetherEngine.shouldDeactivateAudioSession(after: .othersRemain(1))) + #expect(!AetherEngine.shouldDeactivateAudioSession(after: .notMember)) + } + + @Test("a release scheduled by the last engine drops when another engine joined meanwhile") + func pendingReleaseDropsWhenSomeoneJoined() async { + let c = SharedOutputCoordinator() + final class Token {} + let tokenA = Token(), tokenB = Token() + let a = ObjectIdentifier(tokenA), b = ObjectIdentifier(tokenB) + c.join(a, role: .primary, tag: nil) + let outcome = c.leave(a, releasesSession: true) + #expect(outcome == .lastOut(releaseSession: true)) + c.join(b, role: .secondary, tag: "tile2") + #expect(!AetherEngine.releaseStillWanted(coordinatorIsEmpty: c.isEmpty)) + } + + @Test("an engine's log tag is the host's") + func logTagIsHostSet() throws { + let engine = try AetherEngine() + engine.logTag = "tile2" + #expect(engine.logTag == "tile2") + } +} diff --git a/Tests/AetherEngineTests/SharedOutputRoleTests.swift b/Tests/AetherEngineTests/SharedOutputRoleTests.swift new file mode 100644 index 000000000..cadf0d9df --- /dev/null +++ b/Tests/AetherEngineTests/SharedOutputRoleTests.swift @@ -0,0 +1,37 @@ +import Testing +@testable import AetherEngine + +/// Sodalite#175: a multiview tile runs beside the engine that owns the panel and Now Playing. +@Suite("Shared output role (Sodalite#175)") +struct SharedOutputRoleTests { + + @Test("the default role is primary, so a single-instance host is untouched") + func defaultIsPrimary() { + #expect(LoadOptions().sharedOutputRole == .primary) + } + + @Test("a secondary load never writes display criteria") + func secondarySuppressesCriteria() { + var options = LoadOptions() + options.sharedOutputRole = .secondary + #expect(AetherEngine.applyingSharedOutputRole(options).suppressDisplayCriteria) + } + + @Test("a primary load keeps whatever the host asked for") + func primaryKeepsHostChoice() { + #expect(!AetherEngine.applyingSharedOutputRole(LoadOptions()).suppressDisplayCriteria) + #expect(AetherEngine.applyingSharedOutputRole(LoadOptions(suppressDisplayCriteria: true)).suppressDisplayCriteria) + } + + @Test("only a primary may own Now Playing, and only when the host opted in") + func nowPlayingOwnership() { + #expect(AetherEngine.ownsNowPlaying(hostOptIn: true, role: .primary)) + #expect(!AetherEngine.ownsNowPlaying(hostOptIn: true, role: .secondary)) + #expect(!AetherEngine.ownsNowPlaying(hostOptIn: false, role: .primary)) + } + + @Test("the role names the session, so a correction may not flip it") + func roleIsIdentity() { + #expect(SessionOptionCorrection.loadIdentityFields.contains("sharedOutputRole")) + } +} diff --git a/Tests/AetherEngineTests/SidecarPGSDecodeTests.swift b/Tests/AetherEngineTests/SidecarPGSDecodeTests.swift index a371a434a..923fd5896 100644 --- a/Tests/AetherEngineTests/SidecarPGSDecodeTests.swift +++ b/Tests/AetherEngineTests/SidecarPGSDecodeTests.swift @@ -4,23 +4,25 @@ import Foundation /// External PGS sidecar decode (Jellyfin serves external PGS tracks as raw .sup streams). The /// fixture is local-only like the restart-witness clips (extract one with: -/// `ffmpeg -i -map 0:s:0 -c copy Fixtures/external-pgs.sup`); the test skips when -/// absent (CI). Requires FFmpegBuild >= 2.1.3 (sup demuxer in the allowlist). +/// `ffmpeg -i -map 0:s:0 -c copy Fixtures/external-pgs.sup`); the tests are reported +/// as skipped when it is absent (CI). Requires FFmpegBuild >= 2.1.3 (sup demuxer in the allowlist). @Suite("Sidecar PGS decode") struct SidecarPGSDecodeTests { - private var fixtureURL: URL { - URL(fileURLWithPath: #filePath) - .deletingLastPathComponent() // AetherEngineTests - .deletingLastPathComponent() // Tests - .deletingLastPathComponent() // repo root - .appendingPathComponent("Fixtures") - .appendingPathComponent("external-pgs.sup") - } + private static let fixtureURL = URL(fileURLWithPath: #filePath) + .deletingLastPathComponent() // AetherEngineTests + .deletingLastPathComponent() // Tests + .deletingLastPathComponent() // repo root + .appendingPathComponent("Fixtures") + .appendingPathComponent("external-pgs.sup") + + private static let fixtureExists = FileManager.default.fileExists(atPath: fixtureURL.path) + + private static let missingFixture: Comment = "needs Fixtures/external-pgs.sup (extract one with the ffmpeg command above)" - @Test("raw .sup sidecar decodes into image cues with monotonic timing") + @Test("raw .sup sidecar decodes into image cues with monotonic timing", + .enabled(if: SidecarPGSDecodeTests.fixtureExists, SidecarPGSDecodeTests.missingFixture)) func supSidecarDecodes() async throws { - guard FileManager.default.fileExists(atPath: fixtureURL.path) else { return } - let result = try await SubtitleDecoder.decodeFile(url: fixtureURL, httpHeaders: [:], preserveASSMarkup: false) + let result = try await SubtitleDecoder.decodeFile(url: Self.fixtureURL, httpHeaders: [:], preserveASSMarkup: false) #expect(!result.cues.isEmpty) var lastStart = -1.0 for cue in result.cues { @@ -35,10 +37,10 @@ struct SidecarPGSDecodeTests { } } - @Test("OCR fill turns a .sup sidecar's image cues into non-empty text cues") + @Test("OCR fill turns a .sup sidecar's image cues into non-empty text cues", + .enabled(if: SidecarPGSDecodeTests.fixtureExists, SidecarPGSDecodeTests.missingFixture)) func supSidecarOCRFill() async throws { - guard FileManager.default.fileExists(atPath: fixtureURL.path) else { return } - let result = try await SubtitleDecoder.decodeFile(url: fixtureURL, httpHeaders: [:], preserveASSMarkup: false) + let result = try await SubtitleDecoder.decodeFile(url: Self.fixtureURL, httpHeaders: [:], preserveASSMarkup: false) let store = NativeSubtitleCueStore() await SubtitleImageOCR.appendRecognized(cues: Array(result.cues.prefix(8)), language: "eng", to: store) let recognized = store.snapshotCues() diff --git a/Tests/AetherEngineTests/SoftwareDecoderOutputFormatTests.swift b/Tests/AetherEngineTests/SoftwareDecoderOutputFormatTests.swift new file mode 100644 index 000000000..abcc31843 --- /dev/null +++ b/Tests/AetherEngineTests/SoftwareDecoderOutputFormatTests.swift @@ -0,0 +1,144 @@ +import Foundation +import CoreVideo +import CoreMedia +import Testing +import AetherLibavcodec +import AetherLibavutil +@testable import AetherEngine + +/// Audit DEC-107: `SoftwareVideoDecoder` hard-wired its output to 8-bit or 10-bit VIDEO range from the +/// container declaration. `color_range` was never read, so a full-range AV1 / VP9 / HEVC picture was +/// labelled video range and shown crushed, and `bits_per_raw_sample` is 0 for libdav1d and the AV1 / VP9 +/// parsers, so a 10-bit SDR stream was dithered down to 8 bit. The pool format now follows the frame. +@Suite("Software decoder output format follows the frame (DEC-107)") +struct SoftwareDecoderOutputFormatTests { + + // MARK: - The decision + + @Test("full range is carried as a label on 8-bit and on 10-bit output") + func fullRangeIsALabel() { + let eight = SoftwareVideoDecoder.outputFormat( + pixelFormat: AV_PIX_FMT_YUV420P.rawValue, colorRange: AVCOL_RANGE_JPEG, streamIs10Bit: false) + #expect(eight == .init(tenBit: false, fullRange: true)) + #expect(eight.pixelFormatType == kCVPixelFormatType_420YpCbCr8BiPlanarFullRange) + + let ten = SoftwareVideoDecoder.outputFormat( + pixelFormat: AV_PIX_FMT_YUV420P10LE.rawValue, colorRange: AVCOL_RANGE_JPEG, streamIs10Bit: false) + #expect(ten == .init(tenBit: true, fullRange: true)) + #expect(ten.pixelFormatType == kCVPixelFormatType_420YpCbCr10BiPlanarFullRange) + } + + @Test("limited and unspecified range stay video range") + func limitedStaysVideoRange() { + for range in [AVCOL_RANGE_MPEG, AVCOL_RANGE_UNSPECIFIED] { + let format = SoftwareVideoDecoder.outputFormat( + pixelFormat: AV_PIX_FMT_YUV420P.rawValue, colorRange: range, streamIs10Bit: false) + #expect(format == .init(tenBit: false, fullRange: false)) + #expect(format.pixelFormatType == kCVPixelFormatType_420YpCbCr8BiPlanarVideoRange) + } + } + + @Test("a 10-bit frame is 10-bit output even when the stream declared no depth") + func frameDepthWidensTheDeclaration() { + let format = SoftwareVideoDecoder.outputFormat( + pixelFormat: AV_PIX_FMT_YUV420P10LE.rawValue, colorRange: AVCOL_RANGE_MPEG, streamIs10Bit: false) + #expect(format == .init(tenBit: true, fullRange: false)) + #expect(format.swscaleFormat == AV_PIX_FMT_P010LE) + } + + @Test("the stream's own 10-bit verdict is never narrowed by an 8-bit frame") + func declarationIsNeverNarrowed() { + let format = SoftwareVideoDecoder.outputFormat( + pixelFormat: AV_PIX_FMT_YUV420P.rawValue, colorRange: AVCOL_RANGE_MPEG, streamIs10Bit: true) + #expect(format.tenBit) + } + + @Test("a yuvj source stays video range, because swscale converts it itself") + func yuvjIsConvertedBySwscale() { + let format = SoftwareVideoDecoder.outputFormat( + pixelFormat: AV_PIX_FMT_YUVJ420P.rawValue, colorRange: AVCOL_RANGE_JPEG, streamIs10Bit: false) + #expect(format == .init(tenBit: false, fullRange: false)) + } + + // MARK: - Real decodes + + private func firstPixelBuffer(base64: String, formatHint: String?) throws -> CVPixelBuffer { + let data = try #require(Data(base64Encoded: base64, options: .ignoreUnknownCharacters)) + let demuxer = Demuxer() + try demuxer.open(reader: DataIOReader(data: data), formatHint: formatHint) + defer { demuxer.close() } + let videoIndex = demuxer.videoStreamIndex + let stream = try #require(demuxer.stream(at: videoIndex)) + + let box = FrameBox() + let decoder = SoftwareVideoDecoder() + decoder.decodesSingleThreaded = true + try decoder.open(stream: stream) { buffer, _, _ in box.keepFirst(buffer) } + defer { decoder.close() } + + while let pkt = try? demuxer.readPacket() { + if pkt.pointee.stream_index == videoIndex { decoder.decode(packet: pkt) } + var p: UnsafeMutablePointer? = pkt + trackedPacketFree(&p) + } + return try #require(box.first, "the fixture must decode to at least one frame") + } + + private final class FrameBox: @unchecked Sendable { + private let lock = NSLock() + private var buffer: CVPixelBuffer? + func keepFirst(_ candidate: CVPixelBuffer) { lock.withLock { if buffer == nil { buffer = candidate } } } + var first: CVPixelBuffer? { lock.withLock { buffer } } + } + + @Test("a full-range VP9 picture reaches the layer as full range, with its white still at 255") + func fullRangeVP9() throws { + let buffer = try firstPixelBuffer(base64: Self.vp9FullRangeBase64, formatHint: nil) + #expect(CVPixelBufferGetPixelFormatType(buffer) == kCVPixelFormatType_420YpCbCr8BiPlanarFullRange) + + CVPixelBufferLockBaseAddress(buffer, .readOnly) + defer { CVPixelBufferUnlockBaseAddress(buffer, .readOnly) } + let luma = try #require(CVPixelBufferGetBaseAddressOfPlane(buffer, 0)).assumingMemoryBound(to: UInt8.self) + #expect(luma[0] == 255, "no range conversion: the decoder's bytes are copied across and only labelled") + } + + @Test("a 10-bit SDR AV1 picture stays 10-bit instead of being dithered to 8") + func tenBitSDRAV1() throws { + let buffer = try firstPixelBuffer(base64: Self.av1TenBitBase64, formatHint: "mp4") + #expect(CVPixelBufferGetPixelFormatType(buffer) == kCVPixelFormatType_420YpCbCr10BiPlanarVideoRange) + } + + /// 64x64 white, 3 frames, `-color_range pc`. Regenerate: + /// ffmpeg -f lavfi -i "color=c=white:s=64x64:r=10:d=0.3" -vf "scale=out_range=pc:out_color_matrix=bt709,format=yuv420p" \ + /// -c:v libvpx-vp9 -color_range pc -b:v 100k -deadline realtime -cpu-used 8 -g 10 vp9fr.webm + static let vp9FullRangeBase64 = """ + GkXfo59ChoEBQveBAULygQRC84EIQoKEd2VibUKHgQJChYECGFOAZwEAAAAAAAJjEU2bdLpNu4tTq4QVSalmU6yBoU27i1OrhBZU + rmtTrIHYTbuMU6uEElTDZ1OsggEpTbuMU6uEHFO7a1OsggJN7AEAAAAAAABZAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA + AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAVSalmsirXsYMPQkBNgI1M + YXZmNjIuMTIuMTAxV0GNTGF2ZjYyLjEyLjEwMUSJiEBywAAAAAAAFlSua8yuAQAAAAAAAEPXgQFzxYjMgzwgXtS2wJyBACK1nIN1 + bmSIgQCGhVZfVlA5g4EBI+ODhAX14QDglLCBQLqBQJqBAlWwiFWxgQFVuYECElTDZ0CAc3OgY8CAZ8iaRaOHRU5DT0RFUkSHjUxh + dmY2Mi4xMi4xMDFzc9pjwItjxYjMgzwgXtS2wGfIpUWjh0VOQ09ERVJEh5hMYXZjNjIuMjguMTAxIGxpYnZweC12cDlnyKFFo4hE + VVJBVElPTkSHkzAwOjAwOjAwLjMwMDAwMDAwMAAfQ7Z1QJjngQCj6YEAAICCSYNCUAPwA/YGOCQcGEIAACBAAGxb///ZMyRz+Oqo + thxUFRuF7BRUkRNywleB6KhH4xFBDtsK7OkcSQT//+C9lZfb5yBGkxAXMq8VCzCvy+wStgESyCieiKCHbYV2dI4kgNw6QKOTgQBk + AIYAQJKcKElAAANwAABUcKOTgQDIAIYAQJKcLErAAANwAABUcBxTu2uRu4+zgQC3iveBAfGCAa/wgQM= + """ + + /// 64x64 gray, 3 frames, 10-bit 4:2:0, video range, no `bits_per_raw_sample` anywhere. Regenerate: + /// ffmpeg -f lavfi -i "color=c=0x808080:s=64x64:r=10:d=0.3" -vf format=yuv420p10le -c:v libsvtav1 \ + /// -preset 12 -g 10 -pix_fmt yuv420p10le av110.mp4 + static let av1TenBitBase64 = """ + AAAAIGZ0eXBpc29tAAACAGlzb21hdjAxaXNvMm1wNDEAAAAIZnJlZQAAAEttZGF0CgoAAAACr/+JXygIMgwQAK8CCCBBAQAAAv4y + ESgCACSSSRGMAAABAAEAAJwQMhEwAgQJJAAjGAAAAgADAACc6BoBiAAAAyNtb292AAAAbG12aGQAAAAAAAAAAAAAAAAAAAPoAAAB + LAABAAABAAAAAAAAAAAAAAAAAQAAAAAAAAAAAAAAAAAAAAEAAAAAAAAAAAAAAAAAAEAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA + AAAAAAACAAACTXRyYWsAAABcdGtoZAAAAAMAAAAAAAAAAAAAAAEAAAAAAAABLAAAAAAAAAAAAAAAAAAAAAAAAQAAAAAAAAAAAAAA + AAAAAAEAAAAAAAAAAAAAAAAAAEAAAAAAQAAAAEAAAAAAACRlZHRzAAAAHGVsc3QAAAAAAAAAAQAAASwAAAAAAAEAAAAAAcVtZGlh + AAAAIG1kaGQAAAAAAAAAAAAAAAAAACgAAAAMAFXEAAAAAAAtaGRscgAAAAAAAAAAdmlkZQAAAAAAAAAAAAAAAFZpZGVvSGFuZGxl + cgAAAAFwbWluZgAAABR2bWhkAAAAAQAAAAAAAAAAAAAAJGRpbmYAAAAcZHJlZgAAAAAAAAABAAAADHVybCAAAAABAAABMHN0YmwA + AACsc3RzZAAAAAAAAAABAAAAnGF2MDEAAAAAAAAAAQAAAAAAAAAAAAAAAAAAAAAAQABAAEgAAABIAAAAAAAAAAEXTGF2YzYyLjI4 + LjEwMSBsaWJzdnRhdjEAAAAAAAAAAAAY//8AAAAYYXYxQ4EATAAKCgAAAAKv/4BfCAgAAAAKZmllbAEAAAAAEHBhc3AAAAABAAAA + AQAAABRidHJ0AAAAAAAABvoAAAb6AAAAGHN0dHMAAAAAAAAAAQAAAAMAAAQAAAAAFHN0c3MAAAAAAAAAAQAAAAEAAAAcc3RzYwAA + AAAAAAABAAAAAQAAAAMAAAABAAAAIHN0c3oAAAAAAAAAAAAAAAMAAAAaAAAAJgAAAAMAAAAUc3RjbwAAAAAAAAABAAAAMAAAAGJ1 + ZHRhAAAAWm1ldGEAAAAAAAAAIWhkbHIAAAAAAAAAAG1kaXJhcHBsAAAAAAAAAAAAAAAALWlsc3QAAAAlqXRvbwAAAB1kYXRhAAAA + AQAAAABMYXZmNjIuMTIuMTAx + """ +} diff --git a/Tests/AetherEngineTests/SoftwarePacketRecordIOTests.swift b/Tests/AetherEngineTests/SoftwarePacketRecordIOTests.swift new file mode 100644 index 000000000..f85f0e33f --- /dev/null +++ b/Tests/AetherEngineTests/SoftwarePacketRecordIOTests.swift @@ -0,0 +1,125 @@ +import Foundation +import Testing +import AetherLibavcodec +@testable import AetherEngine + +/// Audit PERF-109: a software VOD packet crossed the spool through two write calls, two reads that +/// each allocated, and two payload copies on the way back out. The record layout is pinned by +/// `Issue592PacketEnvelopeTests` and by the cursor tests; these pin what the cheaper I/O must keep. +@Suite("Software packet spool record I/O (audit PERF-109)") +struct SoftwarePacketRecordIOTests { + + private static func makeRoot() throws -> URL { + let root = FileManager.default.temporaryDirectory + .appendingPathComponent("perf109-\(UUID().uuidString)", isDirectory: true) + try FileManager.default.createDirectory(at: root, withIntermediateDirectories: true) + return root + } + + private static func payload(_ size: Int, seed: UInt8) -> Data { + var data = Data(count: size) + data.withUnsafeMutableBytes { raw in + for i in 0.. SoftwarePathEscalation.Availability { SoftwarePathEscalation.Availability( - alreadyEscalated: escalated, preferredDecodePath: path, nativeRemoteHLS: remoteHLS) + alreadyEscalated: escalated, preferredDecodePath: path, nativeRemoteHLS: remoteHLS, + hostAllowsEscalation: hostAllows) } + private static let mediaFailure = SoftwarePathEscalation.Request( + domain: SoftwarePathEscalation.mediaErrorDomain, code: -19602, + message: "item death at a frozen position", positionSeconds: 12) + @Test("A media failure on a fresh native session is escalated") func mediaFailureEscalates() { #expect(SoftwarePathEscalation.shouldEscalate( @@ -71,4 +79,223 @@ struct SoftwarePathEscalationTests { #expect(budget.isSpent) #expect(!budget.take()) } + + /// Audit CORE-1: the rebuild is a full load(), and a stop() landing inside it unwinds that load + /// with a CancellationError. The escalation read every throw as "the software path cannot serve + /// this session" and published `.error` onto an engine the viewer had already left. + @Test("A rebuild superseded by stop() leaves the engine idle, not in error", .timeLimit(.minutes(2))) + @MainActor + func supersededRebuildIsSilent() async throws { + let origin = try ProbeHTTPTestOrigin(data: ProbeTestFixtures.hdr10Plus(), stage: .headers) + defer { origin.stop() } + let engine = try AetherEngine() + engine.loadedURL = try #require(URL(string: "http://127.0.0.1:\(origin.port)/source.mkv")) + + let escalation = Task { @MainActor in + await engine.escalateToSoftwarePath(SoftwarePathEscalation.Request( + domain: SoftwarePathEscalation.mediaErrorDomain, code: 0, + message: "item death at a frozen position", positionSeconds: 0), + expectedGeneration: engine.loadGeneration) + } + // The rebuild's probe is parked on the origin, so the stop lands inside the load. + try await waitFor { origin.blocked.entered } + engine.stop() + origin.stop() + await escalation.value + + #expect(engine.state == .idle) + #expect(engine.errorInfo == nil) + } + + // MARK: - AE#629: a host with its own fallback ladder + + /// The switch is the ask's first form: a declining host gets the failure its ladder reads. + @Test("A host that declines the rung is never escalated") + func hostDeclines() { + #expect(!SoftwarePathEscalation.shouldEscalate( + errorDomain: "CoreMediaErrorDomain", availability: Self.availability(hostAllows: false))) + } + + @Test("The rung is on by default and correctable on a playing session") + func defaultAndCorrectable() { + #expect(LoadOptions().escalatesToSoftwarePath) + var proposed = LoadOptions() + proposed.escalatesToSoftwarePath = false + #expect(SessionOptionCorrection.refusedFields(from: LoadOptions(), to: proposed).isEmpty) + #expect(SessionOptionCorrection.changedFields(from: LoadOptions(), to: proposed) + == ["escalatesToSoftwarePath"]) + } + + @Test("A declined rung spends nothing and says nothing") + @MainActor + func declinedEngineSideSpendsNothing() async throws { + let engine = try AetherEngine() + engine.applySessionOptionCorrection(LoadOptions(escalatesToSoftwarePath: false)) + var events: [SoftwarePathEscalationEvent] = [] + let sub = engine.softwarePathEscalations.sink { events.append($0) } + defer { sub.cancel() } + + await engine.escalateToSoftwarePath(Self.mediaFailure, expectedGeneration: engine.loadGeneration) + + #expect(events.isEmpty) + #expect(!engine.softwarePathEscalationBudget.isSpent) + #expect(engine.softwarePathTakeover == nil) + } + + /// The ask's second form. Measured by the reporter: a host that heard only `videoRoute` could not + /// tell the rescue from a routing decision. + @Test("A taken rung publishes the failure it absorbed, at the position the rebuild resumes", .timeLimit(.minutes(2))) + @MainActor + func takenRungIsPublished() async throws { + let origin = try ProbeHTTPTestOrigin(data: ProbeTestFixtures.hdr10Plus(), stage: .headers) + defer { origin.stop() } + let engine = try AetherEngine() + engine.loadedURL = try #require(URL(string: "http://127.0.0.1:\(origin.port)/source.mkv")) + // AE#629 round 3, measured by the reporter: a mount raised by a load() of the host's own was + // refused while AVPlayer's clock read 12.00 s, the start of the segment under the 15.90 s the + // load was handed. The rebuild resumed at 15.90 s and the event said 12.00 s. + engine.state = .loading + engine.positionUnderReconstruction = 15.9 + var events: [SoftwarePathEscalationEvent] = [] + let sub = engine.softwarePathEscalations.sink { events.append($0) } + defer { sub.cancel() } + + let escalation = Task { @MainActor in await engine.escalateToSoftwarePath(Self.mediaFailure, expectedGeneration: engine.loadGeneration) } + try await waitFor { origin.blocked.entered } + engine.stop() + origin.stop() + await escalation.value + + #expect(events == [SoftwarePathEscalationEvent( + absorbedFailure: PlaybackErrorInfo( + kind: .nativeItemFailed, message: "item death at a frozen position", + underlyingDomain: "CoreMediaErrorDomain", underlyingCode: -19602), + positionSeconds: 15.9, + duringStartup: false)]) + } + + /// Reported as a code reading on AE#629: a rebuild whose load failed after its teardown had + /// already published that failure, and the rung then published the absorbed one on top, so the + /// host saw two `.error`s for one failure and the second contradicted what `load()` threw. + @Test("A rebuild that fails after its teardown surfaces one failure, its own", .timeLimit(.minutes(2))) + @MainActor + func failedRebuildSurfacesOnce() async throws { + let engine = try AetherEngine() + // A port nothing listens on: the rebuild tears the session down and its open is refused. + engine.loadedURL = try #require(URL(string: "http://127.0.0.1:9/source.mkv")) + var errors: [PlaybackErrorInfo?] = [] + let sub = engine.$state.sink { state in + if case .error = state { errors.append(engine.errorInfo) } + } + defer { sub.cancel() } + + await engine.escalateToSoftwarePath(Self.mediaFailure, expectedGeneration: engine.loadGeneration) + + #expect(errors.count == 1) + #expect(errors.first??.kind != .nativeItemFailed) + } + + /// A custom source whose first read parks until the engine cancels it: a load that cannot finish + /// and cannot time out, so the takeover is decided against a startup that is really in flight. + final class ParkedReader: IOReader, @unchecked Sendable { + private let condition = NSCondition() + private var released = false + private var arrivals = 0 + var entered: Bool { condition.withLock { arrivals > 0 } } + + func read(_ buffer: UnsafeMutablePointer?, size: Int32) -> Int32 { + condition.lock(); defer { condition.unlock() } + arrivals += 1 + while !released { condition.wait() } + return -1 + } + func seek(offset: Int64, whence: Int32) -> Int64 { whence == 65536 ? 1 << 20 : offset } + func close() { cancel() } + func cancel() { condition.withLock { released = true; condition.broadcast() } } + func makeIndependentReader() -> IOReader? { nil } + var discImageProbeEnabled: Bool { false } + } + + /// The ask's third form. A load() the rebuild superseded used to throw the same + /// CancellationError as a load the host superseded itself, and the reporter's host read it as a + /// failed load and stopped the rebuilt session. It now follows the rebuild and returns when the + /// rebuild does; before this change the same load threw at once. + @Test("A load followed across a takeover returns when the rebuild comes back", .timeLimit(.minutes(2))) + @MainActor + func followedLoadReturnsOnRebuild() async throws { + let engine = try AetherEngine() + let reader = ParkedReader() + let (rebuildGate, finishRebuild) = AsyncStream.makeStream() + + let hostLoad = Task { @MainActor in + try await engine.load(source: .custom(reader, formatHint: "matroska"), + options: LoadOptions(suppressDisplayCriteria: true)) + } + try await waitFor { reader.entered } + let generation = engine.loadGeneration + #expect(engine.waitingLoadGenerations.contains(generation)) + + // What the escalation's rebuild does at its teardown: arm, then claim for the live generation. + engine.softwarePathRebuild = Task { for await _ in rebuildGate {} } + engine.softwarePathTakeoverArm = generation + engine.claimSoftwarePathTakeover() + engine.stop() + + // The load has unwound its own session and is parked on the rebuild, not finished. + try await waitFor { engine.loadGeneration != generation } + try await Task.sleep(for: .milliseconds(200)) + #expect(engine.waitingLoadGenerations.contains(generation)) + + finishRebuild.finish() + _ = try await hostLoad.value + #expect(engine.waitingLoadGenerations.isEmpty) + } + + /// The other half of the same contract: a rebuild refused before it tore anything down never + /// owned the session, so the waiting load is still the host's and a host stop() ends it with the + /// CancellationError it always threw. Here the refusal is `customSourceNotSeekable`, because the + /// probe that would establish seekability is the one still parked. + @Test("A rebuild refused before its teardown leaves the waiting load to the host", .timeLimit(.minutes(2))) + @MainActor + func refusedRebuildTakesNothingOver() async throws { + let engine = try AetherEngine() + let reader = ParkedReader() + var events: [SoftwarePathEscalationEvent] = [] + let sub = engine.softwarePathEscalations.sink { events.append($0) } + defer { sub.cancel() } + + let hostLoad = Task { @MainActor in + try await engine.load(source: .custom(reader, formatHint: "matroska"), + options: LoadOptions(suppressDisplayCriteria: true)) + } + try await waitFor { reader.entered } + + await engine.escalateToSoftwarePath(Self.mediaFailure, expectedGeneration: engine.loadGeneration) + #expect(events.map(\.duringStartup) == [true]) + #expect(engine.softwarePathTakeover == nil) + #expect(engine.softwarePathTakeoverArm == nil) + #expect(engine.errorInfo?.kind == .nativeItemFailed) + + engine.stop() + await #expect(throws: CancellationError.self) { try await hostLoad.value } + } + + /// A rebuild refused before it tore anything down, or one a host stop() beat to the teardown, + /// never became the session's owner, so the load it would have taken over is still the host's. + @Test("Only the rebuild's own teardown of the failed generation claims the takeover") + @MainActor + func takeoverIsClaimedOnlyByItsOwnTeardown() throws { + let engine = try AetherEngine() + engine.softwarePathRebuild = Task {} + + engine.softwarePathTakeoverArm = engine.loadGeneration &+ 1 + engine.claimSoftwarePathTakeover() + #expect(engine.softwarePathTakeover == nil) + #expect(engine.softwarePathTakeoverArm == nil) + + engine.softwarePathTakeoverArm = engine.loadGeneration + engine.claimSoftwarePathTakeover() + #expect(engine.softwarePathTakeover?.supersededGeneration == engine.loadGeneration) + #expect(engine.softwarePathTakeoverArm == nil) + } } diff --git a/Tests/AetherEngineTests/SoftwareRendererReorderDepthTests.swift b/Tests/AetherEngineTests/SoftwareRendererReorderDepthTests.swift new file mode 100644 index 000000000..eb1c185b3 --- /dev/null +++ b/Tests/AetherEngineTests/SoftwareRendererReorderDepthTests.swift @@ -0,0 +1,239 @@ +import Foundation +import Testing +import AVFoundation +import CoreMedia +import CoreVideo +@testable import AetherEngine + +/// Audit PERF-104: the software renderer held four frames for every decoder, which is 75 MB of +/// IOSurface at 4K P010 that libavcodec and dav1d (presentation order) never needed. Only the +/// VideoToolbox HEVC decoder emits B-frames out of order. The depth follows the decoder, and a frame +/// that arrives after a later one was handed over is dropped and raises the depth instead of reaching +/// the layer out of order. +@Suite("Software renderer reorder depth") +struct SoftwareRendererReorderDepthTests { + + private final class Handed: @unchecked Sendable { + private let lock = NSLock() + private var storage: [Double] = [] + var seconds: [Double] { lock.lock(); defer { lock.unlock() }; return storage } + func append(_ t: SoftwareVideoFrameTime) { lock.lock(); storage.append(t.presentation.seconds); lock.unlock() } + } + + private static func makePixelBuffer(width: Int = 16, height: Int = 16, + format: OSType = kCVPixelFormatType_32BGRA) -> CVPixelBuffer { + var pb: CVPixelBuffer? + CVPixelBufferCreate(kCFAllocatorDefault, width, height, format, + [kCVPixelBufferIOSurfacePropertiesKey: [:]] as CFDictionary, &pb) + return pb! + } + + private static func time(_ ticks: Int) -> CMTime { CMTime(value: Int64(ticks), timescale: 1000) } + + @MainActor + private func feed(_ renderer: SampleBufferRenderer, _ ticks: [Int], drain: Bool = true) -> Handed { + let handed = Handed() + renderer.setFrameEnqueuedObserver { handed.append($0) } + let pixels = Self.makePixelBuffer() + for t in ticks { renderer.enqueue(pixelBuffer: pixels, pts: Self.time(t)) } + if drain { renderer.drainReorderBuffer() } + return handed + } + + @Test("The depth follows the decoder: VideoToolbox keeps four, the presentation-order decoders hold one") + func depthPerDecoder() { + #expect(SampleBufferRenderer.reorderDepth(forHardwareDecoder: true) == 4) + #expect(SampleBufferRenderer.reorderDepth(forHardwareDecoder: false) == 1) + } + + @Test("In-order input at depth one is handed over with one frame of latency") + @MainActor + func inOrderInputHoldsOneFrame() { + let renderer = SampleBufferRenderer() + renderer.setReorderDepth(SampleBufferRenderer.reorderDepth(forHardwareDecoder: false)) + let handed = Handed() + renderer.setFrameEnqueuedObserver { handed.append($0) } + let pixels = Self.makePixelBuffer() + for (i, t) in stride(from: 0, to: 400, by: 40).enumerated() { + renderer.enqueue(pixelBuffer: pixels, pts: Self.time(t)) + #expect(handed.seconds.count == i, "frame \(i) waits for its successor and nothing more") + #expect(renderer.heldFrameCount == min(i + 1, 1)) + } + renderer.drainReorderBuffer() + #expect(handed.seconds == stride(from: 0, to: 400, by: 40).map { Double($0) / 1000 }) + #expect(renderer.outOfOrderFramesDropped == 0) + } + + @Test("A renderer nobody configured keeps holding four frames and sorts a B pyramid") + @MainActor + func defaultDepthStillSortsABPyramid() { + let renderer = SampleBufferRenderer() + // Decode order I0 P4 B2 b1 b3, two pyramids. + let handed = feed(renderer, [0, 160, 80, 40, 120, 320, 240, 200, 280]) + #expect(handed.seconds == [0, 40, 80, 120, 160, 200, 240, 280, 320].map { Double($0) / 1000 }) + #expect(renderer.outOfOrderFramesDropped == 0) + } + + @Test("A VideoToolbox-configured renderer sorts a three deep B pyramid") + @MainActor + func hardwareDepthSortsAThreeDeepPyramid() { + let renderer = SampleBufferRenderer() + renderer.setReorderDepth(SampleBufferRenderer.reorderDepth(forHardwareDecoder: true)) + // Decode order: I0 P4 B2 b1 b3, each gap one frame. + let handed = feed(renderer, [0, 4, 2, 1, 3, 8, 6, 5, 7].map { $0 * 40 }) + #expect(handed.seconds == (0...8).map { Double($0 * 40) / 1000 }) + #expect(renderer.outOfOrderFramesDropped == 0) + } + + @Test("At depth one one local inversion is still sorted") + @MainActor + func depthOneSortsASingleInversion() { + let renderer = SampleBufferRenderer() + renderer.setReorderDepth(1) + let handed = feed(renderer, [30, 10, 20]) + #expect(handed.seconds == [0.010, 0.020, 0.030]) + #expect(renderer.outOfOrderFramesDropped == 0) + } + + @Test("A frame behind one already handed over is dropped once, counted, and raises the depth") + @MainActor + func outOfOrderFrameIsDroppedAndDepthRises() { + let renderer = SampleBufferRenderer() + renderer.setReorderDepth(1) + let handed = Handed() + renderer.setFrameEnqueuedObserver { handed.append($0) } + let pixels = Self.makePixelBuffer() + // 0 4 2 1 3 at depth one: 0 and 2 are handed over, 1 then arrives behind 2. + for t in [0, 4, 2, 1, 3] { renderer.enqueue(pixelBuffer: pixels, pts: Self.time(t * 40)) } + #expect(renderer.outOfOrderFramesDropped == 1) + #expect(renderer.takeCadence().lostBeforeLayer == 1) + renderer.drainReorderBuffer() + #expect(handed.seconds == [0, 0.080, 0.120, 0.160], "the late frame never reached the layer") + + // The depth is now the hardware one: the same shape is sorted rather than dropped. + for t in [10, 14, 12, 11, 13].map({ $0 * 40 }) { renderer.enqueue(pixelBuffer: pixels, pts: Self.time(t)) } + renderer.drainReorderBuffer() + #expect(handed.seconds.suffix(5) == [0.400, 0.440, 0.480, 0.520, 0.560]) + #expect(renderer.outOfOrderFramesDropped == 1) + #expect(zip(handed.seconds, handed.seconds.dropFirst()).allSatisfy { $0 <= $1 }, + "everything the layer was handed is in ascending order") + } + + @Test("The guard also keeps an out of order frame off the layer at the hardware depth") + @MainActor + func guardAppliesAtTheHardwareDepthToo() { + let renderer = SampleBufferRenderer() + let handed = Handed() + renderer.setFrameEnqueuedObserver { handed.append($0) } + let pixels = Self.makePixelBuffer() + for t in [100, 200, 300, 400, 500, 600] { renderer.enqueue(pixelBuffer: pixels, pts: Self.time(t)) } + // 100 and 200 are out by now; 150 is five frames late. + renderer.enqueue(pixelBuffer: pixels, pts: Self.time(150)) + renderer.drainReorderBuffer() + #expect(renderer.outOfOrderFramesDropped == 1) + #expect(handed.seconds == [0.100, 0.200, 0.300, 0.400, 0.500, 0.600]) + } + + /// Stands in for a second enqueuer (a drain, another decoder thread). The frame-enqueued observer + /// runs with no lock held, after a frame reached the layer and before the renderer would have + /// recorded it, which is exactly the window a real second thread lands in. + private final class SecondEnqueuer: @unchecked Sendable { + private let lock = NSLock() + private var armed = true + var renderer: SampleBufferRenderer? + let pixels: CVPixelBuffer + let trigger: Double + let lateTicks: Int + init(pixels: CVPixelBuffer, trigger: Double, lateTicks: Int) { + self.pixels = pixels + self.trigger = trigger + self.lateTicks = lateTicks + } + func frameReachedTheLayer(_ t: SoftwareVideoFrameTime) { + lock.lock() + let fire = armed && abs(t.presentation.seconds - trigger) < 1e-9 + if fire { armed = false } + lock.unlock() + if fire { renderer?.enqueue(pixelBuffer: pixels, pts: SoftwareRendererReorderDepthTests.time(lateTicks)) } + } + } + + @Test("A frame that arrives while another is on its way to the layer is judged against that frame") + @MainActor + func guardCoversAFrameInFlight() { + let renderer = SampleBufferRenderer() + renderer.setReorderDepth(1) + let handed = Handed() + let second = SecondEnqueuer(pixels: Self.makePixelBuffer(), trigger: 0.040, lateTicks: 20) + second.renderer = renderer + renderer.setFrameEnqueuedObserver { t in + handed.append(t) + second.frameReachedTheLayer(t) + } + let pixels = Self.makePixelBuffer() + // The frame at 40 ms leaves the buffer when 80 ms arrives. While it is on its way the second + // enqueuer offers 20 ms, which is already behind it. + for t in [0, 40, 80, 120] { renderer.enqueue(pixelBuffer: pixels, pts: Self.time(t)) } + renderer.drainReorderBuffer() + second.renderer = nil + renderer.setFrameEnqueuedObserver(nil) + + #expect(renderer.outOfOrderFramesDropped == 1) + #expect(handed.seconds == [0, 0.040, 0.080, 0.120], "the late frame never reached the layer") + } + + @Test("A flush while a frame is on its way does not leave the guard standing for the new timeline") + @MainActor + func flushDuringHandoverDoesNotPoisonTheGuard() { + let renderer = SampleBufferRenderer() + renderer.setReorderDepth(1) + let handed = Handed() + let pixels = Self.makePixelBuffer() + // A seek lands (flush) while the frame at 5040 is being handed over. + renderer.setFrameEnqueuedObserver { t in + handed.append(t) + if abs(t.presentation.seconds - 5.040) < 1e-9 { renderer.flush(removingDisplayedImage: false) } + } + for t in [5000, 5040, 5080] { renderer.enqueue(pixelBuffer: pixels, pts: Self.time(t)) } + for t in [1000, 1040, 1080] { renderer.enqueue(pixelBuffer: pixels, pts: Self.time(t)) } + renderer.drainReorderBuffer() + renderer.setFrameEnqueuedObserver(nil) + #expect(renderer.outOfOrderFramesDropped == 0, "the seek backwards is not out of order") + #expect(handed.seconds == [5.000, 5.040, 1.000, 1.040, 1.080], "and the frames after it reach the layer") + } + + @Test("A flush forgets what was handed over, so a seek backwards is not out of order") + @MainActor + func flushResetsTheGuard() { + let renderer = SampleBufferRenderer() + renderer.setReorderDepth(1) + let pixels = Self.makePixelBuffer() + for t in [5000, 5040, 5080] { renderer.enqueue(pixelBuffer: pixels, pts: Self.time(t)) } + renderer.flush(removingDisplayedImage: false) + for t in [1000, 1040, 1080] { renderer.enqueue(pixelBuffer: pixels, pts: Self.time(t)) } + renderer.drainReorderBuffer() + #expect(renderer.outOfOrderFramesDropped == 0) + } + + @Test("Three fewer held frames at 4K P010 is 75 MB of IOSurface") + @MainActor + func heldMemoryAtFourK() throws { + let frame = Self.makePixelBuffer(width: 3840, height: 2160, + format: kCVPixelFormatType_420YpCbCr10BiPlanarVideoRange) + let bytes = CVPixelBufferGetDataSize(frame) + #expect(bytes > 24_000_000, "a 4K P010 frame is ~25 MB") + + func steadyStateHeld(depth: Int) -> Int { + let renderer = SampleBufferRenderer() + renderer.setReorderDepth(depth) + let small = Self.makePixelBuffer() + for i in 0..<12 { renderer.enqueue(pixelBuffer: small, pts: Self.time(i * 40)) } + return renderer.heldFrameCount + } + let before = steadyStateHeld(depth: SampleBufferRenderer.reorderDepth(forHardwareDecoder: true)) + let after = steadyStateHeld(depth: SampleBufferRenderer.reorderDepth(forHardwareDecoder: false)) + #expect(before == 4) + #expect(after == 1) + #expect((before - after) * bytes > 70_000_000) + } +} diff --git a/Tests/AetherEngineTests/SourcePrewarmAdoptionTests.swift b/Tests/AetherEngineTests/SourcePrewarmAdoptionTests.swift index 7317fd4dd..e692fde65 100644 --- a/Tests/AetherEngineTests/SourcePrewarmAdoptionTests.swift +++ b/Tests/AetherEngineTests/SourcePrewarmAdoptionTests.swift @@ -207,7 +207,7 @@ struct SourcePrewarmAdoptionTests { /// write-once rule treats any positive `fileSize` as settled, so a warm that belongs to a /// different response would fix a wrong EOF point for the whole session. The connection that is /// actually serving the session gets to overrule it. - @Test("a connection that states a different size drops the warm") + @Test("a connection that states a different size drops the warm", .timeLimit(.minutes(1))) func sizeMismatchDropsTheWarm() async throws { SourcePrewarmStore.shared.clear() let server = try #require(ThrottledOriginServer(totalSize: fileSize)) @@ -220,9 +220,7 @@ struct SourcePrewarmAdoptionTests { let reader = AVIOReader(url: url(server)) defer { reader.markClosed(); reader.close() } try reader.open() - for _ in 0..<100 where reader.resolvedByteSize != restated { - try? await Task.sleep(nanoseconds: 20_000_000) - } + try await waitFor { reader.resolvedByteSize == restated } #expect(reader.resolvedByteSize == restated, "the session kept the warm's size over the one its own connection stated") diff --git a/Tests/AetherEngineTests/SourcePrewarmPlanTests.swift b/Tests/AetherEngineTests/SourcePrewarmPlanTests.swift index 93f3d7b88..07e2808bc 100644 --- a/Tests/AetherEngineTests/SourcePrewarmPlanTests.swift +++ b/Tests/AetherEngineTests/SourcePrewarmPlanTests.swift @@ -123,6 +123,15 @@ struct SourcePrewarmPlanTests { == .range(start: cuesAbsolute)) } + @Test("a SeekPosition at the top of the 64-bit range is skipped instead of overflowing") + func overflowingSeekPositionIsSkipped() { + let fixture = matroskaHead(entries: [(cuesID, -1)], padTo: 4096) + #expect(SourcePrewarmPlan.trailing(head: fixture.head, total: 63_708_627) == SourcePrewarmPlan.Trailing.none) + let dataStart = Int64(fixture.segmentDataStart) + let mixed = matroskaHead(entries: [(tagsID, Int64.max), (cuesID, 60_000 - dataStart)], padTo: 4096) + #expect(SourcePrewarmPlan.trailing(head: mixed.head, total: 63_000) == .range(start: 60_000)) + } + @Test("a Matroska whose objects all sit inside the warm head needs nothing more") func matroskaWithEverythingInTheHeadNeedsNothing() { let fixture = matroskaHead(entries: [(cuesID, 2048), (tagsID, 4096)], padTo: 64 * 1024) diff --git a/Tests/AetherEngineTests/SourceTimestampBoundsTests.swift b/Tests/AetherEngineTests/SourceTimestampBoundsTests.swift new file mode 100644 index 000000000..777127e96 --- /dev/null +++ b/Tests/AetherEngineTests/SourceTimestampBoundsTests.swift @@ -0,0 +1,192 @@ +import CoreMedia +import Testing +import AetherLibavcodec +@testable import AetherEngine + +/// Audit SEG-1: a crafted Matroska cluster time or a garbage live tfdt must degrade a packet, not +/// trap the pump on Int64 overflow. +struct SourceTimestampBoundsTests { + + private func withPacket(pts: Int64, dts: Int64, duration: Int64 = 0, + _ body: (UnsafeMutablePointer) -> Void) throws { + let packet = try #require(av_packet_alloc()) + defer { + var owned: UnsafeMutablePointer? = packet + av_packet_free(&owned) + } + packet.pointee.pts = pts + packet.pointee.dts = dts + packet.pointee.duration = duration + body(packet) + } + + @Test func extremeSourceTimestampsBecomeUnset() throws { + try withPacket(pts: Int64(1) << 62, dts: -(Int64(1) << 62), duration: -5) { packet in + #expect(SourceTimestampBounds.sanitize(packet)) + #expect(packet.pointee.pts == Int64.min) + #expect(packet.pointee.dts == Int64.min) + #expect(packet.pointee.duration == 0) + } + } + + @Test func plausibleTimestampsPassUntouched() throws { + // An epoch-anchored live tfdt at a 10 MHz timescale sits near 1.8e16. + let epochTfdt: Int64 = 1_790_000_000 * 10_000_000 + try withPacket(pts: epochTfdt + 400_000, dts: epochTfdt, duration: 400_000) { packet in + #expect(!SourceTimestampBounds.sanitize(packet)) + #expect(packet.pointee.pts == epochTfdt + 400_000) + #expect(packet.pointee.dts == epochTfdt) + #expect(packet.pointee.duration == 400_000) + } + try withPacket(pts: Int64.min, dts: -1800) { packet in + #expect(!SourceTimestampBounds.sanitize(packet)) + #expect(packet.pointee.pts == Int64.min) + } + } + + @Test func tickArithmeticSaturatesInsteadOfTrapping() { + let big = Int64(1) << 62 + #expect(SourceTimestampBounds.difference(big, -big) == Int64.max) + #expect(SourceTimestampBounds.difference(-big, big) == Int64.min + 1) + #expect(SourceTimestampBounds.sum(Int64.max, 1) == Int64.max) + #expect(SourceTimestampBounds.sum(Int64.min + 1, -2) == Int64.min + 1) + #expect(SourceTimestampBounds.difference(90_000, 1800) == 88_200) + } + + @Test func rebaseMathSurvivesOppositeSignExtremes() { + let big = Int64(1) << 62 + let shifted = HLSSegmentProducer.rebasedVideoShift( + srcDts: -big, lastSrcDts: big, oldShift: -big, fallbackDurationPts: 1800) + #expect(shifted.continuationDts == Int64.max) + #expect(HLSSegmentProducer.seamDerivedAudioShift(audioBoundarySrcDts: big, seamOutAudioTb: -big) == Int64.max) + #expect(HLSSegmentProducer.resolveVideoSampleDuration( + existingDuration: 0, dts: big, nextDts: -big, fallback: 1800, capTicks: 90_000) == 1800) + } + + // MARK: - The demuxer funnel (audit NAT-101 / DEC-101 / FEA-101 / SUB-105) + + private func sanitized(pts: Int64, dts: Int64, duration: Int64 = 0, _ timeBase: AVRational) + throws -> (changed: Bool, pts: Int64, dts: Int64, duration: Int64) { + var result: (Bool, Int64, Int64, Int64) = (false, 0, 0, 0) + try withPacket(pts: pts, dts: dts, duration: duration) { packet in + let changed = SourceTimestampBounds.sanitize(packet, timeBase: timeBase) + result = (changed, packet.pointee.pts, packet.pointee.dts, packet.pointee.duration) + } + return result + } + + @Test func epochAnchoredTimestampsPassTheFunnel() throws { + let nanos = try sanitized(pts: 1_790_000_000_000_000_000, dts: 1_790_000_000_000_000_000, + duration: 40_000_000, AVRational(num: 1, den: 1_000_000_000)) + #expect(!nanos.changed) + #expect(nanos.pts == 1_790_000_000_000_000_000) + let hundreds = try sanitized(pts: 18_000_000_000_000_000, dts: 18_000_000_000_000_000, + duration: 400_000, AVRational(num: 1, den: 10_000_000)) + #expect(!hundreds.changed) + let mpegts = try sanitized(pts: 8_589_934_591, dts: 8_589_934_590, duration: 3003, + AVRational(num: 1, den: 90_000)) + #expect(!mpegts.changed) + } + + @Test func secondsRuleCatchesWhatTheTickRuleLetsThrough() throws { + let nonUnit = try sanitized(pts: (Int64(1) << 62) - 1, dts: 1000, AVRational(num: 3, den: 1_000_000_000)) + #expect(nonUnit.changed) + #expect(nonUnit.pts == Int64.min) + #expect(nonUnit.dts == 1000) + let ninety = try sanitized(pts: Int64(1) << 61, dts: Int64(1) << 61, AVRational(num: 1, den: 90_000)) + #expect(ninety.pts == Int64.min && ninety.dts == Int64.min) + let coarse = try sanitized(pts: 5_000_000_000, dts: -5_000_000_000, AVRational(num: 1, den: 1)) + #expect(coarse.pts == Int64.min && coarse.dts == Int64.min) + } + + @Test func tickRuleHoldsOnADegenerateTimeBase() throws { + let fine = try sanitized(pts: Int64(1) << 62, dts: (Int64(1) << 62) - 1, AVRational(num: 0, den: 0)) + #expect(fine.pts == Int64.min) + #expect(fine.dts == (Int64(1) << 62) - 1) + let extremes = try sanitized(pts: Int64.max, dts: Int64.min + 1, duration: Int64.max, + AVRational(num: 1, den: 1_000_000_000)) + #expect(extremes.pts == Int64.min && extremes.dts == Int64.min && extremes.duration == 0) + } + + @Test func durationFollowsTheSameRulesAndNeverGoesNegative() throws { + let negative = try sanitized(pts: 0, dts: 0, duration: -1, AVRational(num: 1, den: 1000)) + #expect(negative.duration == 0) + let huge = try sanitized(pts: 0, dts: 0, duration: 10_000_000_000_000, AVRational(num: 1, den: 1000)) + #expect(huge.duration == 0) + let kept = try sanitized(pts: 0, dts: 0, duration: 1_000_000, AVRational(num: 1, den: 1000)) + #expect(!kept.changed && kept.duration == 1_000_000) + } + + @Test func secondPassIsIdempotent() throws { + try withPacket(pts: Int64(1) << 61, dts: 1000, duration: Int64.max) { packet in + let tb = AVRational(num: 1, den: 90_000) + #expect(SourceTimestampBounds.sanitize(packet, timeBase: tb)) + #expect(!SourceTimestampBounds.sanitize(packet, timeBase: tb)) + #expect(packet.pointee.dts == 1000) + } + } + + /// `ticks * num == seconds * den` and 4e9 x (2^31 - 1) < 2^63: the largest tick that passes, times + /// the numerator, stays inside Int64 for every time base libavformat can report. + @Test func noPassingTickOverflowsTheNumerator() { + let values: [Int32] = [1, 3, 1000, 1001, 90_000, 1_000_000_000, Int32.max] + for num in values { + for den in values { + let tb = AVRational(num: num, den: den) + var passing: Int64 = 0 + var failing = SourceTimestampBounds.demuxedMagnitude + while failing - passing > 1 { + let mid = passing + (failing - passing) / 2 + if SourceTimestampBounds.plausible(mid, timeBase: tb) == mid { passing = mid } else { failing = mid } + } + #expect(!passing.multipliedReportingOverflow(by: Int64(num)).overflow, "\(num)/\(den)") + #expect(!(-passing).multipliedReportingOverflow(by: Int64(num)).overflow, "\(num)/\(den)") + #expect(SourceTimestampBounds.plausible(-passing, timeBase: tb) == -passing) + } + } + } + + // MARK: - Checked helpers for values produced after the funnel + + @Test func cmTimeIsInvalidInsteadOfTrapping() { + let nonUnit = AVRational(num: 3, den: 1_000_000_000) + #expect(!SourceTimestampBounds.cmTime(ticks: Int64(1) << 62, timeBase: nonUnit).isValid) + #expect(!SourceTimestampBounds.cmTime(ticks: Int64.min, timeBase: nonUnit).isValid) + #expect(!SourceTimestampBounds.cmTime(ticks: 1, timeBase: AVRational(num: 1, den: 0)).isValid) + let ok = SourceTimestampBounds.cmTime(ticks: 1001, timeBase: nonUnit) + #expect(ok.value == 3003 && ok.timescale == 1_000_000_000) + } + + @Test func roundedTicksRefusesWhatInt64CannotHold() { + #expect(SourceTimestampBounds.roundedTicks(.nan) == nil) + #expect(SourceTimestampBounds.roundedTicks(.infinity) == nil) + #expect(SourceTimestampBounds.roundedTicks(-.infinity) == nil) + #expect(SourceTimestampBounds.roundedTicks(1e300) == nil) + #expect(SourceTimestampBounds.roundedTicks(9.223372036854775807e18) == nil) + #expect(SourceTimestampBounds.roundedTicks(-9.223372036854775808e18) == nil) + #expect(SourceTimestampBounds.roundedTicks(4_999.5) == 5_000) + #expect(SourceTimestampBounds.roundedTicks(-1.4) == -1) + } + + /// Audit NAT-101: the software host's fold. Video at 5 s on 1/1000, then a seam whose offset + /// no stream base can hold, and an offset that converts but walks a far-side packet out of Int64. + @Test func foldShiftDegradesToUnsetInsteadOfTrapping() { + let offsetSeconds = Double(Int64.min + 256) * 0.001 - 5 + let offsetTicks = SourceTimestampBounds.roundedTicks(offsetSeconds / 0.001) + #expect(offsetTicks == nil) + #expect(SourceTimestampBounds.shifted(5000, back: offsetTicks) == Int64.min) + + let audioOffset = SourceTimestampBounds.roundedTicks(8e9 / 1e-9) + #expect(audioOffset == 8_000_000_000_000_000_000) + #expect(SourceTimestampBounds.shifted(-(Int64(1) << 62), back: audioOffset) == Int64.min) + #expect(SourceTimestampBounds.shifted(Int64.min, back: 0) == Int64.min) + #expect(SourceTimestampBounds.shifted(90_000, back: 45_000) == 45_000) + } + + @Test func clampedSecondsPinsNonFiniteAndHugeTimes() { + #expect(SourceTimestampBounds.clampedSeconds(.nan) == 0) + #expect(SourceTimestampBounds.clampedSeconds(.infinity) == SourceTimestampBounds.maxPlausibleSeconds) + #expect(SourceTimestampBounds.clampedSeconds(-.infinity) == -SourceTimestampBounds.maxPlausibleSeconds) + #expect(SourceTimestampBounds.clampedSeconds(12.5) == 12.5) + } +} diff --git a/Tests/AetherEngineTests/SpatialAudioBridgeTests.swift b/Tests/AetherEngineTests/SpatialAudioBridgeTests.swift index e06da9387..b553ab377 100644 --- a/Tests/AetherEngineTests/SpatialAudioBridgeTests.swift +++ b/Tests/AetherEngineTests/SpatialAudioBridgeTests.swift @@ -13,7 +13,7 @@ import AetherLibavutil /// with the encoder's 2048 frames of priming counted in, so the first packet takes the anchor and /// the first content frame plays there. These drive the bridge with a synthetic decoder that /// behaves like the real one on exactly those points. -@Suite("TrueHD Atmos spatial bridge") +@Suite("TrueHD Atmos spatial bridge", .timeLimit(.minutes(2))) struct SpatialAudioBridgeTests { /// Stand-in object decoder: one pushed byte is one 40-frame access unit (TrueHD's 1/1200 s), @@ -75,6 +75,14 @@ struct SpatialAudioBridgeTests { func reset() { skipped = 0; inputFrames = 0; blocks = []; pushes = 0 } } + /// Stand-in decoder that refuses every access unit, as a corrupt or foreign stream would. + final class RejectingDecoder: ObjectAudioDecoding { + struct Refused: Error {} + func push(_ bytes: UnsafeRawBufferPointer) throws { throw Refused() } + func nextBlock() throws -> ObjectAudioDecodedBlock? { nil } + func reset() {} + } + /// Stand-in decoder with something to find: one pushed byte is one 40-frame access unit, /// contiguous from the first push, and the left bed channel carries a 3 ms 1 kHz burst /// starting at input frame `clickFrame`. @@ -171,6 +179,46 @@ struct SpatialAudioBridgeTests { #expect(bridge.feedStats.packetsEmitted == packets.count) } + /// The APAC sample entry is written before any audio, so a decoder that never finds a major sync + /// plays the picture over silence unless the bridge says so itself (AE#641). + @Test("a decoder that returns or accepts nothing is reported once, at the threshold, and a healthy one never") + func decodedNothingIsReportedOnce() throws { + guard #available(macOS 26.0, iOS 26.0, tvOS 26.0, visionOS 26.0, *) else { return } + final class Reports: @unchecked Sendable { + private let lock = NSLock() + private var count = 0 + func add() { lock.lock(); count += 1; lock.unlock() } + var value: Int { lock.lock(); defer { lock.unlock() }; return count } + } + let silent = try SpatialAudioBridge( + srcTimeBase: AVRational(num: 1, den: 1000), layout: .l714, + decoder: SyntheticDecoder(skipAfterReset: .max)) + defer { silent.close() } + let reports = Reports() + silent.onDecoderProducedNothing = { _ in reports.add() } + _ = try feed(silent, startMs: 0, count: AudioBridge.silentFeedPacketThreshold - 1) + #expect(reports.value == 0) + _ = try feed(silent, startMs: 1260, count: 40) + #expect(reports.value == 1) + + let rejecting = try SpatialAudioBridge( + srcTimeBase: AVRational(num: 1, den: 1000), layout: .l714, decoder: RejectingDecoder()) + defer { rejecting.close() } + let rejected = Reports() + rejecting.onDecoderProducedNothing = { _ in rejected.add() } + _ = try feed(rejecting, startMs: 0, count: AudioBridge.silentFeedPacketThreshold + 10) + #expect(rejected.value == 1) + + let healthy = try SpatialAudioBridge( + srcTimeBase: AVRational(num: 1, den: 1000), layout: .l714, + decoder: SyntheticDecoder(skipAfterReset: 5)) + defer { healthy.close() } + let quiet = Reports() + healthy.onDecoderProducedNothing = { _ in quiet.add() } + _ = try feed(healthy, startMs: 0, count: 100) + #expect(quiet.value == 0) + } + @Test("a producer restart re-anchors on the new position") func restartReanchors() throws { guard #available(macOS 26.0, iOS 26.0, tvOS 26.0, visionOS 26.0, *) else { return } @@ -309,15 +357,30 @@ struct SpatialAudioBridgeTests { let asset = AVURLAsset(url: file) let track = try #require(try await asset.loadTracks(withMediaType: .audio).first) + // copyNextSampleBuffer blocks until AVFoundation has decoded the next buffer. Called from the + // cooperative pool it held one of a three-core CI runner's threads while that decode waited + // for one, and the whole test process stalled. A thread of its own cannot starve the pool. + let decoded = await withCheckedContinuation { + (continuation: CheckedContinuation, Never>) in + Thread.detachNewThread { + continuation.resume(returning: Result { try Self.clickOnset(asset: asset, track: track) }) + } + } + let heard = try #require(try decoded.get(), "the click reaches the left channel") + #expect(abs(heard - 0.25) < 0.001, "the click decodes at \(heard) s, not at its source position 0.25 s") + } + + /// Decodes the track to PCM and returns when the left channel first rises above the click + /// threshold, or nil when it never does. Blocks until AVFoundation has decoded that far. + private static func clickOnset(asset: AVURLAsset, track: AVAssetTrack) throws -> Double? { let reader = try AVAssetReader(asset: asset) let output = AVAssetReaderTrackOutput(track: track, outputSettings: [ AVFormatIDKey: kAudioFormatLinearPCM, AVLinearPCMBitDepthKey: 32, AVLinearPCMIsFloatKey: true, AVLinearPCMIsNonInterleaved: false, ]) reader.add(output) - #expect(reader.startReading()) - var onset: Double? - while onset == nil, let buffer = output.copyNextSampleBuffer() { + guard reader.startReading() else { throw reader.error ?? CocoaError(.fileReadUnknown) } + while let buffer = output.copyNextSampleBuffer() { guard let format = CMSampleBufferGetFormatDescription(buffer), let asbd = CMAudioFormatDescriptionGetStreamBasicDescription(format)?.pointee else { continue } let channels = Int(asbd.mChannelsPerFrame) @@ -331,10 +394,9 @@ struct SpatialAudioBridgeTests { blockBufferMemoryAllocator: nil, flags: 0, blockBufferOut: &block) guard let samples = list.mBuffers.mData?.assumingMemoryBound(to: Float.self) else { continue } if let i = (0.. 0.2 }) { - onset = start + Double(i) / 48_000 + return start + Double(i) / 48_000 } } - let heard = try #require(onset, "the click reaches the left channel") - #expect(abs(heard - 0.25) < 0.001, "the click decodes at \(heard) s, not at its source position 0.25 s") + return nil } } diff --git a/Tests/AetherEngineTests/StaleLoadGenerationTests.swift b/Tests/AetherEngineTests/StaleLoadGenerationTests.swift new file mode 100644 index 000000000..beac01908 --- /dev/null +++ b/Tests/AetherEngineTests/StaleLoadGenerationTests.swift @@ -0,0 +1,29 @@ +import Foundation +import Testing +@testable import AetherEngine + +/// A load that a stop() or a newer load() superseded while it was suspended must unwind without +/// touching the engine: the state it would write belongs to its successor. +@Suite("Superseded loads unwind without touching the successor") +@MainActor +struct StaleLoadGenerationTests { + + /// Audit CORE-6: the remote-HLS bypass read `loadGeneration` itself, after load()'s first + /// suspension. A stale load resuming there adopted its successor's generation, so every later + /// guard in the bypass passed, and it mounted its own URL on the shared native host. + @Test("The remote-HLS bypass refuses a generation that has moved") + func bypassRefusesStaleGeneration() async throws { + let engine = try AetherEngine() + let stale = engine.loadGeneration + engine.stop() + + await #expect(throws: CancellationError.self) { + try await engine.loadRemoteHLS( + url: URL(string: "http://127.0.0.1:9/vod.m3u8")!, + options: LoadOptions(nativeRemoteHLS: true), + generation: stale) + } + #expect(engine.nativeHost == nil) + #expect(engine.playbackBackend == .none) + } +} diff --git a/Tests/AetherEngineTests/StaleSessionCallbackTests.swift b/Tests/AetherEngineTests/StaleSessionCallbackTests.swift new file mode 100644 index 000000000..13fda7e65 --- /dev/null +++ b/Tests/AetherEngineTests/StaleSessionCallbackTests.swift @@ -0,0 +1,84 @@ +import Foundation +import Testing +@testable import AetherEngine + +/// Audit LIF-103, Vcore-101, Vcore-102: callbacks a session raises reach the engine through a hop onto +/// the main actor, and a hop that lands after the session ended used to act on whatever owned the +/// engine by then: a stopped engine, or the next load. +@Suite("A callback from an ended session does not reach its successor", .timeLimit(.minutes(2))) +@MainActor +struct StaleSessionCallbackTests { + + private static func customSource() throws -> MediaSource { + .custom(DataIOReader(data: try ProbeTestFixtures.hdr10Plus()), formatHint: "mp4") + } + + private static let mediaFailure = SoftwarePathEscalation.Request( + domain: SoftwarePathEscalation.mediaErrorDomain, code: -19602, + message: "item death at a frozen position", positionSeconds: 12) + + /// LIF-103: Back pressed in the turn between an item death and its escalation job. `stop()` + /// resets neither the options nor the budget, so the escalation ran against a gone session, and + /// its rebuild's `sessionNotReloadable(.noActiveSession)` flipped the stopped engine to `.error`. + @Test("An escalation raised before stop() and run after it does nothing") + func escalationAfterStopIsDropped() async throws { + let engine = try AetherEngine() + engine.loadedURL = try #require(URL(string: "http://127.0.0.1:9/source.mkv")) + let raisedUnder = engine.loadGeneration + var events: [SoftwarePathEscalationEvent] = [] + let sub = engine.softwarePathEscalations.sink { events.append($0) } + defer { sub.cancel() } + + engine.stop() + await engine.escalateToSoftwarePath(Self.mediaFailure, expectedGeneration: raisedUnder) + + #expect(engine.state == .idle) + #expect(engine.errorInfo == nil) + #expect(events.isEmpty) + #expect(!engine.softwarePathEscalationBudget.isSpent) + } + + /// Vcore-102: `@Published` replays its value on subscribe, and the next load subscribes before the + /// fresh item's attach clears it, so a reused host handed the previous item's rejection to the + /// successor's fallback. `-1002` is how AVFoundation fails an item whose URL it cannot load, and a + /// master rejection code at startup. + @Test("A torn-down host replays no master rejection to the next load's subscriber") + func tearDownDropsTheRejection() async throws { + let host = NativeAVPlayerHost() + defer { host.tearDown() } + host.load(url: try #require(URL(string: "aether-unloadable://origin/master.m3u8")), + startPosition: nil, contract: .init()) + try await waitFor { host.pendingDisplayRejection != nil } + + host.tearDown() + var replayed = 0 + let sub = host.$pendingDisplayRejection.compactMap { $0 }.sink { _ in replayed += 1 } + defer { sub.cancel() } + + #expect(replayed == 0) + #expect(host.pendingSoftwarePathEscalation == nil) + } + + /// Vcore-101: the HLS session's callbacks captured only the engine, so a late emitter of a + /// session that had ended wrote its reader stall and its scrub state into the next one. + @Test("A stale session's network phase and scrub state leave the successor alone") + func staleSessionCallbacksAreDropped() async throws { + let engine = try AetherEngine() + defer { engine.stop() } + _ = try await engine.load(source: Self.customSource()) + let ended = try #require(engine.nativeVideoSession) + let networkPhase = try #require(ended.onNetworkPhaseChanged) + let seekState = try #require(ended.onSeekStateChanged) + + _ = try await engine.load(source: Self.customSource()) + #expect(engine.nativeVideoSession !== ended) + networkPhase(.exhausted) + seekState(true, 3) + + let touched = try await waitFor(upTo: .milliseconds(300)) { + if case .stalled = engine.playbackPhase { return true } + return engine.isSeeking + } + #expect(!touched) + } +} diff --git a/Tests/AetherEngineTests/StillColorspaceTests.swift b/Tests/AetherEngineTests/StillColorspaceTests.swift new file mode 100644 index 000000000..5c94fec7b --- /dev/null +++ b/Tests/AetherEngineTests/StillColorspaceTests.swift @@ -0,0 +1,43 @@ +import Testing +import AetherLibavutil +import AetherLibswscale +@testable import AetherEngine + +struct StillColorspaceTests { + + private func described(_ primaries: AVColorPrimaries, _ matrix: AVColorSpace) -> ColorDescription { + ColorDescription(primaries: primaries, transfer: AVCOL_TRC_UNSPECIFIED, matrix: matrix, range: AVCOL_RANGE_MPEG) + } + + @Test("A BT.709 still is converted with the BT.709 matrix, not sws's BT.601 default") + func bt709() { + #expect(FrameDecodeContext.swsColorspace(for: described(AVCOL_PRI_BT709, AVCOL_SPC_BT709)) == SWS_CS_ITU709) + } + + @Test("An untagged still takes the matrix the displayed buffer is tagged with (AE#654)") + func untaggedFollowsPlayback() { + #expect(FrameDecodeContext.swsColorspace(for: .unspecified) == SWS_CS_ITU709) + #expect(FrameDecodeContext.swsColorspace(for: described(AVCOL_PRI_BT470BG, AVCOL_SPC_UNSPECIFIED)) == SWS_CS_ITU601) + } + + @Test("SD and BT.2020 matrices keep their own tables") + func declaredMatrices() { + #expect(FrameDecodeContext.swsColorspace(for: described(AVCOL_PRI_UNSPECIFIED, AVCOL_SPC_SMPTE170M)) == SWS_CS_ITU601) + #expect(FrameDecodeContext.swsColorspace(for: described(AVCOL_PRI_UNSPECIFIED, AVCOL_SPC_BT2020_NCL)) == SWS_CS_BT2020) + } + + @Test("Full range is read from color_range and from the JPEG pixel formats") + func fullRange() throws { + var frame: UnsafeMutablePointer? = av_frame_alloc() + let f = try #require(frame) + defer { av_frame_free(&frame) } + f.pointee.format = AV_PIX_FMT_YUV420P10LE.rawValue + f.pointee.color_range = AVCOL_RANGE_MPEG + #expect(!FrameDecodeContext.isFullRange(f)) + f.pointee.color_range = AVCOL_RANGE_JPEG + #expect(FrameDecodeContext.isFullRange(f)) + f.pointee.color_range = AVCOL_RANGE_UNSPECIFIED + f.pointee.format = AV_PIX_FMT_YUVJ420P.rawValue + #expect(FrameDecodeContext.isFullRange(f)) + } +} diff --git a/Tests/AetherEngineTests/StoppedPumpInFlightReadTests.swift b/Tests/AetherEngineTests/StoppedPumpInFlightReadTests.swift new file mode 100644 index 000000000..096cdd124 --- /dev/null +++ b/Tests/AetherEngineTests/StoppedPumpInFlightReadTests.swift @@ -0,0 +1,119 @@ +import Foundation +import Testing +@testable import AetherEngine + +/// Audit SEG-104 (A): the pump checked for a stop only before it read. A read already in flight when +/// the session stopped the producer (the #79 restart aborts it with `markClosed`, which Matroska, TS +/// and fMP4 report as end of file) came back as source EOF, and the abandoned pump adopted its +/// half-written segment into the cache under the full index, where a later backward seek hits it. +@Suite("A stopped pump does not act on its in-flight read", .serialized) +struct StoppedPumpInFlightReadTests { + + /// Serves the fixture a few bytes at a time and, once armed, parks the read that reaches + /// `blockAt` until released, then answers it with end of file. + private final class ParkingReader: IOReader, @unchecked Sendable { + private let bytes: Data + private let condition = NSCondition() + private var position = 0 + private var blockAt: Int? + private var parked = false + private var released = false + + init(_ bytes: Data) { self.bytes = bytes } + + func arm(blockAt offset: Int) { condition.withLock { blockAt = offset } } + var isParked: Bool { condition.withLock { parked } } + func release() { + condition.lock() + released = true + condition.broadcast() + condition.unlock() + } + + func read(_ buffer: UnsafeMutablePointer?, size: Int32) -> Int32 { + guard let buffer, size > 0 else { return -1 } + condition.lock() + defer { condition.unlock() } + if let blockAt, position >= blockAt { + parked = true + while !released { condition.wait() } + return 0 + } + var n = min(Int(size), 64, bytes.count - position) + if let blockAt { n = min(n, blockAt - position) } + guard n > 0 else { return 0 } + bytes.withUnsafeBytes { raw in _ = memcpy(buffer, raw.baseAddress! + position, n) } + position += n + return Int32(n) + } + + func seek(offset: Int64, whence: Int32) -> Int64 { + condition.withLock { + switch whence { + case 65536: return Int64(bytes.count) + case SEEK_SET: position = Int(max(0, min(offset, Int64(bytes.count)))) + case SEEK_CUR: position = max(0, min(position + Int(offset), bytes.count)) + case SEEK_END: position = max(0, min(bytes.count + Int(offset), bytes.count)) + default: return -1 + } + return Int64(position) + } + } + + func close() {} + var discImageProbeEnabled: Bool { false } + } + + private final class Exit: @unchecked Sendable { + private let lock = NSLock() + private var _reason: HLSSegmentProducer.PumpExitReason? + func set(_ r: HLSSegmentProducer.PumpExitReason) { lock.withLock { _reason = r } } + var reason: HLSSegmentProducer.PumpExitReason? { lock.withLock { _reason } } + } + + /// Start of the SimpleBlock element carrying the 7.2 s frame of the Issue234 fixture (15 s of + /// H.264 at 5 fps in Matroska): past everything the open's stream probe reads, and on an element + /// boundary, so the parked read comes back as a clean end of file rather than a torn element. + private static let parkOffset = 2237 + + @Test("a read that returns end of file after stop() exits as a stop and adopts nothing", + .timeLimit(.minutes(1))) + func eofAfterStopAdoptsNothing() async throws { + let data = try #require(Data(base64Encoded: Issue234SideReaderSeekAnchorTests.fixtureBase64, + options: .ignoreUnknownCharacters)) + let reader = ParkingReader(data) + let demuxer = Demuxer() + try demuxer.open(reader: reader, formatHint: "matroska") + defer { demuxer.close() } + // Runs before the close: a wait the time limit cancels must not leave the parked read + // holding the access lock that close() waits on. + defer { reader.release() } + reader.arm(blockAt: Self.parkOffset) + + let videoIndex = demuxer.videoStreamIndex + let stream = try #require(demuxer.stream(at: videoIndex)) + let cache = SegmentCache(forwardWindow: 4, backwardWindow: 4) + defer { cache.close() } + let producer = try HLSSegmentProducer( + demuxer: demuxer, videoStreamIndex: videoIndex, + video: .init(codecpar: UnsafePointer(stream.pointee.codecpar), + timeBase: stream.pointee.time_base, codecTagOverride: nil), + cache: cache, videoFallbackDurationPts: 200, desiredFirstVideoTfdtPts: 0, + segmentBoundaries: [0, 15_000]) + let exit = Exit() + producer.onPumpFinished = { exit.set($0) } + producer.start() + + try await waitFor { reader.isParked } + #expect(producer.packetsWrittenCount > 0, "the segment in flight has to be partly written") + producer.stop() + reader.release() + try await waitFor { exit.reason != nil } + + let reason = try #require(exit.reason) + var exitedAsStop = false + if case .stopRequested = reason { exitedAsStop = true } + #expect(exitedAsStop, "a stopped pump exited as \(reason)") + #expect(cache.count == 0, "the abandoned pump adopted its partial segment") + } +} diff --git a/Tests/AetherEngineTests/StreamFormatReportTests.swift b/Tests/AetherEngineTests/StreamFormatReportTests.swift new file mode 100644 index 000000000..2426b1950 --- /dev/null +++ b/Tests/AetherEngineTests/StreamFormatReportTests.swift @@ -0,0 +1,111 @@ +import Testing +import CoreVideo +import AetherLibavutil +import AetherLibavcodec +@testable import AetherEngine + +// AE#658: the stream facts a stats panel shows, in libav's names plus the labels a viewer reads. +@Suite("Stream format report (AE#658)") +struct StreamFormatReportTests { + + @Test("HDR10 HEVC reads as 10-bit BT.2020 PQ, Main 10") + func hdr10() { + let f = VideoStreamFormat( + pixelFormat: AV_PIX_FMT_YUV420P10LE, declaredBitDepth: 0, + color: ColorDescription(primaries: AVCOL_PRI_BT2020, transfer: AVCOL_TRC_SMPTE2084, + matrix: AVCOL_SPC_BT2020_NCL, range: AVCOL_RANGE_MPEG), + codecID: AV_CODEC_ID_HEVC, profile: AV_PROFILE_HEVC_MAIN_10) + #expect(f.pixelFormat == "yuv420p10le") + #expect(f.bitDepth == 10) + #expect(f.profile == "Main 10") + #expect(f.colorPrimariesLabel == "BT.2020") + #expect(f.transferLabel == "PQ (SMPTE ST 2084)") + #expect(f.matrixLabel == "BT.2020 NCL") + #expect(f.rangeLabel == "Limited") + } + + @Test("an untagged stream stays untagged instead of reading as BT.709") + func untagged() { + let f = VideoStreamFormat( + pixelFormat: AV_PIX_FMT_YUV420P, declaredBitDepth: 0, color: .unspecified, + codecID: AV_CODEC_ID_H264, profile: AV_PROFILE_UNKNOWN) + #expect(f.bitDepth == 8) + #expect(f.colorPrimaries == nil) + #expect(f.transfer == nil) + #expect(f.matrix == nil) + #expect(f.range == nil) + #expect(f.profile == nil) + } + + @Test("HLG and full range get their viewer names") + func hlgFullRange() { + let f = VideoStreamFormat( + pixelFormat: AV_PIX_FMT_YUV420P10LE, declaredBitDepth: 0, + color: ColorDescription(primaries: AVCOL_PRI_BT2020, transfer: AVCOL_TRC_ARIB_STD_B67, + matrix: AVCOL_SPC_BT2020_NCL, range: AVCOL_RANGE_JPEG), + codecID: AV_CODEC_ID_HEVC, profile: AV_PROFILE_UNKNOWN) + #expect(f.transferLabel == "HLG") + #expect(f.rangeLabel == "Full") + } + + @Test("an unknown pixel format falls back to the declared sample depth") + func declaredDepthFallback() { + let f = VideoStreamFormat( + pixelFormat: AV_PIX_FMT_NONE, declaredBitDepth: 12, color: .unspecified, + codecID: AV_CODEC_ID_HEVC, profile: AV_PROFILE_UNKNOWN) + #expect(f.pixelFormat == nil) + #expect(f.bitDepth == 12) + } + + @Test("a name without a label passes through unchanged") + func unknownLabels() { + #expect(VideoStreamFormat.primariesLabel("film") == "film") + #expect(VideoStreamFormat.transferLabel("log100") == "log100") + } + + @Test("display buffers read as their four-character codes") + func pixelBuffer() { + let nv12 = DecodedVideoFormat.fourCC(kCVPixelFormatType_420YpCbCr8BiPlanarVideoRange) + let p010 = DecodedVideoFormat.fourCC(kCVPixelFormatType_420YpCbCr10BiPlanarVideoRange) + #expect(nv12 == "420v") + #expect(p010 == "x420") + #expect(DecodedVideoFormat(frame: .init(pixelFormat: nil, bitDepth: nil, colorPrimaries: nil, + transfer: nil, matrix: nil, range: nil, profile: nil), + pixelBufferFormat: p010).pixelBufferLabel == "P010 (x420)") + #expect(DecodedVideoFormat.fourCC(0x0000_0001) == "0x1") + } + + @Test("a VideoToolbox buffer maps back to the libav format it holds") + func libavMapping() { + #expect(DecodedVideoFormat.libavPixelFormat( + forPixelBufferType: kCVPixelFormatType_420YpCbCr8BiPlanarVideoRange) == AV_PIX_FMT_NV12) + #expect(DecodedVideoFormat.libavPixelFormat( + forPixelBufferType: kCVPixelFormatType_420YpCbCr10BiPlanarVideoRange) == AV_PIX_FMT_P010LE) + #expect(DecodedVideoFormat.libavPixelFormat( + forPixelBufferType: kCVPixelFormatType_32BGRA) == AV_PIX_FMT_NONE) + } + + @Test("DTS:X and TrueHD Atmos surface through the profile name") + func immersiveProfiles() { + #expect(VideoStreamFormat.profileName( + codecID: AV_CODEC_ID_DTS, profile: AV_PROFILE_DTS_HD_MA_X) == "DTS-HD MA + DTS:X") + #expect(VideoStreamFormat.profileName( + codecID: AV_CODEC_ID_DTS, profile: AV_PROFILE_DTS_HD_MA) == "DTS-HD MA") + #expect(VideoStreamFormat.profileName( + codecID: AV_CODEC_ID_TRUEHD, profile: AV_PROFILE_TRUEHD_ATMOS) == "Dolby TrueHD + Dolby Atmos") + #expect(VideoStreamFormat.profileName(codecID: AV_CODEC_ID_AAC, profile: AV_PROFILE_UNKNOWN) == nil) + } + + @Test("sample format and channel layout use libav's names") + func audioNames() { + #expect(Demuxer.sampleFormatName(AV_SAMPLE_FMT_FLTP.rawValue) == "fltp") + #expect(Demuxer.sampleFormatName(AV_SAMPLE_FMT_S32.rawValue) == "s32") + #expect(Demuxer.sampleFormatName(AV_SAMPLE_FMT_NONE.rawValue) == nil) + + var layout = AVChannelLayout() + av_channel_layout_default(&layout, 6) + #expect(Demuxer.channelLayoutDescription(&layout) == "5.1") + av_channel_layout_uninit(&layout) + #expect(Demuxer.channelLayoutDescription(&layout) == nil) + } +} diff --git a/Tests/AetherEngineTests/StreamingBufferBoundTests.swift b/Tests/AetherEngineTests/StreamingBufferBoundTests.swift new file mode 100644 index 000000000..93b56c387 --- /dev/null +++ b/Tests/AetherEngineTests/StreamingBufferBoundTests.swift @@ -0,0 +1,79 @@ +import Testing +import Foundation +@testable import AetherEngine + +/// Audit DMX-107: the forward-only reader pauses its transfer at a high water and ends it at twice +/// that, because the pause is advisory. The end was a bare EIO, indistinguishable from a dropped +/// connection. It now carries a typed cause, and memory stays bounded either way. +@Suite("Forward-only streaming buffer bound") +struct StreamingBufferBoundTests { + + private func drain(_ reader: AVIOReader) -> (read: Int, last: Int32) { + let chunk = 64 * 1024 + let buf = UnsafeMutablePointer.allocate(capacity: chunk) + defer { buf.deallocate() } + var read = 0 + while true { + let n = reader.read(into: buf, size: Int32(chunk)) + if n <= 0 { return (read, n) } + read += Int(n) + } + } + + @Test("a transport that delivers past the hard cap ends the read with a typed cause", + .timeLimit(.minutes(1))) + func hardCapIsTyped() throws { + let body = 64 * 1024 * 1024 + let maybe = ThrottledOriginServer(totalSize: Int64(body), throttleUs: 0) + let server = try #require(maybe) + defer { server.stop() } + // One delivery is larger than twice this, so the cap is passed without depending on + // whether the transport honours the pause that comes first. The peak is then about one + // URLSession delivery, which measured 2.4 MB on a CI runner, so the bound is a quarter of + // the body rather than a guess at the delivery size. + let reader = AVIOReader(url: URL(string: "http://127.0.0.1:\(server.port)/archive.ts")!, + sequentialOnly: true, streamHighWater: 1024) + defer { reader.markClosed(); reader.close() } + try reader.open() + + let result = drain(reader) + #expect(result.last == FFmpegErr.eio, "a source ended at the cap is lost, not finished") + #expect(reader.lastReadFailure == .originIgnoresFlowControl) + #expect(result.read < body, "the cap let the whole body through") + #expect(reader.streamPeakBufferBytesForTesting < body / 4, + "the buffer held \(reader.streamPeakBufferBytesForTesting / 1024) KB") + } + + @Test("a body that finishes inside the bound is EOF with no failure", .timeLimit(.minutes(1))) + func completeBodyHasNoFailure() throws { + let maybe = ThrottledOriginServer(totalSize: 2 * 1024 * 1024, throttleUs: 0) + let server = try #require(maybe) + defer { server.stop() } + let reader = AVIOReader(url: URL(string: "http://127.0.0.1:\(server.port)/archive.ts")!, + sequentialOnly: true) + defer { reader.markClosed(); reader.close() } + try reader.open() + + let result = drain(reader) + #expect(result.read == 2 * 1024 * 1024) + #expect(result.last == FFmpegErr.eof) + #expect(reader.lastReadFailure == nil) + } + + @Test("a dropped connection is EIO without a cause of ours", .timeLimit(.minutes(1))) + func droppedConnectionHasNoCause() throws { + let total: Int64 = 8 * 1024 * 1024 + let maybe = ThrottledOriginServer(totalSize: total, throttleUs: 0, + respond: { _, _, _ in .serveThenDrop(afterBytes: 1024 * 1024) }) + let server = try #require(maybe) + defer { server.stop() } + let reader = AVIOReader(url: URL(string: "http://127.0.0.1:\(server.port)/archive.ts")!, + sequentialOnly: true) + defer { reader.markClosed(); reader.close() } + try reader.open() + + let result = drain(reader) + #expect(result.last == FFmpegErr.eio) + #expect(reader.lastReadFailure == nil, "a lost connection is the network's, not the origin's flow control") + } +} diff --git a/Tests/AetherEngineTests/StreamingBufferTrimTests.swift b/Tests/AetherEngineTests/StreamingBufferTrimTests.swift new file mode 100644 index 000000000..22c970bf3 --- /dev/null +++ b/Tests/AetherEngineTests/StreamingBufferTrimTests.swift @@ -0,0 +1,85 @@ +import Testing +import Foundation +@testable import AetherEngine + +/// Audit PERF-102: the forward-only reader trimmed its consumed head with `Data.subdata`, which +/// copied the whole 32-64 MB still ahead of the cut on every read. The buffer now holds the chunks +/// as they were delivered (AE#619), so a trim advances an offset and moves nothing. +@Suite("Forward-only streaming buffer") +struct StreamingBufferTrimTests { + + private let total = 6 * 1024 * 1024 + + private func makeReader(_ server: ThrottledOriginServer) -> AVIOReader { + AVIOReader(url: URL(string: "http://127.0.0.1:\(server.port)/archive.ts")!, sequentialOnly: true) + } + + @Test("unaligned reads return exactly the bytes the origin sent", .timeLimit(.minutes(2))) + func byteExactAcrossUnalignedReads() async throws { + let maybe = ThrottledOriginServer(totalSize: Int64(total), throttleUs: 0, patternedBody: true) + let server = try #require(maybe) + defer { server.stop() } + let reader = makeReader(server) + defer { reader.markClosed(); reader.close() } + try reader.open() + // Everything is resident before the first read, so each trim below runs against the whole body. + try await waitFor { reader.streamPeakBufferBytesForTesting >= total } + + let expected = (0...allocate(capacity: sizes.max()!) + defer { buf.deallocate() } + var offset = 0 + var turn = 0 + while offset < total { + let want = sizes[turn % sizes.count] + turn += 1 + let n = Int(reader.read(into: buf, size: Int32(want))) + #expect(n > 0, "read failed at \(offset)") + if n <= 0 { return } + let matches = expected.withUnsafeBufferPointer { memcmp($0.baseAddress! + offset, buf, n) == 0 } + #expect(matches, "\(n) bytes at \(offset) differ from the origin's") + if !matches { return } + offset += n + } + #expect(offset == total) + #expect(reader.read(into: buf, size: 4096) == FFmpegErr.eof) + } + + #if DEBUG + @Test("a trim releases what it passed and moves nothing it keeps", .timeLimit(.minutes(2))) + func trimMovesNothing() async throws { + let maybe = ThrottledOriginServer(totalSize: Int64(total), throttleUs: 0) + let server = try #require(maybe) + defer { server.stop() } + let reader = makeReader(server) + defer { reader.markClosed(); reader.close() } + try reader.open() + try await waitFor { reader.streamPeakBufferBytesForTesting >= total } + + // Everything is resident and nothing consumed yet, so this is every chunk the body arrived in. + let before = reader.streamChunkAddressesForTesting + #expect(before.count > 1, "the body arrived as one chunk, so this could not tell a copy from a slice") + + let chunk = 262_000 + let buf = UnsafeMutablePointer.allocate(capacity: chunk) + defer { buf.deallocate() } + // A chunk is released once the read position is a lookback past its end, and how large + // URLSession makes the chunks is not this test's to know (larger on a CI runner), so read + // until one goes. The body is the bound: if it ends first, nothing was released and that fails. + var read = 0 + var after = before + while after.count >= before.count, read < total { + let n = Int(reader.read(into: buf, size: Int32(chunk))) + #expect(n > 0) + if n <= 0 { return } + read += n + after = reader.streamChunkAddressesForTesting + } + #expect(after.count < before.count, "the trim released nothing before the body ended") + #expect(after.count > 1) + #expect(Array(before.suffix(after.count)) == after, + "a trim reallocated the chunks it kept: the buffer was copied, not sliced") + } + #endif +} diff --git a/Tests/AetherEngineTests/SubtitleFrameCompositorTests.swift b/Tests/AetherEngineTests/SubtitleFrameCompositorTests.swift index 747826fcb..ac6bf6a8e 100644 --- a/Tests/AetherEngineTests/SubtitleFrameCompositorTests.swift +++ b/Tests/AetherEngineTests/SubtitleFrameCompositorTests.swift @@ -18,6 +18,32 @@ struct SubtitleFrameCompositorTests { #expect(SubtitleFrameCompositor.activeCues(in: cues, at: 12.0).isEmpty) } + @Test("software PiP delay and advance use media time for both subtitle channels") + func adjustedCueWindow() { + let cues = [cue(1, 10, 12), cue(2, 10, 11), cue(3, 12, 14)] + #expect(SubtitleFrameCompositor.activeCues(in: cues, at: 10.5, delaySeconds: 1.5).isEmpty) + #expect(SubtitleFrameCompositor.activeCues(in: cues, at: 11.5, delaySeconds: 1.5).map(\.id) == [1, 2]) + #expect(SubtitleFrameCompositor.activeCues(in: cues, at: 13.5, delaySeconds: 1.5).map(\.id) == [3]) + #expect(SubtitleFrameCompositor.activeCues(in: cues, at: 8.5, delaySeconds: -1.5).map(\.id) == [1, 2]) + #expect(SubtitleFrameCompositor.activeCues(in: cues, at: .nan).isEmpty) + #expect(SubtitleFrameCompositor.activeCues(in: cues, at: 11, delaySeconds: .infinity).isEmpty) + } + + @MainActor + @Test("software subtitle preference survives PiP transitions and rejects invalid values") + func delayPreference() throws { + let engine = try AetherEngine() + engine.setSoftwareSubtitleDelay(1.5) + engine.pictureInPictureActive = true + engine.pictureInPictureActive = false + engine.setSoftwareSubtitleDelay(.nan) + #expect(engine.softwareSubtitleDelaySeconds == 1.5) + engine.setSoftwareSubtitleDelay(-0.5) + #expect(engine.softwareSubtitleDelaySeconds == -0.5) + engine.stop() + #expect(engine.softwareSubtitleDelaySeconds == -0.5) + } + @Test("text layout scales with frame height and keeps a safe bottom margin") func textLayoutScales() { let layout = SubtitleFrameCompositor.textLayout(frameWidth: 1920, frameHeight: 1080) @@ -70,11 +96,11 @@ struct SubtitleFrameCompositorTests { CVPixelBufferUnlockBaseAddress(buffer, []) // Disabled: passthrough must be the same instance. - compositor.update(cues: [SubtitleCue(id: 1, startTime: 0, endTime: 10, body: .text("HELLO"))], enabled: false) + compositor.update(cues: [SubtitleCue(id: 1, startTime: 0, endTime: 10, body: .text("HELLO"))], enabled: false, delaySeconds: 0) #expect(compositor.composite(buffer, ptsSeconds: 5) === buffer) // Enabled with an active cue: output keeps the format and the bottom region gains bright pixels. - compositor.update(cues: [SubtitleCue(id: 1, startTime: 0, endTime: 10, body: .text("HELLO"))], enabled: true) + compositor.update(cues: [SubtitleCue(id: 1, startTime: 0, endTime: 10, body: .text("HELLO"))], enabled: true, delaySeconds: 0) let out = compositor.composite(buffer, ptsSeconds: 5) #expect(CVPixelBufferGetPixelFormatType(out) == kCVPixelFormatType_420YpCbCr8BiPlanarVideoRange) #expect(out !== buffer) @@ -94,4 +120,92 @@ struct SubtitleFrameCompositorTests { // No active cue at this PTS: passthrough again. #expect(compositor.composite(buffer, ptsSeconds: 20) === buffer) } + + @Test("rendered PiP frames apply timing changes without a cue or transport update") + func compositeAppliesDelay() throws { + let compositor = SubtitleFrameCompositor() + let buffer = try blackFrame(kCVPixelFormatType_420YpCbCr8BiPlanarVideoRange) + let cues = [cue(1, 10, 12)] + compositor.update(cues: cues, enabled: true, delaySeconds: 1.5) + #expect(compositor.composite(buffer, ptsSeconds: 10.5) === buffer) + #expect(compositor.composite(buffer, ptsSeconds: 11.5) !== buffer) + #expect(compositor.composite(buffer, ptsSeconds: 13.5) === buffer) + compositor.update(cues: cues, enabled: true, delaySeconds: -1.5) + #expect(compositor.composite(buffer, ptsSeconds: 8.5) !== buffer) + compositor.update(cues: cues, enabled: false, delaySeconds: -1.5) + #expect(compositor.composite(buffer, ptsSeconds: 8.5) === buffer) + } + + private func blackFrame(_ format: OSType) throws -> CVPixelBuffer { + var pb: CVPixelBuffer? + let attrs: [CFString: Any] = [kCVPixelBufferIOSurfacePropertiesKey: [:] as CFDictionary] + CVPixelBufferCreate(kCFAllocatorDefault, 720, 480, format, attrs as CFDictionary, &pb) + return try #require(pb) + } + + private func attachment(_ buffer: CVPixelBuffer, _ key: CFString) -> CFTypeRef? { + CVBufferCopyAttachment(buffer, key, nil) + } + + /// Audit DEC-3: the renderer describes the frame from the delivered buffer's attachments, so a + /// composited frame that dropped them showed anamorphic content at coded size and HDR as SDR + /// for exactly as long as a cue was on screen. + @Test("a composited frame keeps the source's pixel aspect ratio and colour tags") + func compositedFrameKeepsSourceAttachments() throws { + let compositor = SubtitleFrameCompositor() + compositor.update(cues: [SubtitleCue(id: 1, startTime: 0, endTime: 10, body: .text("HELLO"))], enabled: true, delaySeconds: 0) + + let hdr = try blackFrame(kCVPixelFormatType_420YpCbCr10BiPlanarVideoRange) + let aspect: NSDictionary = [ + kCVImageBufferPixelAspectRatioHorizontalSpacingKey: 32, + kCVImageBufferPixelAspectRatioVerticalSpacingKey: 27, + ] + CVBufferSetAttachment(hdr, kCVImageBufferPixelAspectRatioKey, aspect, .shouldPropagate) + CVBufferSetAttachment(hdr, kCVImageBufferColorPrimariesKey, kCVImageBufferColorPrimaries_ITU_R_2020, .shouldPropagate) + CVBufferSetAttachment(hdr, kCVImageBufferTransferFunctionKey, kCVImageBufferTransferFunction_SMPTE_ST_2084_PQ, .shouldPropagate) + CVBufferSetAttachment(hdr, kCVImageBufferYCbCrMatrixKey, kCVImageBufferYCbCrMatrix_ITU_R_2020, .shouldPropagate) + + let out = compositor.composite(hdr, ptsSeconds: 5) + #expect(out !== hdr) + let par = attachment(out, kCVImageBufferPixelAspectRatioKey) as? NSDictionary + #expect(par?[kCVImageBufferPixelAspectRatioHorizontalSpacingKey] as? Int == 32) + #expect(par?[kCVImageBufferPixelAspectRatioVerticalSpacingKey] as? Int == 27) + #expect(attachment(out, kCVImageBufferTransferFunctionKey) as? String + == kCVImageBufferTransferFunction_SMPTE_ST_2084_PQ as String) + #expect(attachment(out, kCVImageBufferColorPrimariesKey) as? String + == kCVImageBufferColorPrimaries_ITU_R_2020 as String) + #expect(attachment(out, kCVImageBufferYCbCrMatrixKey) as? String + == kCVImageBufferYCbCrMatrix_ITU_R_2020 as String) + + let space = SubtitleFrameCompositor.renderColorSpace(for: hdr) + #expect(space.name == CGColorSpace.itur_2100_PQ) + } + + @Test("a recycled output buffer does not keep an earlier source's pixel aspect ratio") + func recycledBufferDropsStaleAspect() throws { + let compositor = SubtitleFrameCompositor() + compositor.update(cues: [SubtitleCue(id: 1, startTime: 0, endTime: 10, body: .text("HELLO"))], enabled: true, delaySeconds: 0) + let format = kCVPixelFormatType_420YpCbCr8BiPlanarVideoRange + + let anamorphic = try blackFrame(format) + let aspect: NSDictionary = [ + kCVImageBufferPixelAspectRatioHorizontalSpacingKey: 32, + kCVImageBufferPixelAspectRatioVerticalSpacingKey: 27, + ] + CVBufferSetAttachment(anamorphic, kCVImageBufferPixelAspectRatioKey, aspect, .shouldPropagate) + // Several rounds, so the pool hands a buffer that already held the ratio back out. + for _ in 0..<4 { _ = compositor.composite(anamorphic, ptsSeconds: 5) } + + let square = try blackFrame(format) + for _ in 0..<4 { + let out = compositor.composite(square, ptsSeconds: 5) + #expect(attachment(out, kCVImageBufferPixelAspectRatioKey) == nil) + } + } + + @Test("an untagged source renders through BT.709") + func untaggedSourceFallsBackTo709() throws { + let space = SubtitleFrameCompositor.renderColorSpace(for: try blackFrame(kCVPixelFormatType_420YpCbCr8BiPlanarVideoRange)) + #expect(space.name == CGColorSpace.itur_709) + } } diff --git a/Tests/AetherEngineTests/SubtitlePacketStoreScalingTests.swift b/Tests/AetherEngineTests/SubtitlePacketStoreScalingTests.swift new file mode 100644 index 000000000..6110a4a6b --- /dev/null +++ b/Tests/AetherEngineTests/SubtitlePacketStoreScalingTests.swift @@ -0,0 +1,178 @@ +import Foundation +import Testing +@testable import AetherEngine + +/// Audit SUB-103 / SUB-111: `appendLocked` copied the whole per-stream array on every append (40k +/// appends took 10.9 s), the same-PTS duplicate check walked the whole run, the aggregate pass shifted +/// the array once per evicted packet, and display sets still in assembly sat outside every cap. +struct SubtitlePacketStoreScalingTests { + + private func payload(_ index: Int, size: Int = 24) -> Data { + var data = Data(repeating: 0x41, count: size) + withUnsafeBytes(of: UInt64(index).littleEndian) { data.replaceSubrange(0..<8, with: $0) } + return data + } + + @Test("40k ascending appends stay linear") + func ascendingAppendsAreLinear() { + let store = SubtitlePacketStore() + let started = ContinuousClock.now + for index in 0..<40_000 { + store.append(streamIndex: 3, ptsSeconds: Double(index), durationSeconds: 1, payload: payload(index)) + } + let elapsed = ContinuousClock.now - started + #expect(store.frontier(streamIndex: 3) == 39_999) + #expect(store.entries(streamIndex: 3, from: 0, through: 1e9).count == 40_000) + #expect(elapsed < .seconds(2), "took \(elapsed)") + } + + @Test("20k distinct packets on one timestamp stay linear and keep harvest order") + func samePTSBurstIsLinear() { + let store = SubtitlePacketStore() + let started = ContinuousClock.now + for index in 0..<20_000 { + store.append(streamIndex: 3, ptsSeconds: 10, durationSeconds: 1, payload: payload(index)) + } + let elapsed = ContinuousClock.now - started + let stored = store.entries(streamIndex: 3, from: 10, through: 10) + #expect(stored.count == 20_000) + #expect(stored.map(\.payload) == (0..<20_000).map { payload($0) }) + #expect(elapsed < .seconds(2), "took \(elapsed)") + } + + @Test("a byte-identical packet re-harvested into a long same-PTS run collapses and takes the fresh sequence") + func duplicateInALongRunCollapses() { + let store = SubtitlePacketStore() + for index in 0..<50 { + store.append(streamIndex: 3, ptsSeconds: 10, durationSeconds: 1, payload: payload(index)) + } + let before = store.entries(streamIndex: 3, from: 10, through: 10) + store.append(streamIndex: 3, ptsSeconds: 10, durationSeconds: 1, payload: payload(17)) + let after = store.entries(streamIndex: 3, from: 10, through: 10) + #expect(after.count == 50) + #expect(after[17].payload == payload(17)) + #expect(after[17].sequence > before.map(\.sequence).max()!) + store.append(streamIndex: 3, ptsSeconds: 10, durationSeconds: 1, payload: payload(50)) + let grown = store.entries(streamIndex: 3, from: 10, through: 10) + #expect(grown.count == 51) + #expect(grown.last?.payload == payload(50)) + store.append(streamIndex: 3, ptsSeconds: 10, durationSeconds: 1, payload: payload(50)) + #expect(store.entries(streamIndex: 3, from: 10, through: 10).count == 51) + } + + @Test("the duplicate check still finds a packet after eviction moved the run") + func duplicateSurvivesEviction() { + let store = SubtitlePacketStore(perStreamByteCap: 40 * 24, aggregateByteCap: 1 << 20) + for index in 0..<60 { + store.append(streamIndex: 3, ptsSeconds: 10, durationSeconds: 1, payload: payload(index)) + } + let stored = store.entries(streamIndex: 3, from: 10, through: 10) + #expect(store.totalRetainedBytes <= 40 * 24) + #expect(stored.last?.payload == payload(59)) + store.append(streamIndex: 3, ptsSeconds: 10, durationSeconds: 1, payload: payload(59)) + #expect(store.entries(streamIndex: 3, from: 10, through: 10).count == stored.count) + } + + @Test("the per-stream cap holds after every append and evicts oldest first") + func perStreamCapHolds() { + let store = SubtitlePacketStore(perStreamByteCap: 10_000, aggregateByteCap: 1 << 20) + for index in 0..<500 { + store.append(streamIndex: 4, ptsSeconds: Double(index), durationSeconds: 1, payload: payload(index, size: 100)) + #expect(store.totalRetainedBytes <= 10_000) + } + let pts = store.entries(streamIndex: 4, from: 0, through: 1e9).map(\.ptsSeconds) + #expect(pts.last == 499) + #expect(pts == pts.sorted()) + #expect(pts.count > 80) + } + + @Test("aggregate eviction of a long cold stream is linear") + func aggregateEvictionIsLinear() { + let store = SubtitlePacketStore(perStreamByteCap: 1 << 30, aggregateByteCap: 40_000 * 24) + for index in 0..<40_000 { + store.append(streamIndex: 1, ptsSeconds: Double(index), durationSeconds: 1, payload: payload(index)) + } + let started = ContinuousClock.now + for index in 0..<40_000 { + store.append(streamIndex: 2, ptsSeconds: Double(index), durationSeconds: 1, payload: payload(index)) + } + let elapsed = ContinuousClock.now - started + #expect(store.totalRetainedBytes <= 40_000 * 24) + #expect(store.frontier(streamIndex: 2) == 39_999) + #expect(elapsed < .seconds(2), "took \(elapsed)") + } + + @Test("a window read is found by search and matches the filter it replaced") + func windowReadMatchesAFilter() { + let store = SubtitlePacketStore() + for index in 0..<200 { + store.append(streamIndex: 3, ptsSeconds: Double(index / 2), durationSeconds: 1, payload: payload(index)) + } + for (from, through) in [(0.0, 0.0), (10.0, 20.0), (-5.0, 3.0), (99.0, 500.0), (50.5, 50.9), (20.0, 10.0)] { + let got = store.entries(streamIndex: 3, from: from, through: through).map(\.payload) + let expected = (0..<200).filter { Double($0 / 2) >= from && Double($0 / 2) <= through }.map { payload($0) } + #expect(got == expected, "window \(from)...\(through)") + } + #expect(store.firstPTS(streamIndex: 3, after: 10) == 11) + #expect(store.firstPTS(streamIndex: 3, after: 99) == nil) + } + + // MARK: - SUB-111 part 2: display sets in assembly count toward the aggregate cap + + private func pgsSegment(_ type: UInt8, bodyLen: Int) -> Data { + var data = Data([type, UInt8((bodyLen >> 8) & 0xFF), UInt8(bodyLen & 0xFF)]) + data.append(Data(repeating: type, count: bodyLen)) + return data + } + + @Test("END-less display sets on many streams do not escape the aggregate cap") + func pendingSetsCountTowardTheAggregateCap() { + let cap = 64 * 1024 + let store = SubtitlePacketStore(perStreamByteCap: cap, aggregateByteCap: cap) + let streams = 100 + for stream in 0.. 0) + #expect(completed < streams) + #expect(store.totalRetainedBytes <= cap) + } + + @Test("a display set in assembly on a protected stream survives the pressure and completes") + func protectedPendingSetSurvives() { + let cap = 64 * 1024 + let store = SubtitlePacketStore(perStreamByteCap: cap, aggregateByteCap: cap) + store.setProtectedStreams([7]) + store.harvestChunk(streamIndex: 7, ptsSeconds: 10, durationSeconds: 0, flags: 0, + payload: pgsSegment(0x16, bodyLen: 11), assembleSplitDisplaySets: true) + for stream in 100..<160 { + for chunk in [pgsSegment(0x16, bodyLen: 11), pgsSegment(0x15, bodyLen: 2_000)] { + store.harvestChunk(streamIndex: Int32(stream), ptsSeconds: chunk[0] == 0x16 ? 10 : nil, + durationSeconds: 0, flags: 0, payload: chunk, assembleSplitDisplaySets: true) + } + } + store.harvestChunk(streamIndex: 7, ptsSeconds: nil, durationSeconds: 0, flags: 0, + payload: pgsSegment(0x80, bodyLen: 0), assembleSplitDisplaySets: true) + #expect(store.entries(streamIndex: 7, from: 0, through: 100).count == 1) + } + + @Test("clear forgets the assembly bytes too") + func clearResetsThePendingAccounting() { + let store = SubtitlePacketStore() + store.harvestChunk(streamIndex: 7, ptsSeconds: 10, durationSeconds: 0, flags: 0, + payload: pgsSegment(0x16, bodyLen: 500), assembleSplitDisplaySets: true) + #expect(store.totalRetainedBytes > 0) + store.clear() + #expect(store.totalRetainedBytes == 0) + } +} diff --git a/Tests/AetherEngineTests/SubtitleTimeBackstopTests.swift b/Tests/AetherEngineTests/SubtitleTimeBackstopTests.swift new file mode 100644 index 000000000..51bdf0a40 --- /dev/null +++ b/Tests/AetherEngineTests/SubtitleTimeBackstopTests.swift @@ -0,0 +1,90 @@ +import Foundation +import Testing +import AetherLibavcodec +@testable import AetherEngine + +/// Audit FEA-101 / SUB-105: cue times reach the stores and formatters from more than the demuxer +/// (live WebVTT renditions, external stores, a host calling the public builder), so every Double to +/// integer conversion on a cue time is total: a huge, infinite or NaN time degrades, it never traps. +@Suite("Hostile cue times never trap a subtitle store or formatter") +struct SubtitleTimeBackstopTests { + + private func subripDecoder() throws -> (Demuxer, EmbeddedSubtitleDecoder) { + let demuxer = Demuxer() + try demuxer.open(reader: DataIOReader(data: HostileTimestampMatroska.make( + blocks: [.init(cluster: 0, duration: 1000)])), formatHint: "matroska") + let stream = try #require(demuxer.stream(at: 0)) + let decoder = try #require(EmbeddedSubtitleDecoder(stream: stream, sourceVideoWidth: 16, sourceVideoHeight: 16)) + return (demuxer, decoder) + } + + @Test("the drain rebuilds a stored packet whose seconds do not fit Int64 milliseconds") + func drainDecodeOfOutOfRangeSeconds() throws { + let (demuxer, decoder) = try subripDecoder() + defer { demuxer.close() } + let payload = Data("line".utf8) + let hugeDuration = StoredSubtitlePacket(ptsSeconds: 1, durationSeconds: 9.3e15, flags: 0, payload: payload) + #expect(AetherEngine.decodeStoredSubtitlePacket(hugeDuration, with: decoder)?.cues.first?.startTime == 1) + let hugePTS = StoredSubtitlePacket(ptsSeconds: Double(Int64.max) * 1e-3 * 1000, durationSeconds: 1, + flags: 0, payload: payload) + _ = AetherEngine.decodeStoredSubtitlePacket(hugePTS, with: decoder) + let infinite = StoredSubtitlePacket(ptsSeconds: .infinity, durationSeconds: .nan, flags: 0, payload: payload) + _ = AetherEngine.decodeStoredSubtitlePacket(infinite, with: decoder) + } + + @Test("the store unsets implausible times and zeroes implausible durations for every writer") + func harvestBoundsTimes() throws { + let (demuxer, decoder) = try subripDecoder() + defer { demuxer.close() } + let store = SubtitlePacketStore() + let payload = Data("line".utf8) + store.harvestChunk(streamIndex: 0, ptsSeconds: 1, durationSeconds: 9.2e15, flags: 0, + payload: payload, assembleSplitDisplaySets: false, writer: .prefetch) + store.harvestChunk(streamIndex: 0, ptsSeconds: 2, durationSeconds: .infinity, flags: 0, + payload: payload, assembleSplitDisplaySets: false) + store.harvestChunk(streamIndex: 0, ptsSeconds: 3, durationSeconds: 0.5, flags: 0, + payload: payload, assembleSplitDisplaySets: false) + store.harvestChunk(streamIndex: 0, ptsSeconds: .infinity, durationSeconds: 1, flags: 0, + payload: payload, assembleSplitDisplaySets: false) + store.harvestChunk(streamIndex: 0, ptsSeconds: Double(Int64.max) * 1e-3, durationSeconds: 1, flags: 0, + payload: payload, assembleSplitDisplaySets: false) + let entries = store.entries(streamIndex: 0, from: -.infinity, through: .infinity) + #expect(entries.map(\.ptsSeconds) == [1, 2, 3]) + #expect(entries.map(\.durationSeconds) == [0, 0, 0.5]) + for entry in entries { + #expect(AetherEngine.decodeStoredSubtitlePacket(entry, with: decoder) != nil) + } + } + + @Test("the native cue store dedupes cues at huge and non-finite times") + func cueStoreKeys() { + let store = NativeSubtitleCueStore() + store.appendCues([ + SubtitleCue(id: 1, startTime: 1e16, endTime: 1e16 + 1, body: .text("far")), + SubtitleCue(id: 2, startTime: .infinity, endTime: .infinity, body: .text("inf")), + SubtitleCue(id: 3, startTime: .nan, endTime: .nan, body: .text("nan")), + SubtitleCue(id: 4, startTime: 1, endTime: 2, body: .text("near")), + ]) + #expect(store.cueCount == 4) + } + + @Test("the public ASS timestamp formatter pins huge and non-finite input") + func assTimestamp() { + #expect(ASSScriptBuilder.timestamp(3661.5) == "1:01:01.50") + #expect(ASSScriptBuilder.timestamp(-3) == "0:00:00.00") + #expect(ASSScriptBuilder.timestamp(.nan) == "0:00:00.00") + #expect(ASSScriptBuilder.timestamp(.infinity) == ASSScriptBuilder.timestamp(1e300)) + #expect(ASSScriptBuilder.timestamp(1e17) == ASSScriptBuilder.timestamp(4e9)) + } + + @Test("the WebVTT body survives cues at huge and non-finite times") + func webVTTTimestamps() { + let vtt = WebVTTBuilder.body(cues: [ + (start: 1, end: 2, text: "near"), + (start: 1e19, end: .infinity, text: "far"), + (start: .nan, end: .nan, text: "nan"), + ]) + #expect(vtt.contains("00:00:01.000 --> 00:00:02.000\nnear")) + #expect(vtt.contains("far")) + } +} diff --git a/Tests/AetherEngineTests/SupersededProbeGeometryTests.swift b/Tests/AetherEngineTests/SupersededProbeGeometryTests.swift new file mode 100644 index 000000000..c5690f3d9 --- /dev/null +++ b/Tests/AetherEngineTests/SupersededProbeGeometryTests.swift @@ -0,0 +1,119 @@ +import Combine +import Foundation +import Testing +import AetherLibavcodec +import AetherLibavformat +import AetherLibavutil +@testable import AetherEngine + +/// Audit LIF-107: the probe block wrote the source geometry and codec into engine state before its +/// post-probe generation check. The AE#532 RPU audit (a Dolby Vision Profile 5 record over an HDR10 +/// VUI) is a second network read `stopInternal` cannot abort, so a load superseded inside it came +/// back after its successor's prologue and wrote its own width, height and codec over the reset. +/// A successor that never probes (the remote-HLS bypass) then kept them for the session. +@Suite("A superseded probe leaves its successor's geometry alone", .timeLimit(.minutes(2))) +@MainActor +struct SupersededProbeGeometryTests { + + /// The 64x64 HDR10 fixture with a Dolby Vision Profile 5 record added: the one pairing the + /// AE#532 audit reads the bitstream for. + static func dolbyVisionProfile5OverHDR10() throws -> Data { + let video = Demuxer() + defer { video.close() } + try video.open(reader: DataIOReader(data: ProbeTestFixtures.hdr10Plus()), formatHint: "mp4") + let source = try #require(video.stream(at: video.videoStreamIndex)) + + var output: UnsafeMutablePointer? + try #require(avformat_alloc_output_context2(&output, nil, "mp4", nil) >= 0) + let context = try #require(output) + defer { avformat_free_context(context) } + // movenc writes the dvcC box only when unofficial extensions are allowed. + context.pointee.strict_std_compliance = FF_COMPLIANCE_UNOFFICIAL + + var io: UnsafeMutablePointer? + try #require(avio_open_dyn_buf(&io) >= 0) + let buffer = try #require(io) + context.pointee.pb = buffer + var bytes: UnsafeMutablePointer? + var bufferClosed = false + defer { + if !bufferClosed { _ = avio_close_dyn_buf(buffer, &bytes) } + av_free(bytes) + } + + let target = try #require(avformat_new_stream(context, nil)) + let codecpar = try #require(target.pointee.codecpar) + try #require(avcodec_parameters_copy(codecpar, source.pointee.codecpar) >= 0) + target.pointee.time_base = source.pointee.time_base + var recordSize = 0 + let record = try #require(av_dovi_alloc(&recordSize)) + record.pointee.dv_version_major = 1 + record.pointee.dv_profile = 5 + record.pointee.dv_level = 1 + record.pointee.rpu_present_flag = 1 + record.pointee.bl_present_flag = 1 + record.pointee.dv_bl_signal_compatibility_id = 0 + guard av_packet_side_data_add(&codecpar.pointee.coded_side_data, &codecpar.pointee.nb_coded_side_data, + AV_PKT_DATA_DOVI_CONF, UnsafeMutableRawPointer(record), recordSize, 0) != nil else { + av_free(record) + throw DemuxerError.openFailed(code: -1) + } + + try #require(avformat_write_header(context, nil) >= 0) + while let packet = try video.readPacket() { + var owned: UnsafeMutablePointer? = packet + defer { trackedPacketFree(&owned) } + guard packet.pointee.stream_index == video.videoStreamIndex else { continue } + av_packet_rescale_ts(packet, source.pointee.time_base, target.pointee.time_base) + packet.pointee.stream_index = 0 + packet.pointee.pos = -1 + try #require(av_interleaved_write_frame(context, packet) >= 0) + } + try #require(av_write_trailer(context) >= 0) + let count = avio_close_dyn_buf(buffer, &bytes) + bufferClosed = true + context.pointee.pb = nil + try #require(count > 0) + return Data(bytes: try #require(bytes), count: Int(count)) + } + + @Test("The fixture is the pairing the RPU audit exists for") + func fixtureIsAContradictedRecord() throws { + let demuxer = Demuxer() + defer { demuxer.close() } + try demuxer.open(reader: DataIOReader(data: Self.dolbyVisionProfile5OverHDR10()), formatHint: "mp4") + let stream = try #require(demuxer.stream(at: demuxer.videoStreamIndex)) + let codecpar = try #require(stream.pointee.codecpar) + #expect(AetherEngine.dvConfig(stream: stream)?.profile == 5) + #expect(DolbyVisionRecordAudit.recordIsContradicted( + codecID: codecpar.pointee.codec_id, dvProfile: 5, + colorTransfer: codecpar.pointee.color_trc, colorMatrix: codecpar.pointee.color_space)) + } + + @Test("A load superseded during its RPU audit does not write its width over the successor's") + func supersededAuditPublishesNothing() async throws { + let origin = try ProbeHTTPTestOrigin(data: try Self.dolbyVisionProfile5OverHDR10()) + defer { origin.stop() } + let engine = try AetherEngine() + defer { engine.stop() } + // The open's last stage hops to the main actor ahead of the open's own continuation, so + // every request from here on is the audit's. + let sub = engine.$startupProgress.sink { progress in + if progress?.checkpoint == .streamsProbed { origin.holdLaterRequests() } + } + defer { sub.cancel() } + + let superseded = Task { @MainActor in + try await engine.load(url: try #require(URL(string: "http://127.0.0.1:\(origin.port)/p5.mp4"))) + } + try await waitFor { origin.held.entered } + _ = try await engine.load(url: try #require(URL(string: "http://127.0.0.1:9/vod.m3u8")), + options: LoadOptions(nativeRemoteHLS: true)) + #expect(engine.sourceVideoWidth == 0) + origin.held.open() + + await #expect(throws: CancellationError.self) { try await superseded.value } + #expect(engine.sourceVideoWidth == 0) + #expect(engine.sourceVideoHeight == 0) + } +} diff --git a/Tests/AetherEngineTests/SupersededProducerReportTests.swift b/Tests/AetherEngineTests/SupersededProducerReportTests.swift new file mode 100644 index 000000000..982880e6c --- /dev/null +++ b/Tests/AetherEngineTests/SupersededProducerReportTests.swift @@ -0,0 +1,217 @@ +import Foundation +import Testing +import AetherLibavcodec +import AetherLibavutil +@testable import AetherEngine + +/// Audit HLS-104, HLS-105, SEG-104 (B): a producer the session has already replaced, or an action +/// scheduled on its behalf, used to act on whatever producer was current by then. A gate open from a +/// superseded restart rewrote the epoch table, the delayed #377 revive re-anchored the producer that +/// had replaced the dead one, and a re-cut lost its mark to the seek's 2 s wait while the restart it +/// marked was still running. +@Suite("A superseded producer leaves the current session alone", .serialized) +struct SupersededProducerReportTests { + + private final class Session { + let engine: HLSVideoEngine + private var codecpar: UnsafeMutablePointer? + private let cache = SegmentCache(forwardWindow: 20, backwardWindow: 20) + + init() throws { + engine = HLSVideoEngine(url: URL(fileURLWithPath: "/nonexistent/superseded.mkv"), + dvModeAvailable: false) + codecpar = try #require(avcodec_parameters_alloc()) + codecpar!.pointee.codec_id = AV_CODEC_ID_H264 + codecpar!.pointee.codec_type = AVMEDIA_TYPE_VIDEO + let plan = (0..<10).map { i in + HLSVideoEngine.Segment(startPts: Int64(i) * 4000, endPts: Int64(i + 1) * 4000, + startSeconds: Double(i) * 4.0, durationSeconds: 4.0) + } + engine.segmentPlan = plan + engine.cache = cache + engine.demuxer = Demuxer() + engine.savedVideoConfig = .init(codecpar: UnsafePointer(codecpar!), + timeBase: AVRational(num: 1, den: 1000), + codecTagOverride: nil) + engine.provider = VideoSegmentProvider( + cache: cache, segments: plan, codecsString: "avc1.42C00A", supplementalCodecs: nil, + resolution: (16, 16), videoRange: .sdr, frameRate: 2, hdcpLevel: nil, + sourceBitrate: 100_000) + } + + deinit { + engine.stop() + avcodec_parameters_free(&codecpar) + } + + /// A stored segment that opens on no random-access point, so a seek into it has to re-cut. + func storeWithoutSyncSample(_ index: Int) throws { + let staging = cache.sessionDir.appendingPathComponent("staging-\(index).tmp") + try Data(repeating: 0xAA, count: 64).write(to: staging) + cache.adopt(index: index, stagingPath: staging, byteCount: 64, videoReach: SegmentCache.VideoReach.none) + } + + /// The producer reports its gate open, as the pump does: the item-axis tfdt of its first frame + /// is the start of the segment it was anchored at (plan time base 1/1000). + func openGate(of producer: HLSSegmentProducer, at index: Int, backoffMs: Int64 = 3000) { + producer.onVideoShiftKnown?(backoffMs, Int64(index) * 4000, 0) + } + } + + private final class Restarts: @unchecked Sendable { + private let lock = NSLock() + private var _began = 0 + func note(_ began: Bool) { if began { lock.withLock { _began += 1 } } } + var began: Int { lock.withLock { _began } } + } + + // MARK: - The gate open of a replaced producer (SEG-104 B) + + @Test("a gate open from a producer the session replaced records nothing") + func supersededGateOpenRecordsNothing() throws { + let session = try Session() + let engine = session.engine + let stale = try engine.makeProducer(baseIndex: 2) + engine.producer = stale + engine.producer = nil // performRestart tears it down before building the next one + + session.openGate(of: stale, at: 2) + #expect(engine.epochAxisByIndex.opening(at: 2) == nil, + "the stale epoch dropped the table at and above its own index") + + let current = try engine.makeProducer(baseIndex: 5) + engine.producer = current + session.openGate(of: stale, at: 2) + session.openGate(of: current, at: 5) + #expect(engine.epochAxisByIndex.opening(at: 2) == nil) + #expect(engine.epochAxisByIndex.opening(at: 5) != nil, "the installed producer's gate still records") + } + + // MARK: - The re-cut mark (HLS-105) + + @Test("a re-cut whose gate opens after the seek stopped waiting is still recorded as a re-cut", + .timeLimit(.minutes(1))) + func lateRecutGateKeepsItsMark() async throws { + let session = try Session() + let engine = session.engine + for i in 0...4 { try session.storeWithoutSyncSample(i) } + + // Nothing here ever opens a gate on its own, so the seek's wait runs out. + let landing = await Task.detached { engine.preparedSeekLanding(itemSeconds: 14) }.value + #expect(landing == 14) + try await waitFor { engine.producer?.anchoredBaseIndex == 3 } + let recut = try #require(engine.producer) + + session.openGate(of: recut, at: 3) + #expect(engine.epochAxisByIndex.opening(at: 3)?.isRecut == true, + "the late gate open composed the re-cut's backoff into the axis") + } + + @Test("a mark belongs to the restart it asked for, not to whatever restart runs next") + func markBindsOnlyItsOwnRestart() throws { + let session = try Session() + let engine = session.engine + + engine.markRecut(at: 3) + engine.requestRestart(at: 6, authoritative: true) + let other = try #require(engine.producer) + session.openGate(of: other, at: 6) + #expect(engine.epochAxisByIndex.opening(at: 6)?.isRecut == false) + + engine.requestRestart(at: 3, authoritative: true) + let recut = try #require(engine.producer) + engine.requestRestart(at: 7, authoritative: true) // supersedes the re-cut before its gate opened + let next = try #require(engine.producer) + session.openGate(of: recut, at: 3) + session.openGate(of: next, at: 7) + #expect(engine.epochAxisByIndex.opening(at: 3) == nil) + #expect(engine.epochAxisByIndex.opening(at: 7)?.isRecut == false) + } + + @Test("the mark's lifecycle: bound by its own restart, kept past others, dropped once superseded") + func recutMarkLifecycle() { + let mark = HLSVideoEngine.RecutMark(index: 3) + #expect(!mark.isOpened(byProducerEpoch: 7, at: 3), "unbound, no gate can claim it") + + let pending = mark.installing(producerEpoch: 5, at: 6) + #expect(pending == mark, "a restart elsewhere ran first; the marked one is still to come") + + let bound = pending?.installing(producerEpoch: 7, at: 3) + #expect(bound?.isOpened(byProducerEpoch: 7, at: 3) == true) + #expect(bound?.isOpened(byProducerEpoch: 5, at: 3) == false) + #expect(bound?.installing(producerEpoch: 8, at: 3)?.isOpened(byProducerEpoch: 8, at: 3) == true, + "a second restart at the same index re-cuts the same landing") + #expect(bound?.installing(producerEpoch: 9, at: 6) == nil) + } + + // MARK: - The delayed #377 revive (HLS-104) + + @Test("the metered revive is owed only while its session and its dead producer are both still current") + func meteredReviveOwedMatrix() { + #expect(HLSVideoEngine.meteredReviveStillOwed(sessionUnchanged: true, deadProducerInstalled: true)) + #expect(!HLSVideoEngine.meteredReviveStillOwed(sessionUnchanged: true, deadProducerInstalled: false)) + #expect(!HLSVideoEngine.meteredReviveStillOwed(sessionUnchanged: false, deadProducerInstalled: true)) + #expect(!HLSVideoEngine.meteredReviveStillOwed(sessionUnchanged: false, deadProducerInstalled: false)) + } + + @Test("a revive that woke after another restart replaced its dead producer restarts nothing", + .timeLimit(.minutes(1))) + func meteredReviveYieldsToAReplacement() async throws { + let session = try Session() + let engine = session.engine + let restarts = Restarts() + engine.onSeekStateChanged = { began, _ in restarts.note(began) } + let dead = try engine.makeProducer(baseIndex: 2) + engine.producer = dead + dead.start() + try await waitFor { dead.didFinish } + let epoch = engine.sessionEpochSnapshot() + + // A segment-driven restart during the backoff already put a working producer in. + engine.producer = try engine.makeProducer(baseIndex: 6) + engine.fireMeteredRevive(at: 2, deadProducer: dead, sessionEpoch: epoch) + #expect(restarts.began == 0, "the revive re-anchored the producer that replaced the dead one") + + // Its own dead producer still in place, it runs. + engine.producer = dead + engine.fireMeteredRevive(at: 2, deadProducer: dead, sessionEpoch: epoch) + #expect(restarts.began == 1) + } + + @Test("a revive that wakes as a seek restart starts does not queue its stale target behind it", + .timeLimit(.minutes(1))) + func meteredReviveYieldsToAStartingRestart() async throws { + let session = try Session() + let engine = session.engine + let restarts = Restarts() + let dead = try engine.makeProducer(baseIndex: 2) + engine.producer = dead + dead.start() + try await waitFor { dead.didFinish } + let epoch = engine.sessionEpochSnapshot() + + // The seek restart is admitted and held before it takes the dead producer out, so the + // revive's own producer and session checks still pass. + let admitted = DispatchSemaphore(value: 0) + let proceed = DispatchSemaphore(value: 0) + engine.onSeekStateChanged = { began, _ in + restarts.note(began) + if began, restarts.began == 1 { + admitted.signal() + proceed.wait() + } + } + let began: Int = await withCheckedContinuation { continuation in + DispatchQueue.global().async { + let seek = DispatchGroup() + DispatchQueue.global().async(group: seek) { engine.requestRestart(at: 6) } + admitted.wait() + engine.fireMeteredRevive(at: 2, deadProducer: dead, sessionEpoch: epoch) + proceed.signal() + seek.wait() + continuation.resume(returning: restarts.began) + } + } + #expect(began == 1, "the revive's stale target ran after the seek restart") + } +} diff --git a/Tests/AetherEngineTests/SupersededPumpExitTests.swift b/Tests/AetherEngineTests/SupersededPumpExitTests.swift new file mode 100644 index 000000000..2bdfe787d --- /dev/null +++ b/Tests/AetherEngineTests/SupersededPumpExitTests.swift @@ -0,0 +1,84 @@ +import Testing +import Foundation +import AetherLibavcodec +import AetherLibavutil +@testable import AetherEngine + +/// Audit HLS-1: a producer the session already replaced (the #79 wedge abort, or `stop()`) +/// reports its aborted read as a read error. That exit must not reach the session's recovery +/// arms: no revive-gate spend, no seek-state or failure signal to the host. +@Suite("Superseded pump exit") +struct SupersededPumpExitTests { + + private final class Recorder: @unchecked Sendable { + private let lock = NSLock() + private var _failures = 0 + private var _seekSignals = 0 + var failures: Int { lock.lock(); defer { lock.unlock() }; return _failures } + var seekSignals: Int { lock.lock(); defer { lock.unlock() }; return _seekSignals } + func fail() { lock.lock(); _failures += 1; lock.unlock() } + func seek() { lock.lock(); _seekSignals += 1; lock.unlock() } + } + + private func makeProducer(codecpar: UnsafeMutablePointer) throws -> HLSSegmentProducer { + try HLSSegmentProducer( + demuxer: Demuxer(), + videoStreamIndex: 0, + video: .init(codecpar: UnsafePointer(codecpar), + timeBase: AVRational(num: 1, den: 90_000), + codecTagOverride: nil), + cache: SegmentCache(forwardWindow: 4, backwardWindow: 4), + videoFallbackDurationPts: 3_000, + desiredFirstVideoTfdtPts: 0, + segmentBoundaries: [0, 540_000] + ) + } + + private func makeEngine(_ recorder: Recorder) -> HLSVideoEngine { + let engine = HLSVideoEngine(url: URL(fileURLWithPath: "/nonexistent/superseded.mkv"), + dvModeAvailable: false) + engine.onVODSourceFailed = { _, _, _ in recorder.fail() } + engine.onSeekStateChanged = { _, _ in recorder.seek() } + return engine + } + + @Test("a read error from a replaced producer spends nothing and signals nothing") + func supersededReadErrorIsIgnored() throws { + var codecpar: UnsafeMutablePointer? = try #require(avcodec_parameters_alloc()) + defer { avcodec_parameters_free(&codecpar) } + codecpar!.pointee.codec_id = AV_CODEC_ID_H264 + codecpar!.pointee.codec_type = AVMEDIA_TYPE_VIDEO + + let recorder = Recorder() + let engine = makeEngine(recorder) + let stale = try makeProducer(codecpar: codecpar!) + let current = try makeProducer(codecpar: codecpar!) + engine.producer = current + + engine.handlePumpFinished(stale, reason: .readError(code: -1)) + engine.producer = nil + engine.handlePumpFinished(stale, reason: .readError(code: -1)) + + #expect(engine.readErrorReviveGate.attempts == 0) + #expect(!engine.mainDemuxerSuspectDead) + #expect(recorder.failures == 0) + #expect(recorder.seekSignals == 0) + } + + @Test("the same read error from the session's own producer still reaches the recovery arms") + func currentReadErrorIsHandled() throws { + var codecpar: UnsafeMutablePointer? = try #require(avcodec_parameters_alloc()) + defer { avcodec_parameters_free(&codecpar) } + codecpar!.pointee.codec_id = AV_CODEC_ID_H264 + codecpar!.pointee.codec_type = AVMEDIA_TYPE_VIDEO + + let recorder = Recorder() + let engine = makeEngine(recorder) + let current = try makeProducer(codecpar: codecpar!) + engine.producer = current + + engine.handlePumpFinished(current, reason: .readError(code: -1)) + + #expect(recorder.failures == 1) + } +} diff --git a/Tests/AetherEngineTests/Support/CannedHTTPOrigin.swift b/Tests/AetherEngineTests/Support/CannedHTTPOrigin.swift new file mode 100644 index 000000000..9df0920f7 --- /dev/null +++ b/Tests/AetherEngineTests/Support/CannedHTTPOrigin.swift @@ -0,0 +1,146 @@ +import Darwin +import Foundation + +/// A loopback origin that answers each path from a table and records the headers every request +/// arrived with (names lowercased), so a credential that must not reach a host is checked where it +/// would have landed. One request per connection; the table can be filled after start, so two of +/// these can point at each other. +final class CannedHTTPOrigin: @unchecked Sendable { + enum Answer: Sendable { + case body(String, contentType: String) + case redirect(to: String) + case status(Int) + } + + struct Request: Sendable { + let path: String + let headers: [String: String] + } + + let port: UInt16 + private let listenFD: Int32 + private let lock = NSLock() + private var _routes: [String: Answer] = [:] + private var _requests: [Request] = [] + private var _connections: Set = [] + private var _stopped = false + + init?() { + let fd = socket(AF_INET, SOCK_STREAM, 0) + guard fd >= 0 else { return nil } + var one: Int32 = 1 + setsockopt(fd, SOL_SOCKET, SO_REUSEADDR, &one, socklen_t(MemoryLayout.size)) + var address = sockaddr_in() + address.sin_family = sa_family_t(AF_INET) + address.sin_port = 0 + address.sin_addr.s_addr = inet_addr("127.0.0.1") + let bound = withUnsafePointer(to: &address) { + $0.withMemoryRebound(to: sockaddr.self, capacity: 1) { + bind(fd, $0, socklen_t(MemoryLayout.size)) + } + } + guard bound == 0, listen(fd, 16) == 0 else { + close(fd) + return nil + } + var length = socklen_t(MemoryLayout.size) + let named = withUnsafeMutablePointer(to: &address) { + $0.withMemoryRebound(to: sockaddr.self, capacity: 1) { getsockname(fd, $0, &length) } + } + guard named == 0 else { + close(fd) + return nil + } + listenFD = fd + port = UInt16(bigEndian: address.sin_port) + Thread.detachNewThread { [self] in acceptLoop() } + } + + var baseURL: String { "http://127.0.0.1:\(port)" } + + func route(_ path: String, _ answer: Answer) { + lock.withLock { _routes[path] = answer } + } + + var requests: [Request] { lock.withLock { _requests } } + + func requests(to path: String) -> [Request] { requests.filter { $0.path == path } } + + func stop() { + lock.lock() + let alreadyStopped = _stopped + _stopped = true + for fd in _connections { shutdown(fd, SHUT_RDWR) } + lock.unlock() + guard !alreadyStopped else { return } + shutdown(listenFD, SHUT_RDWR) + close(listenFD) + } + + private func acceptLoop() { + while true { + let fd = accept(listenFD, nil, nil) + if fd < 0 { return } + var one: Int32 = 1 + setsockopt(fd, SOL_SOCKET, SO_NOSIGPIPE, &one, socklen_t(MemoryLayout.size)) + lock.lock() + if _stopped { + lock.unlock() + close(fd) + return + } + _connections.insert(fd) + lock.unlock() + Thread.detachNewThread { [self] in serve(fd) } + } + } + + private func serve(_ fd: Int32) { + defer { + lock.lock() + _connections.remove(fd) + close(fd) + lock.unlock() + } + var buffer = Data() + var chunk = [UInt8](repeating: 0, count: 4096) + while buffer.range(of: Data("\r\n\r\n".utf8)) == nil { + let n = recv(fd, &chunk, chunk.count, 0) + guard n > 0, buffer.count < 65_536 else { return } + buffer.append(chunk, count: n) + } + let lines = String(decoding: buffer, as: UTF8.self).components(separatedBy: "\r\n") + let target = lines.first?.split(separator: " ").dropFirst().first.map(String.init) ?? "/" + let path = String(target.split(separator: "?", maxSplits: 1).first ?? "/") + var headers: [String: String] = [:] + for line in lines.dropFirst() { + guard let colon = line.firstIndex(of: ":") else { continue } + headers[line[.. Answer? in + _requests.append(Request(path: path, headers: headers)) + return _routes[path] + } + let response: String + switch answer { + case .body(let body, let contentType): + response = "HTTP/1.1 200 OK\r\nContent-Type: \(contentType)\r\n" + + "Content-Length: \(body.utf8.count)\r\nConnection: close\r\n\r\n\(body)" + case .redirect(let location): + response = "HTTP/1.1 302 Found\r\nLocation: \(location)\r\n" + + "Content-Length: 0\r\nConnection: close\r\n\r\n" + case .status(let code): + response = "HTTP/1.1 \(code) Status\r\nContent-Length: 0\r\nConnection: close\r\n\r\n" + case nil: + response = "HTTP/1.1 404 Not Found\r\nContent-Length: 0\r\nConnection: close\r\n\r\n" + } + let bytes = Array(response.utf8) + var sent = 0 + while sent < bytes.count { + let n = bytes[sent...].withUnsafeBytes { send(fd, $0.baseAddress, $0.count, 0) } + guard n > 0 else { return } + sent += n + } + } +} diff --git a/Tests/AetherEngineTests/Support/DiagnosticPoolTesting.swift b/Tests/AetherEngineTests/Support/DiagnosticPoolTesting.swift new file mode 100644 index 000000000..a1f5df00b --- /dev/null +++ b/Tests/AetherEngineTests/Support/DiagnosticPoolTesting.swift @@ -0,0 +1,14 @@ +@testable import AetherEngine + +extension ItemDiagnosticReadPool { + /// A pool whose lanes do not give up on a read. + /// + /// The AE#597 read timeout is a deadline on a step these tests hold on purpose: a probe that + /// blocks until the test releases it. On an oversubscribed runner the release can come later + /// than 15 s, the lane then abandons the read, and the test reports the timeout instead of the + /// behaviour it is about (a retirement finishing incomplete, a delivery that never comes, a + /// third lane opening). Tests of the timeout itself pass their own small value. + static func withoutReadTimeout() -> ItemDiagnosticReadPool { + ItemDiagnosticReadPool(readTimeout: 86_400) + } +} diff --git a/Tests/AetherEngineTests/Support/EngineLogCapture.swift b/Tests/AetherEngineTests/Support/EngineLogCapture.swift new file mode 100644 index 000000000..716d9e796 --- /dev/null +++ b/Tests/AetherEngineTests/Support/EngineLogCapture.swift @@ -0,0 +1,76 @@ +import Foundation +@testable import AetherEngine + +/// Collects `EngineLog` lines for one test. +/// +/// `EngineLog.handler` is one process-global closure and swift-testing runs suites in parallel. A +/// test that replaced it and later put back the value it had found could restore a stale closure +/// while another test was still listening, and that test then read an empty capture (three red CI +/// runs of the #450 witness, none of them reproducible on a Mac with cores to spare). So no test +/// assigns the handler any more: one forwarder is installed the first time a capture is made and +/// fans every line out to the captures alive at that moment. +/// +/// Every capture therefore sees the lines of every test that logs at the same time. Filter by a +/// marker that belongs to the test, never count totals. +/// +/// Create one, read `lines` / `matching(_:)`, and `end()` it in a `defer` (an XCTest case does so +/// in `tearDown`). Lines emitted after `end()` are not collected; the ones already collected stay +/// readable. +final class EngineLogCapture: @unchecked Sendable { + private let lock = NSLock() + private var storage: [String] = [] + + init() { + Registry.shared.add(self) + } + + func end() { + Registry.shared.remove(self) + } + + var lines: [String] { + lock.lock(); defer { lock.unlock() } + return storage + } + + func matching(_ needle: String) -> [String] { + lock.lock(); defer { lock.unlock() } + return storage.filter { $0.contains(needle) } + } + + fileprivate func append(_ line: String) { + lock.lock(); defer { lock.unlock() } + storage.append(line) + } + + /// The handler stays installed for the rest of the process once the first capture exists, which + /// is harmless: nothing in the engine branches on it being set, and with no capture registered + /// the forwarder does nothing. + private final class Registry: @unchecked Sendable { + static let shared: Registry = { + let registry = Registry() + EngineLog.handler = { registry.forward($0) } + return registry + }() + + private let lock = NSLock() + private var captures: [ObjectIdentifier: EngineLogCapture] = [:] + + func add(_ capture: EngineLogCapture) { + lock.lock(); defer { lock.unlock() } + captures[ObjectIdentifier(capture)] = capture + } + + func remove(_ capture: EngineLogCapture) { + lock.lock(); defer { lock.unlock() } + captures[ObjectIdentifier(capture)] = nil + } + + func forward(_ line: String) { + lock.lock() + let live = Array(captures.values) + lock.unlock() + for capture in live { capture.append(line) } + } + } +} diff --git a/Tests/AetherEngineTests/Support/KeepAliveRangeOrigin.swift b/Tests/AetherEngineTests/Support/KeepAliveRangeOrigin.swift new file mode 100644 index 000000000..b06fbc4e5 --- /dev/null +++ b/Tests/AetherEngineTests/Support/KeepAliveRangeOrigin.swift @@ -0,0 +1,146 @@ +import Darwin +import Foundation + +/// A loopback origin that serves one byte blob with Range support over keep-alive connections and +/// counts the connections that are still open. A client that leaves its `URLSession` uninvalidated +/// leaves its pooled socket here, which is the only place that leak is visible from outside: the +/// session has no public liveness and the origin is the one party that sees the socket. +final class KeepAliveRangeOrigin: @unchecked Sendable { + let port: UInt16 + private let listenFD: Int32 + private let data: Data + private let lock = NSLock() + private var connections: Set = [] + private var stopped = false + private var accepted = 0 + + init?(data: Data) { + self.data = data + let fd = socket(AF_INET, SOCK_STREAM, 0) + guard fd >= 0 else { return nil } + var one: Int32 = 1 + setsockopt(fd, SOL_SOCKET, SO_REUSEADDR, &one, socklen_t(MemoryLayout.size)) + var address = sockaddr_in() + address.sin_family = sa_family_t(AF_INET) + address.sin_port = 0 + address.sin_addr.s_addr = inet_addr("127.0.0.1") + let bound = withUnsafePointer(to: &address) { + $0.withMemoryRebound(to: sockaddr.self, capacity: 1) { + bind(fd, $0, socklen_t(MemoryLayout.size)) + } + } + guard bound == 0, listen(fd, 16) == 0 else { + close(fd) + return nil + } + var length = socklen_t(MemoryLayout.size) + let named = withUnsafeMutablePointer(to: &address) { + $0.withMemoryRebound(to: sockaddr.self, capacity: 1) { getsockname(fd, $0, &length) } + } + guard named == 0 else { + close(fd) + return nil + } + listenFD = fd + port = UInt16(bigEndian: address.sin_port) + Thread.detachNewThread { [self] in acceptLoop() } + } + + var openConnectionCount: Int { lock.withLock { connections.count } } + var acceptedConnectionCount: Int { lock.withLock { accepted } } + + func stop() { + lock.lock() + let already = stopped + stopped = true + for fd in connections { shutdown(fd, SHUT_RDWR) } + lock.unlock() + guard !already else { return } + shutdown(listenFD, SHUT_RDWR) + close(listenFD) + } + + private func acceptLoop() { + while true { + let fd = accept(listenFD, nil, nil) + if fd < 0 { return } + var one: Int32 = 1 + setsockopt(fd, SOL_SOCKET, SO_NOSIGPIPE, &one, socklen_t(MemoryLayout.size)) + lock.lock() + if stopped { + lock.unlock() + shutdown(fd, SHUT_RDWR) + close(fd) + return + } + connections.insert(fd) + accepted += 1 + lock.unlock() + Thread.detachNewThread { [self] in serve(fd) } + } + } + + private func serve(_ fd: Int32) { + defer { + lock.lock() + connections.remove(fd) + close(fd) + lock.unlock() + } + while let head = readRequestHead(fd) { + var start = 0 + var end = data.count - 1 + if let range = Self.range(in: head) { + start = range.start + end = min(end, range.end ?? end) + } + guard start < data.count, end >= start else { + guard writeAll(fd, Data("HTTP/1.1 416 Range Not Satisfiable\r\nContent-Length: 0\r\n\r\n".utf8)) + else { return } + continue + } + let header = "HTTP/1.1 206 Partial Content\r\n" + + "Content-Length: \(end - start + 1)\r\n" + + "Content-Range: bytes \(start)-\(end)/\(data.count)\r\n" + + "Accept-Ranges: bytes\r\nConnection: keep-alive\r\n\r\n" + guard writeAll(fd, Data(header.utf8)), + writeAll(fd, data.subdata(in: start..<(end + 1))) else { return } + } + } + + private static func range(in head: String) -> (start: Int, end: Int?)? { + guard let line = head.components(separatedBy: "\r\n") + .first(where: { $0.lowercased().hasPrefix("range:") }), + let equals = line.range(of: "bytes=") else { return nil } + let parts = line[equals.upperBound...].split(separator: "-", omittingEmptySubsequences: false) + guard parts.count == 2, let start = Int(parts[0].trimmingCharacters(in: .whitespaces)) else { return nil } + return (start, Int(parts[1].trimmingCharacters(in: .whitespaces))) + } + + private func readRequestHead(_ fd: Int32) -> String? { + var collected = Data() + var buffer = [UInt8](repeating: 0, count: 4096) + let terminator = Data("\r\n\r\n".utf8) + while collected.range(of: terminator) == nil { + let count = recv(fd, &buffer, buffer.count, 0) + guard count > 0 else { return nil } + collected.append(contentsOf: buffer.prefix(count)) + guard collected.count <= 64 * 1024 else { return nil } + } + return String(decoding: collected, as: UTF8.self) + } + + private func writeAll(_ fd: Int32, _ bytes: Data) -> Bool { + bytes.withUnsafeBytes { raw in + guard let base = raw.baseAddress else { return true } + var sent = 0 + while sent < raw.count { + let count = write(fd, base.advanced(by: sent), raw.count - sent) + if count < 0, errno == EINTR { continue } + guard count > 0 else { return false } + sent += count + } + return true + } + } +} diff --git a/Tests/AetherEngineTests/Support/ProbeControlTestSupport.swift b/Tests/AetherEngineTests/Support/ProbeControlTestSupport.swift index 0a0351aaf..39cbcd120 100644 --- a/Tests/AetherEngineTests/Support/ProbeControlTestSupport.swift +++ b/Tests/AetherEngineTests/Support/ProbeControlTestSupport.swift @@ -224,9 +224,10 @@ enum ProbeTestFixtures { try decode(AtmosDetectionProbeIntegrationTests.eac3PlainBase64) } - /// Scratch fixtures stay under the checkout, never the system temporary directory. + /// Scratch fixtures live in the temporary directory: one written into the working directory is left + /// in the checkout when the process dies inside the body (audit OPS-109). static func withFile(_ data: Data, _ body: (URL) throws -> T) throws -> T { - let url = URL(fileURLWithPath: FileManager.default.currentDirectoryPath, isDirectory: true) + let url = FileManager.default.temporaryDirectory .appendingPathComponent(".probe-control-\(UUID().uuidString).mp4") defer { try? FileManager.default.removeItem(at: url) } try data.write(to: url) diff --git a/Tests/AetherEngineTests/Support/ProbeHTTPTestOrigin.swift b/Tests/AetherEngineTests/Support/ProbeHTTPTestOrigin.swift index affc5300a..da436dbe6 100644 --- a/Tests/AetherEngineTests/Support/ProbeHTTPTestOrigin.swift +++ b/Tests/AetherEngineTests/Support/ProbeHTTPTestOrigin.swift @@ -21,6 +21,10 @@ final class ProbeHTTPTestOrigin: @unchecked Sendable { let port: UInt16 let blocked = ProbeTestGate() + /// Every request that arrives after `holdLaterRequests()` waits here, before its response head, + /// until the test opens the gate (or `stop()` does). + let held = ProbeTestGate() + private let holding = ProbeTestBox(false) private let listener: Int32 private let wakeRead: Int32 private let wakeWrite: Int32 @@ -74,6 +78,8 @@ final class ProbeHTTPTestOrigin: @unchecked Sendable { } var requests: [Request] { state.value.requests } + + func holdLaterRequests() { holding.update { $0 = true } } var failure: String? { state.value.failure } var isStopped: Bool { let snapshot = state.value @@ -96,6 +102,7 @@ final class ProbeHTTPTestOrigin: @unchecked Sendable { for fd in state.connections { shutdown(fd, SHUT_RDWR) } } blocked.open() + held.open() } private func acceptLoop() { @@ -167,6 +174,7 @@ final class ProbeHTTPTestOrigin: @unchecked Sendable { $0.requests.append(request) } if stage == .headers, index == 0 { park() } + if holding.value { held.wait() } guard !state.value.stopping else { return } var start = 0 diff --git a/Tests/AetherEngineTests/ThrottledOriginServer.swift b/Tests/AetherEngineTests/ThrottledOriginServer.swift index d2735fedf..8706d1d49 100644 --- a/Tests/AetherEngineTests/ThrottledOriginServer.swift +++ b/Tests/AetherEngineTests/ThrottledOriginServer.swift @@ -30,6 +30,12 @@ final class ThrottledOriginServer: @unchecked Sendable { /// ended SHORT of its Content-Length - the observable behind the sequential reader's /// EIO-not-EOF distinction (a lost source must not read as end-of-media). case serveThenDrop(afterBytes: Int64) + /// Audit DMX-5: a 206 that starts `start` rather than where it was asked, the way an edge + /// that aligns ranges to its own chunk boundary answers. The body is that range's. + case serve206From(start: Int64) + /// Audit DMX-101: a server that cannot address bytes. Whatever range was asked for, the + /// answer is a 200 with the whole source from byte 0 and its full Content-Length. + case serve200 } let port: UInt16 @@ -43,6 +49,16 @@ final class ThrottledOriginServer: @unchecked Sendable { /// its own chunk boundary does this. Default off keeps every existing test on the historical /// behaviour. private let ignoreRangeEnd: Bool + /// AE#619: serve `patternByte(at:)` instead of a constant, so a test can check that the bytes + /// the reader returns are the bytes at the offset it claims. Default off keeps every existing + /// test on the historical constant body. + private let patternedBody: Bool + + /// The byte a `patternedBody` origin serves at `offset`. Varies within every 256-byte run and + /// between runs, so a shifted or reordered read cannot match by accident. + static func patternByte(at offset: Int64) -> UInt8 { + UInt8(truncatingIfNeeded: offset ^ (offset >> 8) ^ (offset >> 16) ^ (offset >> 24)) + } private let chunkBytes: Int private let throttleUs: useconds_t private let firstByteDelayUs: @Sendable (_ isSuffix: Bool) -> useconds_t @@ -133,10 +149,12 @@ final class ThrottledOriginServer: @unchecked Sendable { init?(totalSize: Int64, chunkBytes: Int = 256 * 1024, throttleUs: useconds_t = 5000, refuseAboveConcurrency: Int? = nil, ignoreRangeEnd: Bool = false, + patternedBody: Bool = false, firstByteDelayUs: @escaping @Sendable (_ isSuffix: Bool) -> useconds_t = { _ in 0 }, respond: @escaping @Sendable (_ requestIndex: Int, _ offset: Int64, _ path: String) -> Directive = { _, _, _ in .serve206 }) { self.totalSize = totalSize self.ignoreRangeEnd = ignoreRangeEnd + self.patternedBody = patternedBody self.chunkBytes = chunkBytes self.throttleUs = throttleUs self.refuseAboveConcurrency = refuseAboveConcurrency @@ -299,9 +317,15 @@ final class ThrottledOriginServer: @unchecked Sendable { var silentAfter: Int64? = nil var dropAfter: Int64? = nil + var answersWholeSource = false switch respond(requestIndex, offset, path) { case .serve206: break + case .serve200: + answersWholeSource = true + offset = 0 + case .serve206From(let start): + offset = max(0, min(start, totalSize - 1)) case .serveThenGoSilent(let afterBytes): silentAfter = max(0, afterBytes) case .serveThenDrop(let afterBytes): @@ -331,15 +355,19 @@ final class ThrottledOriginServer: @unchecked Sendable { pendingDelay -= slice } - let last = (ignoreRangeEnd ? nil : rangeEnd) ?? (totalSize - 1) + let last = answersWholeSource ? totalSize - 1 : (ignoreRangeEnd ? nil : rangeEnd) ?? (totalSize - 1) let remaining = last - offset + 1 // Keep-alive, not close: a bounded range that tears the socket down would make every // refill a fresh connection and would hide exactly the pooling question under test. - let header = "HTTP/1.1 206 Partial Content\r\n" - + "Content-Range: bytes \(offset)-\(last)/\(totalSize)\r\n" - + "Content-Length: \(remaining)\r\n" - + "Accept-Ranges: bytes\r\n" - + "Connection: keep-alive\r\n\r\n" + let header = answersWholeSource + ? "HTTP/1.1 200 OK\r\n" + + "Content-Length: \(remaining)\r\n" + + "Connection: keep-alive\r\n\r\n" + : "HTTP/1.1 206 Partial Content\r\n" + + "Content-Range: bytes \(offset)-\(last)/\(totalSize)\r\n" + + "Content-Length: \(remaining)\r\n" + + "Accept-Ranges: bytes\r\n" + + "Connection: keep-alive\r\n\r\n" guard writeFully(fd, Array(header.utf8)) else { return false } let chunk = [UInt8](repeating: 0x55, count: chunkBytes) @@ -370,7 +398,10 @@ final class ThrottledOriginServer: @unchecked Sendable { var n = Int(min(Int64(chunkBytes), remaining - served)) if let silentAfter { n = Int(min(Int64(n), silentAfter - served)) } if let dropAfter { n = Int(min(Int64(n), dropAfter - served)) } - guard writeBody(fd, Array(chunk[0.. 0 { usleep(throttleUs) } } diff --git a/Tests/AetherEngineTests/TinyTransportStreamFixture.swift b/Tests/AetherEngineTests/TinyTransportStreamFixture.swift new file mode 100644 index 000000000..530d28f76 --- /dev/null +++ b/Tests/AetherEngineTests/TinyTransportStreamFixture.swift @@ -0,0 +1,152 @@ +import Foundation +@testable import AetherEngine + +/// An 8 s H.264 MPEG-TS (16x16, 2 fps, an IDR with in-band SPS/PPS every second, PTS from 1.4 s), +/// small enough to embed so the live and pump tests run on CI without a fixture fetch. Regenerate with: +/// +/// ffmpeg -f lavfi -i "color=c=black:s=16x16:r=2:d=8" -c:v libx264 -preset ultrafast \ +/// -profile:v baseline -crf 51 -g 2 -keyint_min 2 -sc_threshold 0 \ +/// -x264-params "bframes=0:repeat-headers=1" -pix_fmt yuv420p -f mpegts \ +/// -mpegts_service_type digital_tv -pat_period 60 -sdt_period 60 tiny-live.ts +enum TinyTransportStreamFixture { + + /// Video keyframes in the stream, one per second. + static let keyframeCount = 8 + + static var data: Data { Data(base64Encoded: base64.joined())! } + + /// The stream as a live source serves it: from byte 0 to its end, then EOF, with no length and + /// no disc probe (a disc probe reads ahead, and a live open does not rewind after it). + final class LiveReader: IOReader, @unchecked Sendable { + private let bytes = TinyTransportStreamFixture.data + private let lock = NSLock() + private var position = 0 + + func read(_ buffer: UnsafeMutablePointer?, size: Int32) -> Int32 { + guard let buffer, size > 0 else { return -1 } + return lock.withLock { + let n = min(Int(size), bytes.count - position) + guard n > 0 else { return 0 } + bytes.withUnsafeBytes { raw in + _ = memcpy(buffer, raw.baseAddress! + position, n) + } + position += n + return Int32(n) + } + } + + func seek(offset: Int64, whence: Int32) -> Int64 { + lock.withLock { + switch whence { + case SEEK_SET where offset >= 0 && offset <= Int64(bytes.count): + position = Int(offset) + case SEEK_CUR where offset == 0: + break + default: + return -1 + } + return Int64(position) + } + } + + func close() {} + var discImageProbeEnabled: Bool { false } + } + + private static let base64 = [ + "R0AREABC8CUAAcEAAP8B/wAB/IAUSBIBBkZGbXBlZwlTZXJ2aWNlMDF3fEPK////////////////////////////////////", + "////////////////////////////////////////////////////////////////////////////////////////////////", + "//////////////////////////////////////////////////////////9HQAAQAACwDQABwQAAAAHwACqxBLL/////////", + "////////////////////////////////////////////////////////////////////////////////////////////////", + "////////////////////////////////////////////////////////////////////////////////////////////////", + "/////////////////////0dQABAAArASAAHBAADhAPAAG+EA8AAVvU1W////////////////////////////////////////", + "////////////////////////////////////////////////////////////////////////////////////////////////", + "////////////////////////////////////////////////////////////////////////////////R0EAMAdQAAB7DH4A", + "AAAB4AAAgIAFIQAH2GEAAAABCfAAAAABZ0LACtp7ARAAAAMAEAAAAwBA8SJqAAAAAWjOAZcgAAABBgX//03cRem95tlIt5Ys", + "2CDZI+7veDI2NCAtIGNvcmUgMTY1IHIzMjIyIGIzNTYwNWEgLSBILjI2NC9NUEVHLTQgQVZDIGNvZGVjIC0gQ29weWxlZnQg", + "MjAwMy0yMDI1IC0gaHR0cDovL3d3dy52aWRlb2xhbi5HAQARb3JnL3gyNjQuaHRtbCAtIG9wdGlvbnM6IGNhYmFjPTAgcmVm", + "PTEgZGVibG9jaz0wOjA6MCBhbmFseXNlPTA6MCBtZT1kaWEgc3VibWU9MCBwc3k9MSBwc3lfcmQ9MS4wMDowLjAwIG1peGVk", + "X3JlZj0wIG1lX3JhbmdlPTE2IGNocm9tYV9tZT0xIHRyZWxsaXM9MCA4eDhkY3Q9MCBjcW09MCBkZWFkem9uZT0yMSwxMSBm", + "YXN0X0cBABJwc2tpcD0xIGNocm9tYV9xcF9vZmZzZXQ9MCB0aHJlYWRzPTEgbG9va2FoZWFkX3RocmVhZHM9MSBzbGljZWRf", + "dGhyZWFkcz0wIG5yPTAgZGVjaW1hdGU9MSBpbnRlcmxhY2VkPTAgYmx1cmF5X2NvbXBhdD0wIGNvbnN0cmFpbmVkX2ludHJh", + "PTAgYmZyYW1lcz0wIHdlaWdodHA9MCBrZXlpbnQ9MiBrZXlpbnRfbWluPTIgc2NlRwEAM0EA////////////////////////", + "/////////////////////////////////////////////////////////////25lY3V0PTAgaW50cmFfcmVmcmVzaD0wIHJj", + "PWNyZiBtYnRyZWU9MCBjcmY9NTEuMCBxY29tcD0wLjYwIHFwbWluPTAgcXBtYXg9NjkgcXBzdGVwPTQgaXBfcmF0aW89MS40", + "MCBhcT0wAIAAAAFliIQ6JigAFcBHQQA0mhAAANLwfgD/////////////////////////////////////////////////////", + "////////////////////////////////////////////////////////////////////////////////////////////////", + "//////////////////////////////////////////////8AAAHgAACAgAUhAAs38QAAAAEJ8AAAAAFBmiASpUdAABEAALAN", + "AAHBAAAAAfAAKrEEsv//////////////////////////////////////////////////////////////////////////////", + "////////////////////////////////////////////////////////////////////////////////////////////////", + "////////////////////////////////////////////////R1AAEQACsBIAAcEAAOEA8AAb4QDwABW9TVb/////////////", + "////////////////////////////////////////////////////////////////////////////////////////////////", + "////////////////////////////////////////////////////////////////////////////////////////////////", + "//////////9HQQA1c1AAASrUfgD/////////////////////////////////////////////////////////////////////", + "//////////////////////////////////////////////////////////////////////////8AAAHgAACAgAUhAA2XgQAA", + "AAEJ8AAAAAFnQsAK2nsBEAAAAwAQAAADAEDxImoAAAABaM4BlyAAAAFliIIDKJigACA/gEdBADaaEAABgrh+AP//////////", + "////////////////////////////////////////////////////////////////////////////////////////////////", + "/////////////////////////////////////////////////////////////////////////////////////////wAAAeAA", + "AICABSEAD/cRAAAAAQnwAAAAAUGaIBKlR0AAEgAAsA0AAcEAAAAB8AAqsQSy////////////////////////////////////", + "////////////////////////////////////////////////////////////////////////////////////////////////", + "//////////////////////////////////////////////////////////////////////////////////////////9HUAAS", + "AAKwEgABwQAA4QDwABvhAPAAFb1NVv//////////////////////////////////////////////////////////////////", + "////////////////////////////////////////////////////////////////////////////////////////////////", + "/////////////////////////////////////////////////////0dBADd0UAAB2px+AP//////////////////////////", + "////////////////////////////////////////////////////////////////////////////////////////////////", + "//////////////////////8AAAHgAACAgAUhABNWoQAAAAEJ8AAAAAFnQsAK2nsBEAAAAwAQAAADAEDxImoAAAABaM4BlyAA", + "AAFliIQOomKAAIGOR0EAOJoQAAIygH4A////////////////////////////////////////////////////////////////", + "////////////////////////////////////////////////////////////////////////////////////////////////", + "////////////////////////////////////AAAB4AAAgIAFIQAVtjEAAAABCfAAAAABQZogEqVHQAATAACwDQABwQAAAAHw", + "ACqxBLL/////////////////////////////////////////////////////////////////////////////////////////", + "////////////////////////////////////////////////////////////////////////////////////////////////", + "/////////////////////////////////////0dQABMAArASAAHBAADhAPAAG+EA8AAVvU1W////////////////////////", + "////////////////////////////////////////////////////////////////////////////////////////////////", + "////////////////////////////////////////////////////////////////////////////////////////////////", + "R0EAOXNQAAKKZH4A////////////////////////////////////////////////////////////////////////////////", + "////////////////////////////////////////////////////////////////AAAB4AAAgIAFIQAZFcEAAAABCfAAAAAB", + "Z0LACtp7ARAAAAMAEAAAAwBA8SJqAAAAAWjOAZcgAAABZYiCAQomKAAIIOBHQQA6mhAAAuJIfgD/////////////////////", + "////////////////////////////////////////////////////////////////////////////////////////////////", + "//////////////////////////////////////////////////////////////////////////////8AAAHgAACAgAUhABt1", + "UQAAAAEJ8AAAAAFBmiASpUdAABQAALANAAHBAAAAAfAAKrEEsv//////////////////////////////////////////////", + "////////////////////////////////////////////////////////////////////////////////////////////////", + "////////////////////////////////////////////////////////////////////////////////R1AAFAACsBIAAcEA", + "AOEA8AAb4QDwABW9TVb/////////////////////////////////////////////////////////////////////////////", + "////////////////////////////////////////////////////////////////////////////////////////////////", + "//////////////////////////////////////////9HQQA7c1AAAzosfgD/////////////////////////////////////", + "////////////////////////////////////////////////////////////////////////////////////////////////", + "//////////8AAAHgAACAgAUhAB3U4QAAAAEJ8AAAAAFnQsAK2nsBEAAAAwAQAAADAEDxImoAAAABaM4BlyAAAAFliIQEKJig", + "ACCDgEdBADyaEAADkhB+AP//////////////////////////////////////////////////////////////////////////", + "////////////////////////////////////////////////////////////////////////////////////////////////", + "/////////////////////////wAAAeAAAICABSEAITRxAAAAAQnwAAAAAUGaIBKlR0AAFQAAsA0AAcEAAAAB8AAqsQSy////", + "////////////////////////////////////////////////////////////////////////////////////////////////", + "////////////////////////////////////////////////////////////////////////////////////////////////", + "//////////////////////////9HUAAVAAKwEgABwQAA4QDwABvhAPAAFb1NVv//////////////////////////////////", + "////////////////////////////////////////////////////////////////////////////////////////////////", + "/////////////////////////////////////////////////////////////////////////////////////0dBAD1zUAAD", + "6fR+AP//////////////////////////////////////////////////////////////////////////////////////////", + "/////////////////////////////////////////////////////wAAAeAAAICABSEAI5QBAAAAAQnwAAAAAWdCwAraewEQ", + "AAADABAAAAMAQPEiagAAAAFozgGXIAAAAWWIggEaJigACCfgR0EAPpoQAARB2H4A////////////////////////////////", + "////////////////////////////////////////////////////////////////////////////////////////////////", + "////////////////////////////////////////////////////////////////////AAAB4AAAgIAFIQAl85EAAAABCfAA", + "AAABQZogEqVHQAAWAACwDQABwQAAAAHwACqxBLL/////////////////////////////////////////////////////////", + "////////////////////////////////////////////////////////////////////////////////////////////////", + "/////////////////////////////////////////////////////////////////////0dQABYAArASAAHBAADhAPAAG+EA", + "8AAVvU1W////////////////////////////////////////////////////////////////////////////////////////", + "////////////////////////////////////////////////////////////////////////////////////////////////", + "////////////////////////////////R0EAP3NQAASZvH4A////////////////////////////////////////////////", + "////////////////////////////////////////////////////////////////////////////////////////////////", + "AAAB4AAAgIAFIQApUyEAAAABCfAAAAABZ0LACtp7ARAAAAMAEAAAAwBA8SJqAAAAAWjOAZcgAAABZYiEBGiYoAAgn4BHQQAw", + "mhAABPGgfgD/////////////////////////////////////////////////////////////////////////////////////", + "////////////////////////////////////////////////////////////////////////////////////////////////", + "//////////////8AAAHgAACAgAUhACuysQAAAAEJ8AAAAAFBmiASpUdAABcAALANAAHBAAAAAfAAKrEEsv//////////////", + "////////////////////////////////////////////////////////////////////////////////////////////////", + "////////////////////////////////////////////////////////////////////////////////////////////////", + "////////////////R1AAFwACsBIAAcEAAOEA8AAb4QDwABW9TVb/////////////////////////////////////////////", + "////////////////////////////////////////////////////////////////////////////////////////////////", + "//////////////////////////////////////////////////////////////////////////9HQQAxc1AABUmEfgD/////", + "////////////////////////////////////////////////////////////////////////////////////////////////", + "//////////////////////////////////////////8AAAHgAACAgAUhAC8SQQAAAAEJ8AAAAAFnQsAK2nsBEAAAAwAQAAAD", + "AEDxImoAAAABaM4BlyAAAAFliIIBKiYoAAgv4EdBADKaEAAFoWh+AP//////////////////////////////////////////", + "////////////////////////////////////////////////////////////////////////////////////////////////", + "/////////////////////////////////////////////////////////wAAAeAAAICABSEAMXHRAAAAAQnwAAAAAUGaIBKl", + ] +} diff --git a/Tests/AetherEngineTests/UDFReaderTests.swift b/Tests/AetherEngineTests/UDFReaderTests.swift index 905aa87b7..688afaba8 100644 --- a/Tests/AetherEngineTests/UDFReaderTests.swift +++ b/Tests/AetherEngineTests/UDFReaderTests.swift @@ -94,6 +94,38 @@ final class UDFReaderTests: XCTestCase { } } + // audit NET-4: `off + len <= lvd.count` only bounds the map's OWN declared length; a type 1 or + // type 2 map short enough to pass that check but too short to hold the fields the reader + // indexes (off+4, off+38..43) used to read past the sector buffer and trap once a run of + // padding maps pushed `off` near the sector's end. + func test_craftedPartitionMapDoesNotReadPastSector() { + var data = [UInt8](image()) + let lvdBase = 258 * 2048 + var maps: [UInt8] = [] + for _ in 0..<6 { maps += [0, 255] + [UInt8](repeating: 0, count: 253) } // type 0, len 255 (padding) + maps += [0, 74] + [UInt8](repeating: 0, count: 72) // type 0, len 74 (padding) + maps += [1, 2] // type 1, len 2: too short for off+4 + XCTAssertLessThanOrEqual(440 + maps.count, 2048, "the crafted layout must still fit the sector") + for (i, b) in maps.enumerated() { data[lvdBase + 440 + i] = b } + for (i, b) in UDFFixture.le32(maps.count).enumerated() { data[lvdBase + 264 + i] = b } // MapTableLength + for (i, b) in UDFFixture.le32(8).enumerated() { data[lvdBase + 268 + i] = b } // NumberOfPartitionMaps + XCTAssertThrowsError(try UDFReader(reader: DataIOReader(data: Data(data)))) + } + + // audit NET-11: `vdsLen` is an untrusted u32 from the anchor; unclamped it can drive up to + // ~2 million sequential sector reads before parsing fails. The scan must stay capped near the + // real Volume Descriptor Sequence's size (ECMA-167: 16 sectors) instead of trusting the anchor. + func test_hostileAnchorLengthCapsTheVDSScan() { + let mock = SparseSectorReader() + var avdp = [UInt8](repeating: 0, count: 2048) + UDFFixture.tag(2, location: 256, into: &avdp) + avdp[16..<24] = ArraySlice(UDFFixture.extentAD(lenBytes: 0xFFFF_FFFF, location: 300)) + mock.setSector(256, avdp) + XCTAssertThrowsError(try UDFReader(reader: mock)) + XCTAssertTrue((200...300).contains(mock.sectorReads), + "expected the VDS scan capped near 256 sectors, got \(mock.sectorReads)") + } + func test_truncatedImageThrowsNotTrap() throws { // UDF image truncated before VDS (AVDP present at sector 256, VDS cut off): must throw, not trap. func be16(_ v: Int) -> [UInt8] { [UInt8((v>>8)&0xff), UInt8(v&0xff)] } @@ -117,3 +149,37 @@ final class UDFReaderTests: XCTestCase { }()) } } + +/// Answers every sector as zero-filled unless a specific one was set, and counts reads: proves a +/// scan is bounded rather than trusting an untrusted extent length (audit NET-11). +private final class SparseSectorReader: IOReader, @unchecked Sendable { + private let ss = 2048 + private var sectors: [Int: [UInt8]] = [:] + private var position: Int64 = 0 + private let lock = NSLock() + private var _sectorReads = 0 + var sectorReads: Int { lock.lock(); defer { lock.unlock() }; return _sectorReads } + + func setSector(_ index: Int, _ bytes: [UInt8]) { sectors[index] = bytes } + + func read(_ buffer: UnsafeMutablePointer?, size: Int32) -> Int32 { + guard let buffer, size > 0 else { return -1 } + lock.lock(); defer { lock.unlock() } + let sector = Int(position / Int64(ss)) + let bytes = sectors[sector] ?? [UInt8](repeating: 0, count: ss) + let n = min(Int(size), bytes.count) + bytes.withUnsafeBufferPointer { buffer.update(from: $0.baseAddress!, count: n) } + position += Int64(n) + _sectorReads += 1 + return Int32(n) + } + + func seek(offset: Int64, whence: Int32) -> Int64 { + if whence == 65536 { return Int64.max / 2 } + guard whence == SEEK_SET else { return -1 } + position = offset + return position + } + + func close() {} +} diff --git a/Tests/AetherEngineTests/VideoRoutingPolicyTests.swift b/Tests/AetherEngineTests/VideoRoutingPolicyTests.swift index 32ae48081..770f28135 100644 --- a/Tests/AetherEngineTests/VideoRoutingPolicyTests.swift +++ b/Tests/AetherEngineTests/VideoRoutingPolicyTests.swift @@ -67,6 +67,21 @@ struct VideoRoutingPolicyTests { codecID: AV_CODEC_ID_AV1, fieldOrder: AV_FIELD_PROGRESSIVE, av1Available: true)) } + @Test("Only AV1 routing evaluates the hardware capability probe") + func hardwareProbeIsLazy() { + var calls = 0 + func probe() -> Bool { calls += 1; return true } + for codec in [AV_CODEC_ID_H264, AV_CODEC_ID_HEVC, AV_CODEC_ID_NONE, + AV_CODEC_ID_VP9, AV_CODEC_ID_MPEG2VIDEO] { + _ = VideoRoutingPolicy.requiresSoftwarePath( + codecID: codec, fieldOrder: AV_FIELD_PROGRESSIVE, av1Available: probe()) + } + #expect(calls == 0) + #expect(!VideoRoutingPolicy.requiresSoftwarePath( + codecID: AV_CODEC_ID_AV1, fieldOrder: AV_FIELD_PROGRESSIVE, av1Available: probe())) + #expect(calls == 1) + } + // MARK: - #150 SPS frame_mbs_only fallback for UNKNOWN field order @Test("UNKNOWN field order + SPS frame_mbs_only=0 routes to software") @@ -148,16 +163,54 @@ struct VideoRoutingPolicyTests { codecID: AV_CODEC_ID_HEVC, dvProfile: nil, canHardwareDecode: { true })) } - @Test("non-H.264/HEVC codecs ignore the undecodable gate (own policy governs them)") + @Test("codecs outside the gate ignore it (own policy governs them)") func otherCodecsIgnoreUndecodableGate() { - // AV1 without HW is already routed software by requiresSoftwarePath; this gate must not double-handle - // it, and must never reclassify VP9 / MPEG-2 / etc. off a codec they don't apply to. - for codec in [AV_CODEC_ID_AV1, AV_CODEC_ID_VP9, AV_CODEC_ID_MPEG2VIDEO, AV_CODEC_ID_VC1] { + // Must never reclassify VP9 / MPEG-2 / etc. off a codec it doesn't apply to. + for codec in [AV_CODEC_ID_VP9, AV_CODEC_ID_MPEG2VIDEO, AV_CODEC_ID_VC1] { #expect(!VideoRoutingPolicy.forcesSoftwareForUndecodableFormat( codecID: codec, dvProfile: nil, canHardwareDecode: { false })) } } + // MARK: - Audit HLS-5: HW AV1 is Main profile only + + @Test("AV1 on a HW-AV1 device the decoder cannot take falls back to software") + func av1OutsideHardwareProfileSoftware() { + #expect(VideoRoutingPolicy.forcesSoftwareForUndecodableFormat( + codecID: AV_CODEC_ID_AV1, dvProfile: nil, canHardwareDecode: { false })) + #expect(!VideoRoutingPolicy.forcesSoftwareForUndecodableFormat( + codecID: AV_CODEC_ID_AV1, dvProfile: nil, canHardwareDecode: { true })) + } + + @Test("live consults the format gate for AV1 only; VOD consults it for every codec") + func liveConsultsFormatGateForAV1Only() { + #expect(VideoRoutingPolicy.consultsUndecodableFormatGate(codecID: AV_CODEC_ID_AV1, isLive: true)) + #expect(!VideoRoutingPolicy.consultsUndecodableFormatGate(codecID: AV_CODEC_ID_H264, isLive: true)) + #expect(!VideoRoutingPolicy.consultsUndecodableFormatGate(codecID: AV_CODEC_ID_HEVC, isLive: true)) + for codec in [AV_CODEC_ID_AV1, AV_CODEC_ID_H264, AV_CODEC_ID_HEVC] { + #expect(VideoRoutingPolicy.consultsUndecodableFormatGate(codecID: codec, isLive: false)) + } + } + + @Test("av1C seq_profile decides: Main fits the hardware decoder, High and Professional do not") + func av1ProfileFromAV1C() { + // marker|version 0x81, then seq_profile (3 bits) | seq_level_idx_0 (5 bits), flags, reserved. + let main10 = [UInt8]([0x81, 0x08, 0x4C, 0x00]) // profile 0, high_bitdepth + let high444 = [UInt8]([0x81, 0x28, 0x00, 0x00]) // profile 1 + let pro12bit = [UInt8]([0x81, 0x48, 0x60, 0x00]) // profile 2, high_bitdepth + twelve_bit + #expect(VideoRoutingPolicy.av1FitsHardwareDecoder(av1C: main10, codecparProfile: -99)) + #expect(!VideoRoutingPolicy.av1FitsHardwareDecoder(av1C: high444, codecparProfile: -99)) + #expect(!VideoRoutingPolicy.av1FitsHardwareDecoder(av1C: pro12bit, codecparProfile: -99)) + } + + @Test("without an av1C the codecpar profile decides, and unknown keeps the native route") + func av1ProfileFallback() { + #expect(VideoRoutingPolicy.av1FitsHardwareDecoder(av1C: nil, codecparProfile: 0)) + #expect(!VideoRoutingPolicy.av1FitsHardwareDecoder(av1C: nil, codecparProfile: 1)) + #expect(!VideoRoutingPolicy.av1FitsHardwareDecoder(av1C: [], codecparProfile: 2)) + #expect(VideoRoutingPolicy.av1FitsHardwareDecoder(av1C: nil, codecparProfile: -99)) + } + // MARK: - #176 DV Profile 5 bypasses the VT probe gate @Test("HEVC DV P5 stays native even when the raw-hvcC probe says undecodable") diff --git a/Tests/AetherEngineTests/WholeSecondsCoveringTotalityTests.swift b/Tests/AetherEngineTests/WholeSecondsCoveringTotalityTests.swift new file mode 100644 index 000000000..2c9a43f06 --- /dev/null +++ b/Tests/AetherEngineTests/WholeSecondsCoveringTotalityTests.swift @@ -0,0 +1,39 @@ +import Testing +@testable import AetherEngine + +/// `wholeSecondsCovering` feeds `#EXT-X-TARGETDURATION` and the holdback. A duration from a broken source +/// used to reach `Int(_:)` unchecked, which traps on infinity, NaN and anything beyond `Int.max`. +@Suite("wholeSecondsCovering is total") +struct WholeSecondsCoveringTotalityTests { + + @Test("Finite durations keep their covering whole second", arguments: [ + (2.0, 2), (2.0000000000000004, 2), (2.0005, 3), (0.2, 1), (6.5, 7), + ]) + func finiteDurationsUnchanged(seconds: Double, expected: Int) { + #expect(LiveEdgePolicy.wholeSecondsCovering(seconds) == expected) + } + + @Test("Hostile durations yield a bounded value instead of trapping", arguments: [ + Double.infinity, 1e300, Double(Int.max), Double.greatestFiniteMagnitude, + ]) + func unboundedSaturates(seconds: Double) { + #expect(LiveEdgePolicy.wholeSecondsCovering(seconds) == LiveEdgePolicy.maxCoveredWholeSeconds) + } + + @Test("NaN, zero and negative durations cover nothing", arguments: [ + Double.nan, -Double.infinity, -1e300, -3.0, 0.0, + ]) + func emptyCoversNothing(seconds: Double) { + #expect(LiveEdgePolicy.wholeSecondsCovering(seconds) == 0) + } + + @Test("The target duration survives a non-finite term, and the holdback stays computable") + func targetDurationSurvivesHostileTerms() { + let td = LiveEdgePolicy.targetDurationSeconds( + maxSegmentDuration: .infinity, cutTargetSeconds: .nan, cadenceFloorSeconds: .infinity) + #expect(td == LiveEdgePolicy.maxCoveredWholeSeconds) + #expect(LiveEdgePolicy.holdBackSeconds(targetDuration: td) == Double(3 * td)) + #expect(LiveEdgePolicy.targetDurationSeconds( + maxSegmentDuration: 2.0, cutTargetSeconds: .nan, cadenceFloorSeconds: .nan) == 2) + } +} diff --git a/docs/api.md b/docs/api.md index 4d04f8d47..e29f2b49f 100644 --- a/docs/api.md +++ b/docs/api.md @@ -35,13 +35,15 @@ A working shape for the live contracts below, compiled against the engine: [`Exa | --- | --- | --- | | The source could not be opened, probed, or routed | throws | `.error(message)` is published as well | | A newer `load()` or a `stop()` superseded this one | throws `CancellationError` | belongs to the successor, untouched | +| The Task awaiting it was cancelled while it ran (Sodalite#173) | throws `CancellationError` | `.idle`; the native host is kept for the next load, as a newer `load()` would keep it | +| AVPlayer refused the media during startup and the engine rebuilt the session on the software path (AE#561) | keeps waiting, then returns (or throws what the rebuild threw) | whatever the rebuilt session reaches; `softwarePathEscalations` fires with `duringStartup` | | A custom `IOReader` whose initial probe failed | throws | `.error` | | Dolby Vision with no compatible base layer on the software path | throws `AetherEngineError.dolbyVisionUnplayableOnSoftwarePath` | `.error` | | An HLS playlist handed to the raw live path by a custom reader | throws `AetherEngineError.hlsPlaylistOnRawLivePath` | `.error` | | The session died after the load returned (source loss, a reload that never became ready, a track switch that failed) | already returned | `.error(message)` only | | A live session the engine cannot revive | already returned | no `.error`; `liveSourceReset` fires instead | -**`CancellationError` is not a playback failure.** It is what a superseded load throws at its first checkpoint, so every channel zap, every next-episode call and every `stop()` during a load produces one. A host that retries, falls back to a second engine, or shows an error on "load threw" reacts to its own navigation unless it lets `CancellationError` through untouched. +**`CancellationError` is not a playback failure.** It is what a superseded load throws at its first checkpoint, so every channel zap, every next-episode call and every `stop()` during a load produces one. A load the ENGINE superseded does not throw it: the software-path rebuild below takes over the startup, and the `load()` the host is awaiting follows it, the way a #361 reroute keeps its wait (AE#629). A host that retries, falls back to a second engine, or shows an error on "load threw" reacts to its own navigation unless it lets `CancellationError` through untouched. The message inside `.error` is worth logging verbatim, and it comes from two different places. Some are the engine's own sentence and name the cause rather than the symptom (`"AVFoundation built no track for it within 45s and the source's carriage could not be identified"`, `"Live source unavailable"`, the Dolby Vision hardware refusal); a host timeout that fires first replaces that sentence with its own. The rest are forwarded from the failure underneath, and on the native paths that is `AVPlayerItem.error.localizedDescription` verbatim, which AVFoundation localizes into the device language and whose `NSError` domain and code reach the host only as whatever the localized text happens to embed. @@ -75,7 +77,7 @@ It is assigned before `state`, so a `$state` sink reads this failure's own info To replay, call `load(...)` again. The engine keeps `.ended` terminal deliberately: a play press racing a host's end card must not silently restart a finished session (#63/#164). A VOD **parked at its final frame** without having ended (scrubbed there, paused there) is the other case and does resume: `play()` rewinds to the start first. -The clock stops with it, on both backends: `clock.currentTime` and `clock.sourceTime` settle on the last sample and stay there, so a progress bar bound to them holds at the end instead of walking past `duration`. Through 6.28.0 the software path was the exception: its master clock kept its rate past end of media, so a session left standing published a position that grew without bound (20.13 s on a 12.0 s source after 20 s, against 11.97 s from a native session on the same file). A host reading the clock after `.ended` on one of those builds is reading that, not a drifting session (AE#374). +The clock stops with it, on both backends: `clock.currentTime` and `clock.sourceTime` settle on the last sample and stay there, so a progress bar bound to them holds at the end instead of walking past `duration`. Through 6.28.0 the software path was the exception: its master clock kept its rate past end of media, so a session left standing published a position that grew without bound (20.13 s on a 12.0 s source after 20 s, against 11.97 s from a native session on the same file). A host reading the clock after `.ended` on one of those builds is reading that, not a drifting session (AE#374). The audio-only path (a source without a video track decoded through FFmpeg) had the same defect through 7.27.0 and parks the same way since (AE#694). ### Live: the retune request @@ -118,6 +120,21 @@ player.systemCaptionRequest // PassthroughSubject; subscribe per session +options.escalatesToSoftwarePath // default true; false declines the rebuild +``` + +When AVPlayer fails a native item with a verdict on the MEDIA (`CoreMediaErrorDomain`, typically `-19602` on a sample Apple's parser refuses), every native recovery reloads the same bytes, so the engine spends one rebuild per session on `SoftwarePlaybackHost` at the session's playhead instead of making the failure terminal (AE#561). libavcodec skips the frame Apple refused and plays on. It is not taken on the remote-HLS bypass, on a session already asked onto `.software`, or for a URL-loading failure, which both paths would read through the same reader; whether the software path can serve the source at all is the same check a `preferredDecodePath` correction runs. + +The event fires the moment the rebuild is taken and carries the failure it absorbed, in the `PlaybackErrorInfo` shape it would have surfaced in (`kind` `.nativeItemFailed`), plus the position and whether a `load()` was still waiting. If the rebuild then fails, the original failure is published as `.error` the usual way; if it succeeds, `videoRoute` moves to `.software`. A startup take does not end the host's `load()` with a `CancellationError`: that call keeps waiting across the rebuild and returns the probe it already assembled. + +A host that re-plans a failing title with a ladder of its own (a transcode rung, a second player) sets `escalatesToSoftwarePath = false` and gets the failure as `.error` instead, as before 7.9.0. It is a tuning field, correctable through `reloadAtCurrentPosition(applying:)`. + +A live join takes the same rung when the source delivers video for the whole keyframe wait (15 s) and none of it is a picture the native route can open a segment on: no IDR and no recovery point with `recovery_frame_cnt` 0, which is what a feed carrying only gradual intra refresh looks like (AE#627). Reopening would join the same bitstream, so the first join does not spend its reopen cycles on it. The absorbed failure carries the domain `AetherEngine.LiveJoin`. With `escalatesToSoftwarePath = false`, or when the rung is not on offer, the join ends as `liveSourceReset` at that point, as the reopen cycles would have ended it a minute later. A starvation after the session has produced segments keeps its reopens. + ### The audio tap ends with its session `installAudioTap()` returns an `AsyncStream` bound to the session it was installed against. It finishes on `load()`, on `stop()`, and on a **session-preserving reload**: an audio-track switch, a subtitle-track switch, a disc-title switch, `reloadAtCurrentPosition()`. Re-install on stream end to follow the new session (#356). This is deliberate rather than a gap: each install gets a fresh monotonic filter, so the new session's timeline starts clean instead of stitched across a reload that resumes slightly behind the old position. @@ -298,8 +315,9 @@ try await player.reloadAtCurrentPosition { $0.preferredDecodePath = .software } the native path. That is the right default, and occasionally wrong. When VideoToolbox then cannot build a decoder for what arrives, the item reaches `readyToPlay` and renders nothing, and in-band parameter sets (`hev1` / `avc1` with an empty config record) are the class where the deciding -evidence genuinely is not present at load time. A live load never reaches that gate at all: the -capability check is VOD-only, so a live session keeps the native path with no classification step. +evidence genuinely is not present at load time. A live H.264 / HEVC load never reaches that gate at +all: for those codecs the capability check is VOD-only, so a live session keeps the native path with +no classification step. Live AV1 is judged by profile like VOD (High and Professional go software). Detecting the symptom is a host's own job and is not hard (`AVPlayerItemVideoOutput.hasNewPixelBuffer` against `softwareHostFramesEnqueued`, both already exposed). The escape was the missing half. Before @@ -338,17 +356,21 @@ typed fact rather than as something to reconstruct (AE#462). | Value | Meaning | | --- | --- | | `.none` | no session | -| `.noAudioInSource` | the source carries no audio track, or none was selected | +| `.noAudioInSource` | the source carries no audio track | | `.streamCopy` | the source bitstream is muxed into fMP4 unchanged (Atmos, DTS-HD and everything else reach the renderer as authored) | | `.bridged` | decoded and re-encoded for the fMP4 pipeline: to FLAC (lossless bed channels) or E-AC-3, where object metadata does not survive the PCM intermediate, or, under `objectAudioRendering`, to lossy APAC with the TrueHD Atmos objects rendered into the speaker bed | | `.decoded` | libavcodec decodes and the engine renders it (the software path, the FFmpeg audio-only host) | -| `.droppedNoPipeline` | the source HAS audio and none of it could be delivered: no decoder for it in this build, or the bridge could not be built or could not write its header. The session plays video-only and silently | +| `.droppedNoPipeline` | the source HAS audio and none of it could be delivered: no decoder for it in this build, the bridge could not be built or could not write its header, the probe left its parameters empty so no stream could be picked, or (live only) the bridge was built and its decoder produced nothing, after which the engine rebuilds the session without the track (AE#641). The session plays video-only and silently | | `.playerManaged` | AVFoundation owns the audio (the remote-HLS bypass, the native audio-only host). The engine has no pipeline of its own to classify and does not answer on AVFoundation's behalf | **`.droppedNoPipeline` is the one a fallback ladder acts on**, the same way it demotes on `PlaybackErrorKind.audioBridgeProducedNoOutput`. The two are the same user outcome from opposite -ends of the cascade: that kind fails loudly when a bridge WAS built and then decoded nothing, this -value reports a bridge that could never be built at all. Neither ends a ladder: re-serving the +ends of the cascade: that kind fails loudly when a VOD bridge WAS built and then decoded nothing +(on the FLAC route as well since AE#641, which used to play silently while reporting `.bridged`), +this value reports a bridge that could never be built at all. A live session whose bridge decodes +nothing arrives here as well rather than at the error: its video is playable and a live source has +no position to hand to a second player, so the engine rebuilds it video-only by itself and the +value changes from `.bridged` to `.droppedNoPipeline` (AE#641). Neither ends a ladder: re-serving the source with audio the pipeline can carry (a server-side transcode, a second player that decodes it itself) plays it. @@ -379,6 +401,7 @@ arrive together. | `LoadOptions.prepareNativeSubtitles`, `externalSubtitles` | the native renditions are declared in the init segment | | `LoadOptions.panelIsInHDRMode`, `panelPresentsDolbyVision`, `matchContentEnabled` | the display-criteria handshake and the format clamp both run synchronously inside `load` | | `pictureInPictureActive` | governs the background teardown decision at the moment it happens | +| `logTag` | entered into the shared-output book when the load joins it, so a tag set later is missing from those log lines until the next `load()` | ### Isolation, and what runs off the main actor @@ -450,6 +473,18 @@ $ otool -L PlayerEngineKit.framework/PlayerEngineKit | grep -i libavcodec And the engine says it itself: the `[FFmpeg] libavcodec …` line at `init` reports the versions that actually answered, not the ones it was built against, so a wrong binding shows up in the log before it shows up as a defect. +### Running several engines at once + +Each `AetherEngine` has its own loopback server, caches and lifecycle, so two or more can play side by side (a multiview grid). What they share is the process: the audio session, its channel preference and the panel's display criteria. The engine keeps one book of who is still playing (`SharedOutputCoordinator`, internal) and acts on it: + +- The audio session is released only by the last engine to stop, and only if some engine in that round had `deactivatesAudioSessionOnStop`. +- The preferred output channel count is the widest source among the playing engines, so a muted stereo tile cannot downmix a 5.1 one. +- A display-criteria reset waits until the last engine stops, the ones a load runs to clear stale criteria included; an engine that loads again as `.primary` in the meantime cancels its own pending reset, one that loads again as `.secondary` keeps it. +- The disk allowance for seek and timeshift history (a quarter of the free space, at most 2 GiB per session) is taken from what the other running engines have not already claimed, so side-by-side engines cannot claim the same space twice. A claim ends with the session. +- Stop every engine before you let go of it. An engine counts as playing from `load()` until `stop()`, whatever its state, `.error` and `.ended` included, so an engine kept for reuse must be stopped when its tile ends or fails, or the last real stop will not release the session. An engine released without `stop()` is noticed and dropped the next time another engine asks, but until then it counts as playing. + +Load one engine as `.primary` and the others with `LoadOptions(sharedOutputRole: .secondary)`, set `logTag` (set before `load()`) on each, and control what is audible with `volume`. Every engine keeps decoding its audio, so switching the audible one is a volume change and not a reload. + ## Constructing and binding @@ -478,19 +513,21 @@ try await player.reloadAtCurrentPosition() | --- | --- | | `load(url:startPosition:options:audioSourceStreamIndex:discTitleID:)` | `async throws -> SourceProbe?`. Discardable. Tears down any running session first. | | `load(source:startPosition:options:audioSourceStreamIndex:discTitleID:)` | Same, for `MediaSource.url` or `.custom(IOReader, formatHint:)`. A custom source whose initial probe fails throws, since it cannot be reopened by URL. | -| `reloadAtCurrentPosition()` | `async throws`. Background reopen at the current position, preserving options. Session-preserving: it finishes an installed audio tap and keeps the native host where it can. It also preserves the session's TRANSPORT rather than replaying `autoplay`, so a session that was playing comes back playing and one that was paused comes back paused, whatever the mount was given (AE#464 round 2). The one exception is the resume after a background teardown, which has no transport left to read and is the host's call, so there the mount flag still decides. | +| `reloadAtCurrentPosition()` | `async throws`. Background reopen at the current position, preserving options. Session-preserving: it finishes an installed audio tap and keeps the native host where it can. It also preserves the session's TRANSPORT rather than replaying `autoplay`, so a session that was playing comes back playing and one that was paused comes back paused, whatever the mount was given (AE#464 round 2). A `play()` or `pause()` that arrives while it runs is the newer word and decides how it comes back. The one exception is the resume after a background teardown, which has no transport left to read and is the host's call, so there the mount flag still decides. | | `prepareForItemReplacement()` | One-shot. Keeps the current native `AVPlayerItem` attached until the next `load()` replaces it atomically, for a host that mounts the engine's own player layer and would otherwise show a black layer across the nil-item gap of a foreground episode or playlist change. Consumed by the next `load()`, cancelled by `stop()`, no effect when the outgoing session is not native. PiP hosts do not need it: an active PiP window already forces the handover (AE#158). | | `stop(resetDisplayCriteria:finalTeardown:)` | Ends the session, `state` becomes `.idle`, `startupProgress` becomes nil. `resetDisplayCriteria: false` keeps the panel in its current mode across an item handoff. | | `AetherEngine.probe(url:options:)` / `probe(source:options:)` | `nonisolated static throws -> SourceProbe`. Container/stream metadata read, no detail-pass decoder or session. `options` is read for `httpHeaders` only. Optional trailing `limits` and `cancellation` control the whole operation (below). For a custom reader the caller keeps ownership, `close()` is not called, and the cursor is left unspecified. | | `AetherEngine.probeDetectingAtmos(url:options:atmosDetection:)` | `probe` plus a bounded decode pass that authoritatively resolves E-AC-3 JOC for an Atmos badge. Strictly more expensive; never on the playback-start path. Decode-side failures degrade to "not confirmed" rather than throwing. Same thing as `probe(url:detecting: .atmos)`. | -| `AetherEngine.probe(url:options:detecting:atmosDetection:hdr10PlusDetection:)` / `probe(source:...)` | `probe` plus the opt-in passes named in `ProbeDetail`, over one demuxer: `.atmos` (the bounded JOC decode above) and `.hdr10Plus` (structurally validated ST 2094-40 carriage). Both passes share the optional trailing `limits` and `cancellation`. Empty set is the header probe with the same controls. Both passes only ever SET `isAtmos` / `carriesHDR10PlusMetadata`; ordinary pass failures and per-pass caps leave the detail unconfirmed. A whole-probe stop throws, with no partial result. | -| `ProbeDetail` | `OptionSet`: `.atmos`, `.hdr10Plus`. | -| `HDR10PlusDetectionOptions` | Bounds for the HDR10+ scan: `maxPackets` (32), `maxBytes` (16 MiB), `timeBudget` (2 s). The byte cap is the one that binds on UHD remuxes, where a single keyframe runs to several MB. | +| `AetherEngine.probe(url:options:detecting:atmosDetection:hdr10PlusDetection:)` / `probe(source:...)` | `probe` plus the opt-in passes named in `ProbeDetail`, over one demuxer: `.atmos` (the bounded JOC decode above), `.hdr10Plus` (structurally validated ST 2094-40 carriage) and `.hdrVivid` (structurally validated CUVA HDR Vivid carriage in HEVC, scanned in the same packet pass as `.hdr10Plus`). Both passes share the optional trailing `limits` and `cancellation`. Empty set is the header probe with the same controls. The passes only ever SET `isAtmos` / `carriesHDR10PlusMetadata` / `carriesHDRVividMetadata`; ordinary pass failures and per-pass caps leave the detail unconfirmed. A whole-probe stop throws, with no partial result. | +| `ProbeDetail` | `OptionSet`: `.atmos`, `.hdr10Plus`, `.hdrVivid`. | +| `HDR10PlusDetectionOptions` | Bounds for the HDR10+ / HDR Vivid packet scan (one pass for both): `maxPackets` (32), `maxBytes` (16 MiB), `timeBudget` (2 s). The byte cap is the one that binds on UHD remuxes, where a single keyframe runs to several MB. It also bounds the total source read to four times `maxBytes` (at least 4 MiB), below the demuxer, so blocks of dropped streams count. | | `ProbeLimits` | Optional whole-probe controls: `maxInputBytes` (8 MiB), `maxPackets` (128), `maxPacketBytes` (2 MiB), `timeBudget` (5 s). Nonnegative values required; the time budget must be finite. | | `ProbeCancellation` | Thread-safe, one-shot token: `init()`, `isCancelled`, `cancel()`. Available on every URL/custom header, detail and `probeDetectingAtmos` overload. Cancellation is a request, not a completion notification. | | `ProbeError` | `invalidLimits`, `inputLimit`, `packetLimit`, `packetSizeLimit`, `timedOut`, `invalidReaderResult`, `unsupportedURL`, `sourceBusy`; `errorDescription` describes the stop. Explicit caller cancellation throws `CancellationError` instead. | | `AetherEngine.externalSubtitleTrackIDBase` | `100_000`. Synthetic ids of external subtitle tracks start here. | +Cancelling the Task that awaits `load()` ends the load at once: it throws `CancellationError` instead of running its connect, open, retry and probe budgets to the end (an unreachable live origin used to hold it for about 19 s). A blocked network read, `avformat_open_input` / `find_stream_info`, a size probe, an `HLSLiveIngestReader` retry and a custom reader's blocking `read` are all aborted, typically within tens of milliseconds; the one stretch it cannot cut short is the AE#532 Dolby Vision record audit, a detached read the load waits out before it throws. The load is left the way a newer `load()` leaves it: the generation moves on, a custom reader is closed, `state` becomes `.idle`, and the native host and AVPlayer (#15), an AE#158 handover item and the panel mode are kept for the next load. Until that next `load()` or a `stop()`, the engine reads `.idle` but still holds the native host and its AVPlayer with the item unloaded, Now Playing may stay registered, the audio session stays active, and with an AE#158 handover the previous item can keep playing in PiP; a host that LEAVES on cancel (a dismissed view, say) must call `stop()`. A cancellation only ends the load it belongs to: one that arrives after a newer `load()` or `stop()` took over, or after the load returned, changes nothing. A load that follows an engine rebuild (AE#629) ends that rebuild when cancelled. A `load()` called from a Task that is already cancelled throws before it touches the running session. A real failure that coincides with the cancellation is reported as `CancellationError` too, and `state` ends `.idle`, though a transient `.error` may have been published on the way. The same holds for a `reloadAtCurrentPosition()` of a URL source, which goes through `load()`. A load that returned before the cancellation is not undone; on the `nativeRemoteHLS` bypass `load()` returns at the mount, so a host leaving that startup calls `stop()`. Not cancelling changes nothing. + ### Whole-probe limits and cancellation Existing calls retain their open policy: `limits: nil, cancellation: nil`. Passing `limits: .init()` @@ -571,6 +608,8 @@ What it fetches: one ranged GET from byte zero for the budget, plus a second one A warm also hands the load **where the bytes live**. A resolver URL that answers 302 with a temporary edge target is resolved once, by the warm, and the session starts at that target instead of resolving the chain again; a lease that has since run out falls back to the source URL through the same ladder a mid-session expiry uses. Credential headers (`Authorization`, `Cookie`, `X-Emby-Token` and the rest of the #126 set) never travel to a cross-origin target, whether it was reached through a redirect or pinned from a warm. +Remote disc images (`.iso` / `.img` / `.udf` over HTTP) adopt a warm too (#647). The disc reader takes the size from the warm instead of probing for it, answers the disc layer's structure reads and the title's opening extents out of the head, and starts its first request at the warm frontier; its forks (the subtitle side reader, the forward prefetcher) share the same bytes without a copy. It does not pin the warm's redirect target, because that reader follows redirects per request. A disc-image URL that turns out not to be a disc hands the warm on to the streaming reader. + Three limits are part of the contract rather than implementation detail: - **It never queues for the origin.** A warm takes a request slot only if one is free right now, and declines when the origin is metered down to one request at a time or is pacing the engine (`maxConcurrentSourceRequests`, #377). A prewarm that would have to wait for the playing session's uplink has stopped helping. @@ -592,7 +631,7 @@ Warms are serialised, one at a time across the process. A second `prewarm` while | `seek(to:)` | `async`. Source-axis seconds. Rejected when idle, errored, ended, or live without a DVR window. | | `seek(toSourceTime:)` | Deprecated alias for `seek(to:)`. The clock is unified onto source PTS, so the two are the same call. | | `setRate(_:)` | Clamped to `maxSupportedRate`; `0` pauses. The speed holds across pause and resume and across the host rebuilds a session makes on its own (reload at position, audio-track switch, AirPlay LAN swap, background return), and it belongs to the item: loading a different source, or `stop()`, returns to 1.0. Pitch-preserving on both decode routes: `audioTimePitchAlgorithm` is pinned to TimeDomain on the native player item and on the software path's audio renderer, so a speed control never has to be gated on which route a title took. | -| `audioDelaySeconds`, `setAudioDelay(_:)` | Lip-sync correction for the viewer's chain, in seconds; positive presents audio later than video. Clamped to +/-2 s and, like `setRate`, it holds across the rebuilds a session makes on its own and belongs to the session. Applied where the engine still holds the timestamps: the stamped sample PTS on `.software`, the audio track of the fMP4 segments on `.loopback`. AVFoundation exposes no equivalent on either surface a host can reach, so on `.remoteBypass` and on audio-only sessions the value is kept for the next load and the no-op is logged rather than faked. Not free the way `setRate` is: the audio already decoded or already fetched is committed to the previous value and cannot be re-timed in place, so a change re-anchors at the playhead. On `.software` that is a seek to the current position; on `.loopback` it is the session-preserving reload, because seeking to the position AVPlayer already holds is a buffer hit and plays the old-offset segments out anyway. Measured on a 30 fps H.264 fixture: about 0.3 s of held picture on the loopback route, position preserved. A live session without a DVR window has no position to return to and takes the new value at the next seam it makes on its own. Presses arriving while a re-anchor is running are folded into it rather than stacking one re-anchor per press (AE#464 round 3), so a stepper held down converges on one rebuild instead of queueing a rebuild per press; the value does not ride the call, so the rebuild in flight delivers the newest one. A call landing while the session is being rebuilt at all is neither applied nor refused: it is kept in the options the load in flight reads, and says so. | +| `audioDelaySeconds`, `setAudioDelay(_:)` | Lip-sync correction for the viewer's chain, in seconds; positive presents audio later than video. Clamped to +/-2 s and, like `setRate`, it holds across the rebuilds a session makes on its own and belongs to the session. Applied where the engine still holds the timestamps: the stamped sample PTS on `.software`, the audio track of the fMP4 segments on `.loopback`. AVFoundation exposes no equivalent on either surface a host can reach, so on `.remoteBypass` and on audio-only sessions the value is kept for the next load and the no-op is logged rather than faked. Not free the way `setRate` is: the audio already decoded or already fetched is committed to the previous value and cannot be re-timed in place, so a change re-anchors at the playhead. On `.software` that is a seek to the current position; on `.loopback` it is the session-preserving reload, because seeking to the position AVPlayer already holds is a buffer hit and plays the old-offset segments out anyway. Measured on a 30 fps H.264 fixture: about 0.3 s of held picture on the loopback route, position preserved. A live session without a DVR window has no position to return to and takes the new value at the next seam it makes on its own. Presses arriving while a re-anchor is running are folded into it rather than stacking one re-anchor per press (AE#464 round 3), and a press that lands after the in-flight re-anchor read the value gets exactly one catch-up pass; the value does not ride the call, so the rebuild in flight delivers the newest one. Presses at a human cadence arrive after a re-anchor has finished and are each their own. A re-anchor or a `reloadAtCurrentPosition(applying:)` raised the moment a previous rebuild returned, before the new host has published a position, rebuilds at the position that rebuild was mounted at, not at the head (round 5). A call landing while the session is being rebuilt at all is neither applied nor refused: it is kept in the options the load in flight reads, and says so. | | `maxSupportedRate` | 2.0 for video, 3.0 audio-only. Query after load; returns 2.0 while idle. Size a speed picker against it. | | `volume` | 0.0 to 1.0. A write before a session exists is remembered and applied at load. | | `selectTitle(id:)`, `selectChapter(id:)` | Disc titles and chapters. | @@ -604,14 +643,35 @@ Time lives on `player.clock`, a separate `ObservableObject`, so ~10 Hz ticks nev | Symbol | Axis | | --- | --- | | `clock.$currentTime` | playback clock, the scrubber axis | -| `clock.$sourceTime` | source PTS of the displayed frame; render subtitle overlays against this | +| `clock.$sourceTime` | source PTS of the displayed frame; render subtitle overlays against this. On `nativeRemoteHLS` it is item time, corrected by the lead over the picture the engine measures on its injected #316 renditions while one is selected (AE#616); see below the table. | +| `clock.$sourceTimeFollowsPicture` | whether `sourceTime` is known to follow the displayed frame. Always true except on `nativeRemoteHLS`, where it is true only while the lead is measured since the last time jump (AE#616); see below the table. | | `clock.$progress` | `currentTime / duration` | | `clock.$bufferedPosition` | source-axis position buffered ahead | -| `clock.$liveEdgeTime`, `clock.$seekableLiveRange`, `clock.$behindLiveSeconds`, `clock.$isAtLiveEdge` | live-window surfaces. `seekableLiveRange` is the intersection of the DVR window (policy) and what the segment cache actually holds and can play forward from (fact), so it is honest to scale a rewind strip on and `seek(to:)` clamps to the same floor (AE#441). The two diverge for the whole first `dvrWindowSeconds` of a session and again whenever retention evicts faster than the window slides. Software live sessions have no such cache and keep the arithmetic bound. | +| `clock.$liveEdgeTime`, `clock.$seekableLiveRange`, `clock.$behindLiveSeconds`, `clock.$isAtLiveEdge` | live-window surfaces. `seekableLiveRange` is the intersection of the DVR window (policy) and what the segment cache actually holds and can play forward from (fact), so it is honest to scale a rewind strip on and `seek(to:)` clamps to the same floor (AE#441). The two diverge for the whole first `dvrWindowSeconds` of a session and again whenever retention evicts faster than the window slides. Software live sessions hold a packet ring instead, which evicts by the same byte budget as the native cache (a quarter of the volume's free space, at most 2 GiB, at least 64 MiB), so the range's floor follows the ring once eviction has moved it past the window. | | `$residentRanges` | where the loopback segment cache holds picture right now, as disjoint ascending spans on the `currentTime` axis. This is the cache's own truth, not AVPlayer's `loadedTimeRanges`: a measured session held 64 segments over four minutes across several islands while AVPlayer exposed roughly twelve seconds around the playhead and forgot a seeked-ahead island as soon as the playhead left it. Empty is the nil-equivalent, and a live session publishes empty always (its rewind depth is `seekableLiveRange`, which answers a different question). Residency is not a promise that a seek inside a span is instant: the player may still re-anchor and decode at the target, and a segment can start mid-GOP (AE#412). Coalesced to at most four updates a second, cleared on `load()` and teardown. | -| `player.currentTime`, `sourceTime`, `progress`, `bufferedPosition`, `liveEdgeTime`, `seekableLiveRange`, `behindLiveSeconds`, `isAtLiveEdge` | non-published mirrors of the same values for one-shot reads | +| `player.currentTime`, `sourceTime`, `sourceTimeFollowsPicture`, `progress`, `bufferedPosition`, `liveEdgeTime`, `seekableLiveRange`, `behindLiveSeconds`, `isAtLiveEdge` | non-published mirrors of the same values for one-shot reads | | `$duration` | seconds; a `LoadOptions.declaredDurationSeconds` outranks the container's | +**`sourceTime` on `nativeRemoteHLS` (AE#616).** The engine sees no segment on that route, so the clock is +AVPlayer's item time. An origin whose playlist places a segment at its slot while the segment starts at +the keyframe before it (a Jellyfin transcode restarted by `-ss -noaccurate_seek -copyts`) makes +item time lead the frame on screen, by a different amount after every seek: 1.1 s to 8.3 s measured, and +AVPlayer keeps the anchor of the first segment it loaded after the seek. The WebVTT renditions the +engine injects for `LoadOptions.externalSubtitles` are placed by media timestamp, so each line reaches +AVPlayer's legible output at its cue start plus that lead. The engine watches its own renditions with a +non-suppressing legible output, matches each presented line back to the cue it wrote, and publishes +`sourceTime` as item time less the measured lead (log line `AE#616: item time leads the presented +frame by ...`). It re-measures on every line; between a seek and the next line it keeps the previous +value, and `clock.sourceTimeFollowsPicture` is false from the time jump until that line, so a host can +hold its overlay instead of detecting seeks itself. Without an injected rendition selected nothing is +measured, `sourceTime` is item time and `sourceTimeFollowsPicture` stays false. +`currentTime`, `seek(to:)` and the scrubber stay on item time throughout, so a seek round-trips. A host +drawing its own overlay from `sourceTime` (libass) can keep the rendition selected behind its own +suppressing `AVPlayerItemLegibleOutput` to keep the measurement running. An external `.ass` / `.ssa` sidecar declared with +`preserveASSMarkup` gets this from the engine: while the host overlay draws its raw events, the plain +rendition stays selected behind an engine-owned suppressing output, and PiP, AirPlay or an external +display drops the suppression and shows it. + ## What the session is doing | Symbol | Reading | @@ -624,14 +684,18 @@ Time lives on `player.clock`, a separate `ObservableObject`, so ~10 Hz ticks nev | `$isSessionReady` | The session is ready in the AVFoundation sense. Not the edge a black cover comes off on. | | `$hasFirstFrameReadyForDisplay` | The picture for **this** load is up. A picture, not motion: a live join presents its first frame and can then hold it bit-static for seconds while AVPlayer decides whether to start (AE#440), so a host dropping a spinner here drops it onto a frozen frame. Use `$playbackPhase` for "it is moving". Latched for the load, cleared at the next `load()` / `stop()`. Audio-only sessions never arm it. On an external screen (`isExternalPlaybackActive`) the local layer never reaches readiness, so the item's readiness is the honest edge and the flag latches there (#315). | | `$startupProgress` | `StartupProgress?` for a determinate loading bar. | +| `softwarePathEscalations`, `SoftwarePathEscalationEvent` | A native session AVPlayer refused, rebuilt on the software path, with the failure it absorbed. See [The engine moves a refused session onto the software path](#the-engine-moves-a-refused-session-onto-the-software-path). | | `$videoRoute` | `VideoRoute`: which pipeline is actually serving, one of `.none`, `.remoteBypass`, `.loopback`, `.software`, `.audio`. `LoadOptions.nativeRemoteHLS` is only the request; the carriage watchdog, the remembered verdict and the HLS reroutes move a session between routes, mid-session too. Branch on this, above all for who draws subtitles. | -| `$audioDelivery` | `AudioDelivery`: how the audio reaches the renderer, one of `.none`, `.noAudioInSource`, `.streamCopy`, `.bridged`, `.decoded`, `.droppedNoPipeline`, `.playerManaged`. `.droppedNoPipeline` is a source that HAS audio playing video-only because no pipeline could be built for it: the value a fallback ladder demotes on. See [Reading whether the audio was delivered](#reading-whether-the-audio-was-delivered). | +| `$airPlayPictureStaysLocal` | iOS: true while a wireless AirPlay receiver holds the audio route but the session runs on the software host (VP9, AV1 without hardware decode, deinterlaced MPEG-2 / VC-1 / H.264, `preferredDecodePath = .software`). That host draws into an `AVSampleBufferDisplayLayer`, and only its audio renderer follows the route, so the receiver plays sound only. Nothing fails, which is why the engine says so: show the viewer that this title's picture stays on the device. False on the native paths, where the receiver gets the stream, and everywhere off iOS. | +| `$audioDelivery` | `AudioDelivery`: how the audio reaches the renderer, one of `.none`, `.noAudioInSource`, `.streamCopy`, `.bridged`, `.decoded`, `.droppedNoPipeline`, `.playerManaged`. `.droppedNoPipeline` is a source that HAS audio playing video-only because no pipeline could be built for it, or because a live bridge decoded nothing (AE#641): the value a fallback ladder demotes on. See [Reading whether the audio was delivered](#reading-whether-the-audio-was-delivered). | | `$videoFormat` | The format being presented: `.sdr`, `.hdr10`, `.hdr10Plus`, `.dolbyVision`, `.hlg`. On a platform with no per-mode capability table (macOS), a Dolby Vision session the clamp sent to `.hdr10` is upgraded back to `.dolbyVision` once the item AVFoundation is playing turns out to carry a `dvh1` / `dvhe` sample entry (AE#515). On tvOS the panel term behind it no longer comes from the headroom alone: a session serving an HDR master that AVFoundation has not refused publishes the presented format half a second in, because a display that takes an HDR master is presenting HDR while one that is not refuses in 54 to 61 ms, and `currentEDRHeadroom` has been measured reading 1.00 through exactly that acceptance (AE#459). | | `$sourceVideoFormat` | The format the **source** carries, before any panel-driven mapping. The pair is what an honest badge needs: HDR content on an SDR panel differs between the two. | | `$sourceDVProfile`, `$sourceVideoFrameRate`, `$sourceVideoBitrate` | Source detail for an info panel. | | `$dolbyVisionConversion` | A `DolbyVisionConversion?`: the Dolby Vision profile rewrite applied to the served stream, nil when the session serves the source's own profile or no Dolby Vision. `.profile7ToProfile81` is a Profile 7 source played on a display presenting Dolby Vision, where `$videoFormat` reads `.dolbyVision` and `$sourceDVProfile` keeps 7, so a label can say "P7 -> P8.1". The enhancement layer is dropped by that conversion. Set when the loopback session starts, cleared on stop. | | `$sourceVideoCodecName` | The source video codec in the libavcodec spelling ("hevc", "h264", "av1"), nil when the source carries no video. The probe-free remote-HLS bypass maps it back from the item's video sample type, so the field answers on every route rather than going quiet on one of them. Not the same question as `$activeVideoDecoder`: a codec has several decoders, and which one runs depends on the hardware. | | `$sourceContainerFormat` | The container libavformat opened ("matroska,webm", "mpegts"), nil on the remote-HLS bypass, where AVFoundation opens the source and there is no libav context to ask. This is the container that ARRIVED, which on a remux or transcode session is not the one a host's library metadata describes. | +| `$sourceVideoStreamFormat` | A `VideoStreamFormat?` (AE#658): the source video stream's `pixelFormat` ("yuv420p10le"), `bitDepth`, `colorPrimaries`, `transfer`, `matrix`, `range` and `profile` ("Main 10"), in libav's names as the container and the probe's decoder declared them. A field the stream leaves unspecified is nil, never a guessed BT.709. `colorPrimariesLabel`, `transferLabel`, `matrixLabel` and `rangeLabel` give the names a viewer reads ("BT.2020", "PQ (SMPTE ST 2084)", "BT.2020 NCL", "Limited"). nil before load and without video. On the probe-free remote-HLS bypass it is the DELIVERED stream, read back from AVPlayer's item video track once it resolves: colour and range from the format description, `profile`, `bitDepth` and `pixelFormat` from its avcC / hvcC record (nil where the codec has none of those, AV1 and VP9 included). Under a server-side transcode that is the transcode, not the library's file, which is the point of reading it there. | +| `$decodedVideoFormat` | A `DecodedVideoFormat?` (AE#658): what the engine's own decoder actually produced. `frame` is a `VideoStreamFormat` for the decoded picture (libavcodec's output pixel format, or the libav name of the VideoToolbox buffer on the software host's hardware decoder), `pixelBufferFormat` the CoreVideo buffer it was displayed from (`420v`, `x420`; `pixelBufferLabel` reads "P010 (x420)"). Republished when either changes mid-stream. **nil on every native route**: AVPlayer decodes there and its frames never pass through the engine, so a panel shows `$sourceVideoStreamFormat` and names AVPlayer as the decoder instead of inferring a pixel format it never saw. | | `$activeVideoDecoder`, `$activeAudioDecoder` | The decoder names actually in use, for a stats overlay. This is the honest "what is decoding this" surface; `playbackBackend` is not. Show `$activeAudioDecoder`, classify on `$audioDelivery`: a label cannot separate a source without audio from a source whose audio was dropped. | | `$metadata` | `MediaMetadata` parsed at load (title / artist / album / cover). | | `$mediaChapters`, `$discChapters`, `$discTitles`, `$selectedDiscTitle` | Container chapters, and disc titles / chapters for DVD and Blu-ray ISO sources. | @@ -658,9 +722,9 @@ Time lives on `player.clock`, a separate `ObservableObject`, so ~10 Hz ticks nev | Symbol | Notes | | --- | --- | -| `$audioTracks` | `[TrackInfo]`. Republished when `LoadOptions.confirmAtmos` confirms a track. | -| `$activeAudioTrackIndex` | The selected track's id. | -| `selectAudioTrack(index:)` | Session-preserving reload, roughly 0.5 to 1 s of black. `index` is `TrackInfo.id`. A no-op when out of range, already active, or on a forward-only custom source (live ingest included), which cannot rebuild its pipeline: there a track change is a fresh `load` naming the stream. Every refusal is logged, so a picker that does nothing is explainable. | +| `$audioTracks` | `[TrackInfo]`. Republished when `LoadOptions.confirmAtmos` confirms a track. On `.remoteBypass` it lists the audio tracks AVPlayer built for the item (normally just the one playing), read from their format descriptions: `codec` in libavcodec's spelling (`eac3`, `ac3`, `aac`), `channels`, `sampleRate`, `profile` (AAC's object type; "Dolby Digital Plus + Dolby Atmos" with `isAtmos` when the E-AC-3 dec3 record announces JOC) and `language` from the track or the selected audible option. The ids are synthetic (from 400000 up) because there is no stream index on that route; compare them only with `activeAudioTrackIndex`. | +| `$activeAudioTrackIndex` | The selected track's id. On `.remoteBypass`, the enabled item track's. | +| `selectAudioTrack(index:)` | Session-preserving reload. On the native path the old item stays mounted until its replacement is ready (not gapless; a media-services reset still discards it); elsewhere expect roughly 0.5 to 1 s of black. Rapid picks coalesce into one rebuild at a time, a pick back to the active track costs nothing, and `stop()` or a new `load` drops what is queued. It comes back in the session's transport like `reloadAtCurrentPosition()`: a paused session stays paused, and a `play()` or `pause()` that arrives while the rebuild awaits I/O decides the final state. `index` is `TrackInfo.id`. A no-op when out of range, already active, or on a forward-only custom source (live ingest included), which cannot rebuild its pipeline: there a track change is a fresh `load` naming the stream. Every refusal is logged, so a picker that does nothing is explainable. Also a no-op on `.remoteBypass`, where AVPlayer owns the audio selection and the list above is informational; a different language there is a different URL. | | `installAudioTap()`, `removeAudioTap()`, `audioTapHasDeliverySource`, `AetherEngine.audioTapFormat` | Opt-in decoded PCM, mono Float32 48 kHz with source-PTS stamps, off the render path. See the contract above. | ## Subtitles @@ -681,6 +745,7 @@ Time lives on `player.clock`, a separate `ObservableObject`, so ~10 Hz ticks nev | `$nativeSubtitleRenditionAvailable` | At least one cue exists for the native track. Gate the AVMediaSelection picker on it. | | `$nativeSubtitleRenditionsServed` | Whether the served playlist is the master. A reload signal and a diagnostic, nothing more: whether a legible rendition reaches a wired external display is AVKit's business and is not observable from here. | | `setNativeSubtitleRendering(_:)` | Hand subtitle drawing to AVKit while the video leaves the host's view hierarchy (PiP, AirPlay, wired external display) and take it back on return. No-op when the active subtitle has no native text equivalent (bitmap, or a track added after load). | +| `setSoftwareSubtitleDelay(_:)`, `softwareSubtitleDelaySeconds` | Media-time offset for both channels composited into software PiP frames: positive delays cues, negative advances them. Non-finite values are ignored; the default is 0 and the value survives later loads on the same engine. Takes effect on the next composited frame, without a seek, reopen or A/V clock change. A host overlay applies the same value as `clock.sourceTime - delay`. AVPlayer-owned native renditions, native PiP and AirPlay are not shifted; a host offering a timing control should say so. | | `teletextPage`, `setTeletextPage(_:)` | The DVB teletext caption page, at load and while the channel plays. | When an upstream server reanchors the played media, set `ExternalSubtitleTrack.nativeTimelineOffsetSeconds` to the source-time offset for that item. Native HLS/PiP/AirPlay subtitle renditions subtract it from external cue timestamps; host overlay cues retain their original timestamps. Pass the offset when registering each track, including tracks registered after load. A non-finite offset is treated as zero. Whole-program native renditions return HTTP 503 while extraction remains unfinished, so a temporary timeout cannot make AVPlayer cache an incomplete track for the session. @@ -691,12 +756,13 @@ When an upstream server reanchors the played media, set `ExternalSubtitleTrack.n | Symbol | Notes | | --- | --- | | `$isLive` | Mirrors `LoadOptions.isLive` for the session. | +| `$isSequentialOrigin` | True while a VOD session is served as a sequential origin, declared (`LoadOptions.sequentialOrigin`) or found by the reader (the origin ignores `Range`, or refused the ranged open twice and served a plain GET, AE#693). The session plays from the start, so a `startPosition` given to `load()` was not honoured, and a seek lands only inside what `$residentRanges` already holds. A host that needs the position reopens the source or tells the user. Set during `load()`, cleared on stop. | | `seekToLiveEdge()` | `async`. | | `liveSourceReset` | The retune contract above. | | `liveResumeClamped`, `LiveResumeClamp` | A resume that found the playhead outside the window and moved it; see above. | | `liveScrubThumbnail(atSessionSeconds:maxWidth:)` | Still on the live session axis, decoded from what the session already holds. A native session reads its DVR segment cache; a software session reads its DVR packet ring (#544), so a tuner channel the box decodes in software has a scrub preview too. | | `$playlistShiftSeconds` | Seconds the producer subtracted from source PTS. Published values already fold it back; exposed for hosts pairing their own samples against AVPlayer's raw clock. | -| `HLSLiveIngestReader(playlistURL:)`, `HLSLiveIngestReader(playlistURL:httpHeaders:)` | The ready-made `IOReader` for ingesting an upstream HLS playlist directly, with AES-128 clear-key and SSAI handling. The headers ride the playlist, every segment and every AES key, which is what a tokenized IPTV origin enforces per request. Unsupported shapes surface a typed `HLSIngestError`. | +| `HLSLiveIngestReader(playlistURL:)`, `HLSLiveIngestReader(playlistURL:httpHeaders:)` | The ready-made `IOReader` for ingesting an upstream HLS playlist directly, with AES-128 clear-key and SSAI handling. The headers ride the playlist, every segment and every AES key, which is what a tokenized IPTV origin enforces per request. Credential headers (`Authorization`, `Cookie`, `X-Emby-Token` and the like) go only to the playlist's own origin with no https to http downgrade; a URI the playlist points at another host gets the other headers without them, the rule a redirect already follows. Unsupported shapes surface a typed `HLSIngestError`. | ### Where a live start's seconds go @@ -718,7 +784,96 @@ measured against, whether it waited or was satisfied immediately. `LoadOptions.liveJoinProfile` is the lever. `.fastZap` collapses `TARGETDURATION` to the source keyframe cadence and the holdback follows it down, so the win belongs to the source GOP rather than to the flag: `TARGETDURATION` can never fall below `ceil(max EXTINF)`, and a long-GOP source therefore keeps most of -its runway under either profile. +its runway under either profile. Where the engine cuts the segments itself, each one is a whole GOP and +the value is sealed from the first few, so it carries headroom for a GOP 1.5 x the longest seen: a +broadcast whose GOPs run 1.0 to 2.4 s sealed TARGETDURATION 1 on its first three and then broke +`EXTINF <= TD` on every longer one (AE#670). The headroom is the smallest value under which that longer +GOP still lists (RFC 8216 rounds EXTINF to the nearest integer) and finishes inside AVPlayer's +`1.5 x TARGETDURATION` patience with 0.5 s to spare, so 1 s GOPs serve TARGETDURATION 2 and a 6 s +holdback, and so does a 59.94 fps stream whose scene-cut keyframes merge GOPs into 80 to 89-frame +segments (up to 1.666 s seal 2). + +**An ingested HLS upstream seals over the segment it is delivered in, as far as its join pays for it, +under either profile (AE#684).** The engine re-cuts an upstream segment (at its GOPs under `.fastZap`, +at about 4 s under `.standard`), but the window still changes once per UPSTREAM segment, so what the +seal asks for is the longest segment the upstream has served, whole, where `ceil(segment / 1.5)` used +to leave the client a patience of exactly one delivery. The upstream's advertised target duration is +still not read. What it gets is bounded three ways: + +- **By the join, which is not deepened for it on an upstream of uniform segment length.** A seal is a + holdback of `3 x` itself, and the join there is the three upstream segments it was before (one more + would be one more download before the first picture on every zap, which AE#678 measured and + declined). Their last GOP stays open until the next delivery, so three 6 s segments are 18 s joined + and 16 s cut: the seal is 5, the largest value whose holdback that window holds, not 6. Three 10 s + segments cut 28 s and seal 9. The one addition is that the join is equalised over the PHASES of an + upstream whose segment lengths alternate: a tune never loads more than the deepest tune of the same + channel already did. Alternating 6 s and 4 s segments used to join four (20 s listed, 18 s cut, + seal 6) with a 4 s one newest and three (16 s listed, 14 s cut, seal 4, as in 7.25.1) with a 6 s + one newest, so the same channel behaved differently from one tune to the next. The depth is now the + deepest the coverage rule takes over the tune itself and the tunes one, two, ... segments earlier + that its own join still reaches, at most one segment above its own: both phases join four and seal + 6. That costs the three-segment phase (about four tunes in ten on that shape) one additional short + segment at the join, which the other six already load: 3.47 s to 4.23 s to first picture behind an + 8 Mbit/s link, median of five, against 4.27 s in the other phase on either build. Shapes whose + phases already joined alike keep their join and what it pays: 10 s / 8 s stays at three segments + and seals 8 (7 before), 6 s / 6 s / 4 s stays at three and seals 4, unchanged, and a shape like + 6 s / 5 s keeps a seal that follows the phase (5 in one, 4 in the other), because equalising is + about how much a tune loads, not about what it seals. The seal line says what was paid + (`of which the join pays 5s of 6s (16.000s cut of the 18.000s it listed)`), and "the join has no + more to give" is a fact the reader states (its join batch is handed over and its consumer is + waiting), not arithmetic on EXTINF, which a playlist that rounds its durations up would fail. + With a demuxed audio rendition the fact is the cutter's rather than one reader's: both join + batches handed over and the cutter waiting on either of them, because an audio rendition whose + segments end a little before the video's runs dry first and the video reader is then never the + one being waited on (three runs per arm on three listed segments, audio aligned and audio 0.2 s + short: first picture 0.18 to 0.21 s against 0.17 to 0.36 s on 7.25.1, TD 5 on 6 s and 9 on 10 s + under both profiles). A join that is SLOWER than the `.fastZap` grace, a single-connection + provider or a slow link, takes the bounded start before it is spent and then seals the full + value (6 or 10, not 5 or 9) over a window shorter than its holdback, which AVPlayer notes with a + `-16832` warning at the start; measured one request at a time behind a 4 to 5 Mbit/s link that + start is not worse than 7.25.1's (first picture 5.05 to 6.08 s against 7.05 to 8.26 s, no + `-12888` where 7.25.1 drew 1 to 3). +- **By a ceiling.** Segments longer than 10 s (`LiveEdgePolicy.upstreamSegmentSealCeilingSeconds`) are + not asked for whole: a 20 s one seals 14, as it did before, because the live-only window is 60 s. +- **Never under what it was.** Where the old value already covered the segment nothing moves: 2 s + segments keep 2 and a 6 s holdback, and `.standard` keeps its 6 on anything up to 6 s. + +So on a provider of uniform 6 s segments under `.fastZap` the seal goes from 4 to 5 (holdback 12 to +15 s), on a uniform 10 s one under either profile from 7 to 9 (21 to 27 s), and on alternating +6 s / 4 s segments from 4 to 6 (12 to 18 s) in both phases. A provider of 3 s segments with 1 s GOPs is +unchanged at 2: three joined segments cut 8 s, which pays for no more, so the patience of one +delivery this issue is about stays as it is there. On uniform upstreams the join, and with a host that +fetches in parallel the time to the first picture, is what it was: behind an 8 Mbit/s link, three runs +per arm, 3.81 to 4.04 s before and 3.80 to 3.85 s after on 6 s segments, 6.15 to 6.36 s and 6.18 to +6.28 s on 10 s ones. A host that allows fewer parallel requests than the join has segments +(`maxConcurrentSourceRequests` of 3 or less) sees its first picture later than before, because the +gate now waits for the join to be handed over where it used to serve at 12 s of cut content: one +request at a time behind 14 Mbit/s on the mixed shape, three runs per arm, 2.79 to 3.08 s before and +3.27 to 3.28 s after in the four-segment phase, 2.80 to 2.83 s and 3.44 to 3.49 s in the phase that +now also loads the extra segment. The other cost is standing latency: 3 s more behind the upstream's +edge on uniform 6 s segments, 6 s on alternating 6 s / 4 s and on 10 s ones, 3 s on 10 s / 8 s. + +Measured on loopback against `Scripts/hls-burst-origin.py` (deliveries 3 to 9.3 s apart on 6 s +segments), 60 s per run, three runs per arm and row, 7.25.1 against this; first picture is 0.17 to +0.51 s in both arms wherever a row does not say otherwise: + +| upstream, profile, segments listed | 7.25.1 | now | +|---|---|---| +| 6 s, `.fastZap`, 3 | TD 4, 1 x `-12888` per run, 2 stalls in 3 runs | TD 5, none | +| 6 s, `.fastZap`, 4 | TD 4, 2 x `-12888` per run, 2 stalls in 3 runs | TD 5, 1 to 2 x `-12888`, no stall | +| 6 s, `.fastZap`, 8 | TD 4, 1 to 3 x `-12888`, 3 stalls in 3 runs | TD 5, 0 to 2 x `-12888`, no stall | +| 10 s, `.fastZap`, 3 / 4 / 8 | TD 7, 1 x `-12888` in 7 of 9 runs, window closed and item rebuilt in 4 of 9 | TD 9, none in 9 of 9 | +| 10 s, `.standard`, 3 / 8 | TD 7, item rebuilt in 6 of 6 runs | TD 9, none in 6 of 6 | +| 6 s, `.standard`, 3 / 8 | TD 6, first picture 6.18 to 6.53 s | unchanged: TD 6, 6.28 to 6.46 s | + +Two things this does not do, said plainly. On a 6 s provider `-12888` still comes: over one 180 s run +5 of them and a stall before, 3 and no stall after, because 7.5 s of patience still loses to a 9 s gap +and what changed is the holdback under it. (A join one segment deeper seals 6 and drew none in three +such runs; it was measured and not shipped, for the download it costs.) And `.standard` on 6 s segments +still takes about 6.3 s to its first picture, waiting for the next upstream segment, exactly as it did: +its seal there is its own `1.5 x cut target` floor, which three joined segments never covered. A real +outage longer than the holdback's low point stalls under any of these; the capture that started this +had 10 of its 110 delivery gaps above 9 s. **An HLS source with a window of its own now fills that cushion at the join rather than in wall clock** (6.77.0). The ingest used to enter a live playlist three segments behind the edge, and three joined @@ -784,6 +939,17 @@ The depth is the contiguous span ahead of the playhead, so an island past a gap floor is not met the engine says so once per load (`leaving the stall-avoidance wait alone (buffer ahead ...s, ...)`), which is what separates the two mechanisms in a report after the fact. +**And the item has to be able to play (AE#684).** The depth is measured from the item's own +`currentTime()`, and before `readyToPlay` that is where the loader began fetching, not where playback +will begin. Under an `EXT-X-START` placement, which is how a rejoin lands a fresh item at the place the +session held, AVPlayer fetches from about 6 s below the target first, so the reading is the lookback +itself: one device capture shows `buffer ahead 4.00s` from a playhead of 48.00 s on an item placed at +53.908 s, every second of it behind the start, and `playImmediately` 36 ms before the item was ready. It +was the one item of five in that capture to be forced and the one its viewer reported out of sync, which +is a correlation and not yet a mechanism. A hold on an item that cannot play yet is now left alone +(`leaving the stall-avoidance wait alone on an item that cannot play yet ...; asked again at readiness`) +and decided again when the item becomes ready, from the playhead it really starts on. + ### The rewind depth a live session really has `seekableLiveRange` used to be `max(0, edgeTime - dvrWindowSeconds) ... edgeTime`, pure arithmetic that @@ -870,6 +1036,14 @@ declared from the old source produces a file that is unplayable or silently wron host has the event and starts part two if it wants one. `stop()` and a new `load()` end it the same way with `.ended(.sessionEnded)`; a recording never outlives its session. +`.ended` is published once the file is closed. The queued tail and the trailer are written off the +main actor, so it can arrive a moment after the call that ended the recording; `stopRecording()` +returns only after it, and a `startRecording(to:)` issued in that moment waits for it first. If a +`stopRecording()` or a new `load()` overtakes that wait, the start throws `CancellationError` and +records nothing. A stop that lands while the writer is already tearing itself down after a failure +(`.writeTooSlow`, `.diskFull`, `.writeFailed`) waits for that teardown and publishes the failure, not +`.ended`. + **Not implemented: recording from the start of what is already buffered.** A recording begins at the call, not at the back of the DVR window. On `.loopback` what is retained is remuxed fMP4 with **bridged** audio, not source packets, so prepending it would produce one file whose audio codec @@ -885,8 +1059,9 @@ say so on the tracker rather than working around it. | `nativePlayerLayer` | The engine's own `AVPlayerLayer`, for a host building `AVPictureInPictureController` around a layer rather than around `currentAVPlayer`. | | `$softwarePiPSource` | `SoftwarePiPSource` for sample-buffer PiP on the software path: the display layer plus transport answers on the enqueued frames' axis. iOS only in practice; tvOS AVKit does not evaluate sample-buffer content sources (FB9751461). | | `$softwareDisplaySize` | The rectangle the software path's picture presents at: coded size under the decoder's pixel aspect. Mirrored, not latched, so a mid-stream resolution change re-shapes it. nil on every other path. | -| `sourceVideoWidth`, `sourceVideoHeight`, `sourceVideoPixelAspectRatio` | The source's CODED size and the multiplier that turns it into the presented one (`width * ratio`), read once from the probe. The ratio is 1 on square pixels and on a declared ratio the engine refuses (#290), never a guess; on the paths that draw, prefer what is on screen (`softwareDisplaySize`, `AVPlayerLayer.videoRect`) over recomputing it here. | +| `sourceVideoWidth`, `sourceVideoHeight`, `sourceVideoPixelAspectRatio` | The source's CODED size and the multiplier that turns it into the presented one (`width * ratio`), read once from the probe (on `.remoteBypass`, from AVPlayer's item video track once it resolves: the delivered size). The ratio is 1 on square pixels and on a declared ratio the engine refuses (#290), never a guess; on the paths that draw, prefer what is on screen (`softwareDisplaySize`, `AVPlayerLayer.videoRect`) over recomputing it here. | | `pictureInPictureActive` | Host-set. Keeps the pipeline and the loopback server alive across a background transition, and keeps the software path decoding video for the window. | +| `needsForegroundVideoRestore` | True after background policy actually releases the video item. Restore with `reloadAtCurrentPosition(applying:)` before play, choosing the desired transport intent. False when video was retained or after stop/load. | | `backgroundPlaybackEnabled`, `backgroundTeardownGraceSeconds` | Background audio policy; the grace window (15 s default) is what lets a paused session survive a quick app switch. | | `presentationAxisMap` | `PresentationAxisMap`: `sourceSeconds(forItemSeconds:)`, `itemSeconds(forSourceSeconds:)`, `shiftSeconds(atItemSeconds:)`, `seams` (each a `PresentationAxisMap.Seam` of `itemSeconds` and `shiftSeconds`), `isEmpty`. Readable off the main actor. Cue times and `sourceTime` live on the source axis; `AVPlayerItem.currentTime()` lives on the item axis, and they differ by the producer shift. Returns nil rather than a guess where no axis is established. | | `setNativeVideoFrameTimeObserver(_:)` | Takes a `NativeVideoFrameTimeObserver`, called per muxed frame on both axes (`NativeVideoFrameTime`: `source`, `item`, `segmentIndex`, `isKeyframe`, `epoch`). Called on the producer pump thread in **decode** order, so `source` is not monotonic under B-frames. `epoch` rises process-wide. | @@ -901,15 +1076,16 @@ say so on the tracker rather than working around it. | `videoNowPlayingSession`, `setVideoNowPlayingInfo(_:)` | The session and its staged identity dictionary. Elapsed / rate / duration are merged from the player; do not stage them. | | `audioNowPlayingSession`, `setAudioNowPlayingInfo(_:)` | The same pair for the audio-only path, which owns its session unconditionally (there is no AVKit fork there). Pass an empty dictionary to clear. | | `setExternalMetadata(_:)` | AVKit's on-screen info pane on the video path. Safe before `load()`; replayed at host creation. | -| `deactivatesAudioSessionOnStop` | Off by default. The engine declares the audio-session category at init and never activates it on the native path, because AVKit activates per playback and that is what lets tvOS negotiate the HDMI route (#24), so it never deactivates it either. Set true only when the app owns the session outright; the engine then releases it on a genuine final teardown, meaning `stop()` and never a reload, handoff or live retune. | +| `deactivatesAudioSessionOnStop` | Off by default. The engine declares the audio-session category at init and never activates it on the native path, because AVKit activates per playback and that is what lets tvOS negotiate the HDMI route (#24), so it never deactivates it either. Set true only when the app owns the session outright; the engine then releases it on a genuine final teardown, meaning `stop()` and never a reload, handoff or live retune. With several engines in one process the release belongs to the last one out: a stop while another engine is still active keeps the session and logs `[SharedOutput] ... still active, session kept`. | +| `logTag` | A short name for this instance (`tile2`), carried by the `[SharedOutput]` lines and a `[AetherEngine:] state=` line per transition. Set it before `load()` (the tag is captured when the engine joins), on every engine of a host that runs several at once. | ## Stills and thumbnails | Symbol | Notes | | --- | --- | -| `scrubThumbnail(atSeconds:maxWidth:)` | Cache-backed still for the active native session, live or VOD. Decodes bytes already produced, so it opens no second connection and works on single-connection sources (debrid / torrent links) where a second demuxer is refused. | -| `vodScrubThumbnail(atSeconds:maxWidth:)`, `liveScrubThumbnail(atSessionSeconds:maxWidth:)` | The two arms, for callers that know which axis they hold. The live arm also serves software sessions, out of the DVR packet ring rather than a segment cache (#544). | -| `supportsCacheBackedStills` | True while a native session exists, which is what the SEGMENT CACHE needs. Gate the scrub-preview affordance on it: it reports capability, not per-frame availability, so a transient nil from `scrubThumbnail` while a segment is still being produced is expected and means "time only, no image". It stays false on a software session, and a live one nonetheless serves stills from its packet ring, so a live caller asks `liveScrubThumbnail` rather than this flag. | +| `scrubThumbnail(atSeconds:maxWidth:)` | Cache-backed still for the active session, live or VOD. Decodes bytes the session already holds, so it opens no second connection and works on single-connection sources (debrid / torrent links, IPTV accounts capped at one request) where a second demuxer is refused. A native session decodes from its segment cache, a software session from its packet cache (VOD, AE#605) or its DVR packet ring (live, #544). A VOD target outside what is retained returns nil rather than the nearest retained frame. | +| `vodScrubThumbnail(atSeconds:maxWidth:)`, `liveScrubThumbnail(atSessionSeconds:maxWidth:)` | The two arms, for callers that know which axis they hold. Both arms also serve software sessions: the live arm out of the DVR packet ring (#544), the VOD arm out of the packet cache its seeks land in (AE#605), keyframe to target, on the same axis `seek(to:)` takes. | +| `supportsCacheBackedStills` | True while the session can serve `scrubThumbnail` without a second connection: any native session, a software live session, and a software VOD session reading a remote source. Gate the scrub-preview affordance on it: it reports capability, not per-frame availability, so a transient nil from `scrubThumbnail` while a segment is still being produced, or for a position the cache does not hold, is expected and means "time only, no image". False on a software session playing a local file, which keeps no cache because re-reading the file is free; `makeFrameExtractor()` serves that case. Before AE#605 this was false on every software session, live included. | ## Certificate trust @@ -950,11 +1126,12 @@ as well. | Symbol | Notes | | --- | --- | | `AetherEngine.version` | The engine release this source descends from, as the string a published tag carries. SwiftPM resolves a package to a revision rather than to a tag, so an About panel or the header of a diagnostic log has nothing else to name the engine with. Between releases, and under a pin on an unreleased commit, it names the last published version the checkout descends from. | -| `diagnostics.liveTelemetry` | 1 Hz `LiveTelemetry?` snapshot while playing or paused, nil while idle. On a separate `ObservableObject` so its ticks cannot re-render a host observing the engine. | +| `diagnostics.liveTelemetry` | 1 Hz `LiveTelemetry?` snapshot while playing or paused, nil while idle. On a separate `ObservableObject` so its ticks cannot re-render a host observing the engine. On the loopback and software routes `instantBitrateMbps` and `averageBitrateMbps` are the rate of the MEDIA played, not of the transfer (AE#514): the bytes of the played video and audio packets the playhead crossed, over the media seconds it crossed, about the last 10 s for the first and the whole session for the second. Read-ahead, a re-fetch after a seek and a paused live source draining into its DVR window do not move them, and both stand still through a pause; the transfer is `networkThroughputMbps`. On `.remoteBypass` it is fed from AVPlayer's access log alone: `instantBitrateMbps` and `averageBitrateMbps` are what the playing variant declares (BANDWIDTH, and AVERAGE-BANDWIDTH or BANDWIDTH where the master omits it), because the bytes AVPlayer transferred are buffer fill at link speed rather than the stream's rate, `networkThroughputMbps` is the access log's `observedBitrate`, `networkTransferredBytes` and `droppedFrameCount` its session totals, `forwardBufferSeconds` the loaded range ahead of the playhead. `avSyncGapMs` and `observedFps` are nil, and the loopback counters (`producerRestartCount`, `muxedBytesLifetime`, `serverBytesSentLifetime`, `serverRequestCount`, `demuxerBytesFetched`) read 0 because there is no loopback. | | `LiveTelemetry.softwareCacheSeekHits`, `softwareCacheSeekMisses`, `softwareCacheSourceEpoch` | Optional cumulative software-VOD packet-cache counters. A hit repositions the retained consumer cursor without changing the source epoch; a miss repositions the demuxer and advances it. `nil` on other paths. `cachedBytes` includes retained compressed packet records on software VOD, distinct from decoded `displayCushionSeconds` and the underlying byte-reader window. | | `EngineLog.handler` | Mirror every info-level line into a host capture path. Fires from whatever thread emitted it, so it must be thread-safe and non-blocking. | | `EngineLog.subsystem`, `EngineLog.Category` | `de.superuser404.AetherEngine`, one category per subsystem: `engine`, `ffmpeg`, `session`, `muxer`, `demux`, `hls.server`, `audio.bridge`, `sw.playback`, `scrub`. | | `EngineLog.Level` | `.info` reaches os_log and the host handler; `.verbose` is per-segment trace and reaches os_log's debug level **only**, never the handler, which is what keeps a mirrored stream readable. Read the verbose ones with `log stream --level debug`. | +| `EngineLog.registerSecret(_:)`, `EngineLog.unregisterSecret(_:)`, `EngineLog.redacted(_:)` | Name a value that must never be logged, such as an IPTV password the host holds. Every line reaching os_log or the handler has it replaced, raw or percent-encoded, wherever it sits. The engine already strips named parameters, `Bearer` / `Basic` credentials, userinfo, encoded payloads and the Xtream Codes path layout on its own, also inside a URL carried percent-encoded in another URL's query; this covers what only the host can know, such as a provider URL carrying the password as a bare path segment. Registrations are counted: a value registered twice stays redacted until it is unregistered twice. Returns false for a value under four bytes. `redacted(_:)` returns a line as the handler would receive it, for a line a host or tool prints itself next to the engine's. | | `EngineLog.emit(_:category:level:)` | Emit a host line into the same stream, for a host that wants its own events interleaved with the engine's. | | `segmentCacheDiskBytes`, `softwareHostFramesEnqueued` | Point reads for a stats overlay. | | `activeProducerShiftSeconds`, `frameAhead`, `clockLeadSeconds` | Divergence diagnostics for tracing a clock that disagrees with the picture. Not for production playback logic. | @@ -966,15 +1143,15 @@ All flags default to safe values; the table is the full set. Depth for the media | Option | Default | What it does | | --- | --- | --- | -| `httpHeaders` | empty | Static headers on probes, range and segment fetches. On direct `nativeRemoteHLS` loads they ride into the `AVURLAsset`; when relayed they stay on upstream requests. Forwarded to sidecar subtitle fetches unless overridden. | +| `httpHeaders` | empty | Static headers on every probe, range and segment fetch. On direct `nativeRemoteHLS` loads they ride into the `AVURLAsset`, so header-enforcing IPTV origins work; when the engine relays the load they stay on its upstream requests. Forwarded to sidecar subtitle fetches unless overridden. **Scoping differs by route.** On the plain `nativeRemoteHLS` bypass AVFoundation itself sends them to every host the playlists name (cross-host variants, segments and `EXT-X-KEY` URIs) and carries them across 302 redirects, so credential headers there reach every such host and the engine cannot narrow that. The per-origin credential scoping described for `HLSLiveIngestReader` applies only on the routes the engine fetches itself: the ingest, the disc reader, the origin relay, the subtitle proxy, the audio tap and the live subtitle renditions. A credential that must not reach every host a playlist names should not ride in `httpHeaders` on the bypass; pass it through `httpRequestAuthorization`, which keeps it on the engine relay. | | `httpRequestAuthorization` | nil | An `HTTPRequestAuthorization` resolver for native HLS requests and playlist preparation. Forces the engine relay, replaces static application headers per request, and supports a bounded changed-bearer retry after 401. See [the credential contract](#rotating-native-hls-credentials-without-replacing-the-item). | | `isLive` | false | Treat the source as live. Set it explicitly; duration-based auto-detection is too noisy. | -| `dvrWindowSeconds` | nil | Timeshift window. nil means live-only and `seek` is a no-op. | -| `liveJoinProfile` | `.standard` | A `LiveJoinProfile`. `.fastZap` collapses TARGETDURATION to the source GOP so an IPTV join costs seconds instead of a full holdback. | +| `dvrWindowSeconds` | nil | Timeshift window. nil means live-only and `seek` is a no-op. The window is a ceiling: the disk budget (a quarter of the free space, at most 2 GiB) bounds what is actually kept, so a long window on a high-bitrate channel or a small volume holds less than it asks for. With several engines running, each one's quarter is taken from the free space the others have not already claimed, so a tile added on a tight volume holds less history than the first one (AE#687). | +| `liveJoinProfile` | `.standard` | A `LiveJoinProfile`. `.fastZap` collapses TARGETDURATION to the source GOP plus headroom for one 1.5 x longer (AE#670) so an IPTV join costs seconds instead of a full holdback. | | `clampsLiveResumeToWindow` | true | Whether `play()` may move a behind-live playhead by itself (edge snap on a live-only source more than 45 s behind, or a landing above the retained floor when a DVR window has slid past it). `false` hands the whole decision to the host, which then also owns the eviction case. | -| `liveJoinStartsImmediately` | true | Cuts AVPlayer's stall-avoidance wait short once at the live join, over a buffer that is non-empty and at least 1.5 s deep. The join tail no host can otherwise reach; default since 6.55.0 on a device A/B, see the live-join section. | +| `liveJoinStartsImmediately` | true | Cuts AVPlayer's stall-avoidance wait short once at the live join, over a buffer that is non-empty and at least 1.5 s deep, on an item that has reached `readyToPlay` (AE#684). The join tail no host can otherwise reach; default since 6.55.0 on a device A/B, see the live-join section. | | `liveBlockingReload` | nil (auto) | LL-HLS blocking-reload override for loopback live sessions. Auto derives eligibility from observed upstream cadence, which is what keeps a bursty relay off a `-15410` loop. | -| `nativeRemoteHLS` | false | Hand a remote `master.m3u8` straight to AVPlayer: no demuxer probe, no loopback. Pair with `isLive: true`. | +| `nativeRemoteHLS` | false | Hand a remote `master.m3u8` straight to AVPlayer: no demuxer probe, no loopback. Built for `isLive: true`; a remote HLS VOD URL reaches this route regardless (AE#154). The clock here is item time, see `clock.$sourceTime` (AE#616). | | `nativeRemoteHLSIngestFallback` | true | The #168 / #293 carriage recovery and the #363 401/403 bypass refusal recovery. Setting it false turns both off. | | `audioOnly` | false | Lean audio pipeline, no video machinery. Also set automatically when the probe finds no video stream. | | `audioBridgeMode` | `.surroundCompat` | Bridge encoder for codecs that cannot stream-copy into fMP4. `.surroundCompat` uses EAC3 for a source with more than two channels and FLAC for one with two or fewer (no surround to carry). `.lossless` uses FLAC up to 7.1 throughout and needs a sink that accepts multichannel LPCM. | @@ -985,11 +1162,12 @@ All flags default to safe values; the table is the full set. Depth for the media | `externalSubtitles` | empty | Sidecar files registered at load, so they rank in the language preference and can join the native renditions. | | `prepareNativeSubtitles` | false | Declare WebVTT renditions so subtitles survive PiP / AirPlay / external display. | | `eagerNativeSubtitleReaders` | false | Populate those renditions at load instead of on first selection, for playlists AVKit auto-selects. Only meaningful with `prepareNativeSubtitles`. | +| `serveIFramePlaylist` | false | Serve an I-frame rendition next to the master, so a stock `AVPlayerViewController` shows scrub thumbnails and scans on I-frames with no host code. One keyframe per segment at full resolution; opens a second reader on the source. Absent (one log line says why) for live, sources without a keyframe index, `sequentialOrigin`, `heldSourceConnection`, single-request origins, discs and media-playlist routing. Hosts with their own transport bar use `scrubThumbnail`. | | `nativeSubtitlePreferredLanguages` | empty | Which rendition is marked `DEFAULT=YES`. Resolved by the same BCP-47 matching as the overlay pick, so inline and PiP / AirPlay agree (#590). Read back as `nativeSubtitleDefaultOrdinal`. Does not activate the overlay path, so it cannot double up with the native render. | | `preserveASSMarkup` | false | Emit raw ASS event lines instead of extracted text; pair with `TrackInfo.assHeader`. ASS / SSA codecs only, embedded and sidecar alike, so a session mixing an ASS track with a SubRip one needs no reload to cross between them (AE#587). | | `teletextPage` | nil | Fix the DVB teletext caption page instead of letting libzvbi auto-detect. | | `audioDelaySeconds` | 0 | Start the session with a lip-sync offset already in force; positive presents audio later. Same value `setAudioDelay(_:)` reads and writes, and a `reloadAtCurrentPosition(applying:)` can correct it. | -| `suppressDisplayCriteria` | false | Skip the display-criteria handshake entirely. For previews and headless runs. | +| `suppressDisplayCriteria` | false | Skip the display-criteria handshake entirely. For previews and headless runs. A host that clears `preferredDisplayCriteria` itself between loads (leaving full screen for a preview) no longer strands the engine's same-format skip: the next non-suppressed load reads the display manager, finds nothing set and writes again (AE#678). Clearing through `stop()` stays the cleaner path, because it resets the engine's record in the same step. | | `matchContentEnabled` | true | Mirror of `AVDisplayManager.isDisplayCriteriaMatchingEnabled`. False routes HDR through the auto-tonemap path. | | `panelIsInHDRMode` | false | **Host assertion** that the panel is presenting HDR right now, and the gate on whether the HDR10-to-DV upgrade is accepted upfront. It is an OR term over the engine's own readout on every platform, not a replacement for it and no longer confined to `suppressDisplayCriteria` hosts (AE#459). The readout it backs up is `currentEDRHeadroom > 1`, which answers only around a dynamic-range transition: an Apple TV whose output format is locked to HDR never makes one and reads as an SDR panel forever, and on tvOS 27 the property has stopped answering at all on at least one box. A wrong assertion costs one in-place media-playlist fallback (`-11848`), not the item. An in-place rebuild (an audio pick, a custom-source reload) runs no handshake, so it routes on the load's readout and display eligibility together with the session's current assertion and the refusal latch, rather than on this option alone (AE#541). | | `attemptsHDRMasterOnUnprovenPanel` | true | Serve the HDR master to an HDR-eligible display whose panel state is unproven, and let AVFoundation's acceptance or refusal be the readout `UIScreen` will not give (AE#459). VOD only. The proxy this replaces is `currentEDRHeadroom`, which is measurably unreliable: on one Apple TV 4K 3rd gen on tvOS 26.6 it read a flat 1.00 across 46 samples of HDR content while the TV's own info display reported HDR, and later the same day, same box, same output format, same title, it read 1.20. The panel was presenting HDR and the property was wrong about it; what moves it is not established. What media-direct costs there is not the picture but the manifest: the SUBTITLES rendition, the AUDIO rendition that is the only place AVFoundation reads an HLS language from, and SUPPLEMENTAL-CODECS. A panel that proves itself through the headroom short-circuits and attempts nothing. A refusal costs one in-place media fallback, measured at 223 ms end to end (`-11868` after 54 ms, zero `errorLog` events, position kept, no visible black frame), and is latched for the process, so a genuinely SDR display pays it once rather than per title. Set it false on a host that knows its display is SDR. Live never attempts regardless: a live fallback is a rejoin at the edge rather than a restored position, and that cost is unmeasured. The published `videoFormat` is not moved by an ATTEMPT. It is moved by an ACCEPTANCE: a master AVFoundation has not refused after the settle window publishes the presented format, which is a stronger reading than the headroom rather than a substitute for it. | @@ -998,23 +1176,25 @@ All flags default to safe values; the table is the full set. Depth for the media | `keepDvh1TagWithoutDV` | false | Diagnostic lever: force dvh1 tags and a master playlist regardless of display capability. | | `forceDolbyVisionOnNonDVDisplay` | false | **Experimental (AE#455).** On a display with no Dolby Vision of its own, serve an HEVC Profile 8.1 source the way a Profile 5 source is served (`dvh1` sample entry, `dvcC` rewritten to profile 5 / compatibility 0, `CODECS="dvh1.05.LL"`), so AVPlayer composes the RPU itself instead of handing the panel the static-metadata HDR10 base layer. The bitstream is untouched; only the container's claim about it changes. Ignored on a display that does Dolby Vision, and Profile 8.1 only. See [formats.md](formats.md#dolby-vision-signaling) for what it buys and what it risks. | | `dolbyVisionHandling` | `.automatic` | A `DolbyVisionHandling`. `.baseLayerOnly` presents the HDR10 / HLG base layer of a Dolby Vision source and leaves the Dolby Vision out of the container on every display: plain `hvc1` / `av01` sample entry, `dvcC` stripped, no `SUPPLEMENTAL-CODECS`, HDR10 / HLG display criteria, `videoFormat` reads the base layer's format while `sourceVideoFormat` and `sourceDVProfile` keep saying what the file carries. The route a host offers as "Dolby Vision: off (HDR10)", for a source whose Dolby Vision is wrong and whose base layer is right: the reported shape is a remux carrying a Profile 7 RPU under a container record claiming Profile 5, where a player that believes the record decodes YCbCr as IPT and the picture comes out green / violet. Applies to HEVC Profile 7 / 8.1 / 8.4, AV1 Profile 10.1 / 10.4, and a Profile 5 (or AV1 10.0) record whose VUI declares a BT.2020 YCbCr PQ or HLG base; a Profile 5 whose VUI is unspecified carries IPT-PQ-c2, has no base layer to present, keeps its route, and the engine says so in the log. Takes precedence over `forceDolbyVisionOnNonDVDisplay`. A tuning field, correctable through `reloadAtCurrentPosition(applying:)`; a Profile 5 record the VUI contradicts also stops refusing the software path under it. See [formats.md](formats.md#dolby-vision-signaling). | -| `preferredDecodePath` | `.automatic` | A `DecodePath`. `.software` serves this source through `SoftwarePlaybackHost` whatever the routing concluded, scoped to this session and costing the source nothing (seeks, the audio switch and the title switch all keep working). The escape for the formats `VTCapabilityProbe` deliberately cannot classify, and for live, which never reaches that gate at all. One-way: there is no `.native`. See [Overriding the decode path](#overriding-the-decode-path). | +| `preferredDecodePath` | `.automatic` | A `DecodePath`. `.software` serves this source through `SoftwarePlaybackHost` whatever the routing concluded, scoped to this session and costing the source nothing (seeks, the audio switch and the title switch all keep working). The escape for the formats `VTCapabilityProbe` deliberately cannot classify, and for live H.264 / HEVC, which never reaches that gate at all. One-way: there is no `.native`. See [Overriding the decode path](#overriding-the-decode-path). | +| `escalatesToSoftwarePath` | `true` | Whether a native session AVPlayer refuses on its merits (`CoreMediaErrorDomain`), or a live join without an entry point the native route can open (`AetherEngine.LiveJoin`, AE#627), may be rebuilt once on the software path (AE#561). `false` surfaces the failure as `.error` instead, for a host with its own fallback ladder (AE#629). Correctable. | +| `sharedOutputRole` | `.primary` (default) or `.secondary` | A `SharedOutputRole`. A secondary engine never writes display criteria and never owns Now Playing, whatever `suppressDisplayCriteria` and `ownsVideoNowPlayingSession` say, on the audio-only path too. It also skips the play gate's wait for a criteria write and the panel readout (`criteriaPanelReadout` is nil, as for any suppressed load), so its HDR labelling relies on `panelIsInHDRMode`. The role is fixed for a session, but a later `load()` may change it: as an identity field it only blocks corrections (Sodalite#175). A host that opted into `ownsVideoNowPlayingSession` and changes the role without a `stop()` keeps the native host's original Now Playing ownership until the next stop. | | `deinterlaceMode` | `.auto` | A `DeinterlaceMode` for the software path: the Metal / VideoToolbox graph with a CPU bwdif fallback, or `.software` to force the CPU path. | | `deinterlaceFieldRate` | `.field` | A `DeinterlaceFieldRate`: the hardware deinterlacer emits one frame per field (25i to 50p) or per frame. The software fallback is always frame rate, because doubling a CPU bwdif is the wrong trade and a fallback should not change cost class. | | `probesize`, `maxAnalyzeDuration` | nil | Caller-bounded open-time probe budget (defaults 50 MB / 60 s). They fail **open**: an over-tight budget loads with late-resolving tracks silently missing rather than throwing, so validate track presence if you tighten them. Do not pass `0` for `maxAnalyzeDuration`; FFmpeg maps it to a shorter heuristic. | | `forwardBufferSegments` | nil (10, about 40 s) | How far the producer may race ahead and how much the cache keeps resident. Clamped to 4...2700; past the historical 150 the real bound is the session's disk budget, so a "buffer without limit" option can pass `Int.max`. Ignored on `nativeRemoteHLS`. Native HLS retention is readable as `$residentRanges` (see Time). Software VOD uses the same forward-window and volume-budget policy for compressed packet read-ahead; its continuous selected A/V frontier is `bufferedPosition`, not native `residentRanges`. | -| `sequentialOrigin` | false | Declare an origin that fabricates range answers: one long-lived unranged GET, no ranged probes, non-seekable pb. **Seeking is unavailable**; re-request the archive at a shifted start instead. | +| `sequentialOrigin` | false | Declare an origin that fabricates range answers: one long-lived unranged GET, no ranged probes, non-seekable pb. **Seeking is unavailable**; re-request the archive at a shifted start instead. A URL source the reader finds forward-only by itself (`Range` ignored, so it can only stream front to back; the open confirms a 200 answer with a one byte range, because some CDNs answer a cache miss that way) is served this way without the declaration when its container states a duration, which a stated `Content-Length` lets an MPEG-TS source estimate; `loadedOptions` then reads `true` with that duration declared, and without a duration it stays on the software path. An AirPlay hop on such a session swaps the item onto the LAN address instead of reopening the source, since a reopen could only start again from byte 0. | | `declaredDurationSeconds` | nil | Trusted duration, overriding the container's. Required alongside `sequentialOrigin` on VOD, where the tail read is gone. | -| `maxConcurrentSourceRequests` | nil | Most requests the reader may have open against this origin at once, across every path it fetches on (pump ranges, detour blocks, size probes, tail prefetch, subtitle side reader). nil counts without capping and lowers the ceiling on its own after a 429/503/509. Set it when the provider states a limit; `1` also switches off the speculative parallel paths, which exist only to overlap with the pump. Counts **requests**, not TCP connections, because over HTTP/2 a session multiplexes every request onto one connection while the origin still counts each one (AE#377). It is also the only ceiling: several engines playing from one origin are bounded by this value and by what the origin refuses, not by a transport pool underneath it (AE#450). | -| `heldSourceConnection` | false | Ask the source **once** and pull it, instead of ending the connection at the reader's window high water and asking again every 8 to 16 MB of drain (AE#377). For an origin that punishes repeated requests rather than concurrency: some CDNs refuse new requests for minutes at a stretch while serving an already open connection at full rate, and against one of those the request cadence is the defect, at any range size (measured at the reporting origin: 32 MB ranges raised to 256 MB, eight times fewer requests, the refusals unchanged). Where `maxConcurrentSourceRequests` bounds how many requests are in flight, this removes the second request. The read path is the engine's own HTTP/1.1 over a demand-driven stream task, so: **HTTP/1.1 only** (no ALPN, an HTTP/2-only origin is out of scope), the **system proxy configuration does not apply**, and TLS is the OS's through the same host trust decision as every other engine session but has not been exercised against a self-signed origin. A viewer who pauses ends the connection after five seconds and resuming costs one request at the frontier, because a held flow nobody reads is the process-wide Network.framework starvation of AE#310. Applies to the playback reader; the subtitle and enrichment side readers keep the default transport, since they park for minutes at a time. **Names the session**: the transport is chosen at open, so a reload that changes it is refused. | +| `maxConcurrentSourceRequests` | nil | Most requests the reader may have open against this origin at once, across every path it fetches on (pump ranges, detour blocks, size probes, tail prefetch, subtitle side reader, and the HLS live ingest's playlist, segment and key fetches, whose prefetch window narrows to it, AE#678). nil counts without capping and lowers the ceiling on its own after a 429/503/509. Set it when the provider states a limit; `1` also switches off the speculative parallel paths, which exist only to overlap with the pump. Counts **requests**, not TCP connections, because over HTTP/2 a session multiplexes every request onto one connection while the origin still counts each one (AE#377). It is also the only ceiling: several engines playing from one origin are bounded by this value and by what the origin refuses, not by a transport pool underneath it (AE#450). | +| `heldSourceConnection` | false | Ask the source **once** and pull it, instead of ending the connection at the reader's window high water and asking again every 8 to 16 MB of drain (AE#377). For an origin that punishes repeated requests rather than concurrency: some CDNs refuse new requests for minutes at a stretch while serving an already open connection at full rate, and against one of those the request cadence is the defect, at any range size (measured at the reporting origin: 32 MB ranges raised to 256 MB, eight times fewer requests, the refusals unchanged). Where `maxConcurrentSourceRequests` bounds how many requests are in flight, this removes the second request. The read path is the engine's own HTTP/1.1 over a demand-driven stream task, so: **HTTP/1.1 only** (no ALPN, an HTTP/2-only origin is out of scope), the **system proxy configuration does not apply**, and TLS is the OS's through the same host trust decision as every other engine session but has not been exercised against a self-signed origin. A viewer who pauses ends the connection after five seconds and resuming costs one request at the frontier, because a held flow nobody reads is the process-wide Network.framework starvation of AE#310. Applies to the playback reader; the subtitle and enrichment side readers keep the default transport, since they park for minutes at a time. The open-time detection of an origin that ignores `Range` (see `sequentialOrigin`) does not cover it, because its open is open-ended. **Names the session**: the transport is chosen at open, so a reload that changes it is refused. | | `autoplay` | true | False mounts paused: the load skips the terminal `play()` and settles at `.paused` for a host that resumes later. It describes THIS MOUNT and nothing after it: the rebuilds a session makes on its own (`reloadAtCurrentPosition`, an option correction, the AirPlay LAN swap, an audio-delay nudge) come back in the transport state the session is in, not in this one. Correcting it through `reloadAtCurrentPosition(applying:)` therefore does nothing (the log names it as not applied rather than as applied, AE#464 round 3, and the call returns it as `sessionOwned` with `rebuilt` false rather than spending a teardown on it, AE#464 round 4); call `play()` or `pause()` instead. | ## Value types | Type | Carries | | --- | --- | -| `SourceProbe` | `url`, `durationSeconds`, `videoFormat`, `videoCodecID` / `videoCodecName`, `videoWidth` / `videoHeight`, `videoFrameRate`, `isDolbyVision`, `dvProfile`, `carriesHDR10PlusMetadata`, `audioTracks`, `subtitleTracks`, `metadata`, `isLive`. `carriesHDR10PlusMetadata` is `false` unless the probe was asked for `.hdr10Plus`, and a `false` means "not asked, or not seen inside the budget", never "proven absent". When it is true and the container said HDR10, `videoFormat` reads `.hdr10Plus`; a Dolby Vision source keeps `.dolbyVision` and carries the flag alongside. | -| `TrackInfo` | `id`, `name`, `codec`, `language`, `channels`, `bitrate`, `isDefault`, `isForced`, `isHearingImpaired`, `isCommentary`, `isAtmos`, `assHeader`, `isExternal`, `isNativelyRenderedSubtitle`. The last one marks a subtitle the playback backend draws itself (a remote-HLS rendition AVFoundation renders), so no cue reaches `subtitleCues` and an overlay control (position, delay, styling) has nothing to act on. | +| `SourceProbe` | `url`, `durationSeconds`, `videoFormat`, `videoCodecID` / `videoCodecName`, `videoWidth` / `videoHeight`, `videoFrameRate`, `isDolbyVision`, `dvProfile`, `carriesHDR10PlusMetadata`, `carriesHDRVividMetadata`, `audioTracks`, `subtitleTracks`, `metadata`, `isLive`, `videoStreamFormat` (the same `VideoStreamFormat` a session publishes as `$sourceVideoStreamFormat`). `carriesHDR10PlusMetadata` is `false` unless the probe was asked for `.hdr10Plus`, and a `false` means "not asked, or not seen inside the budget", never "proven absent". When it is true and the container said HDR10, `videoFormat` reads `.hdr10Plus`; a Dolby Vision source keeps `.dolbyVision` and carries the flag alongside. `carriesHDRVividMetadata` follows the same contract for `.hdrVivid` (#699) and never moves `videoFormat`: HDR Vivid rides an HLG or PQ base, which stays the label. | +| `TrackInfo` | `id`, `name`, `codec`, `language`, `channels`, `bitrate`, `isDefault`, `isForced`, `isHearingImpaired`, `isCommentary`, `isAtmos`, `assHeader`, `isExternal`, `isNativelyRenderedSubtitle`, and for audio `sampleRate`, `bitsPerSample`, `sampleFormat`, `channelLayout`, plus `profile` for every track (AE#658). `bitsPerSample` is 0 where the codec has no fixed depth (AAC, AC-3, E-AC-3 and Opus decode to float); `sampleFormat` is the probe decoder's output in libav's names ("fltp", "s32"); `profile` is libavcodec's profile name and the place DTS:X ("DTS-HD MA + DTS:X") and TrueHD Atmos ("Dolby TrueHD + Dolby Atmos") show up, where `isAtmos` covers E-AC-3 JOC only. `isNativelyRenderedSubtitle` marks a subtitle the playback backend draws itself (a remote-HLS rendition AVFoundation renders), so no cue reaches `subtitleCues` and an overlay control (position, delay, styling) has nothing to act on. | | `MediaMetadata` | `title`, `artist`, `album`, `artworkData`, `hasDisplayMetadata`. There is no separate album-artist field: a container's album artist is a fallback the parser folds into `artist`. | | `SubtitleCue` | `id`, `startTime`, `endTime`, `body` (a `SubtitleCue.Body`: `.text`, `.richText`, `.image`), `placement`, plus `text` and `isForced` conveniences. | | `SubtitleTextRun` | `text`, `color`, `isBold`, `isItalic`, `isUnderlined`, `isStruckThrough`, `fontName`, `fontSize`, `isStyled`. | diff --git a/docs/architecture.md b/docs/architecture.md index 2685ea960..e626e706c 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -8,7 +8,7 @@ AetherEngine has three playback pipelines, picked once at `load(url:)`: the audi ### Native AVPlayer pipeline (default) -Demux the source with libavformat, re-mux the elementary streams on the fly into HLS-fMP4, serve them from a local HTTP server on `127.0.0.1:`, point `AVPlayer` at the playlist. Apple's stack does all decode, all HDR / Dolby Vision signaling over HDMI, all audio routing. This is the path for HEVC and progressive H.264, which is what AVPlayer's HLS-fMP4 pipeline reliably accepts (interlaced H.264 routes to the software path for deinterlacing, #107). Atmos passthrough, DV HDMI handshake, HDR10 / HDR10+ system-side tone-mapping all live on this path. +Demux the source with libavformat, re-mux the elementary streams on the fly into HLS-fMP4, serve them from a local HTTP server on an ephemeral port (the player is handed `127.0.0.1://...`; the listener binds all interfaces so an AirPlay receiver can reach it over the LAN, behind the per-session path token, a 32-connection cap with 24 for non-loopback peers, and a 10 s head deadline for unauthenticated connections), point `AVPlayer` at the playlist. Apple's stack does all decode, all HDR / Dolby Vision signaling over HDMI, all audio routing. This is the path for HEVC and progressive H.264, which is what AVPlayer's HLS-fMP4 pipeline reliably accepts (interlaced H.264 routes to the software path for deinterlacing, #107). Atmos passthrough, DV HDMI handshake, HDR10 / HDR10+ system-side tone-mapping all live on this path. ``` Source URL ──► Demuxer ──► HLSSegmentProducer ──► SegmentCache ──► HLSLocalServer @@ -50,7 +50,7 @@ read is discarded: its observed totals remain, but final totals are incomplete, session rather than fabricated. A new playback session (including an episode handover on the same player) clears those totals and rejects any old reconciliation. -The playlist's segment boundaries come from a keyframe-aligned plan that mirrors the `hls` muxer's cut algorithm (segment N ends at the first IRAP at-or-after `(N+1) * targetSegmentDuration`), built in `HLSVideoEngine+SegmentPlanning.swift`. It needs the source's keyframe positions, which for MKV / MP4 come from a brief cue prewarm (a bounded seek that loads the Cues / `stss` index) and for MPEG-TS / M2TS come only from whatever `avformat_find_stream_info` plus that seek happened to scan. `keyframeIndexIsTrustworthy` gates the plan on two witnesses before trusting that index, falling back to a uniform-stride plan otherwise: the largest **gap** between consecutive keyframes must stay under a cap (a clustered TS index gaps by thousands of seconds; trusting it builds a multi-thousand-second first segment the `frag_custom` muxer buffers whole in RAM, #64), and the **coverage** from first to last indexed keyframe must span at least one `targetSegmentDuration`. The coverage check catches a remote MKV whose Cues tail read fails: the prewarm loads nothing, only the open-time keyframes survive bunched in the first few seconds, their gaps are tiny so the gap check passes, yet no keyframe reaches the first segment boundary, so the keyframe planner would degenerate to a single whole-file segment AVPlayer loads zero tracks from (`kFigAssetError_TrackNotFound`, #91). The uniform fallback anchors segment 0 at the content start so a late-starting title doesn't advertise empty leading segments. +The playlist's segment boundaries come from a keyframe-aligned plan that mirrors the `hls` muxer's cut algorithm (segment N ends at the first IRAP at-or-after `(N+1) * targetSegmentDuration`), built in `HLSVideoEngine+SegmentPlanning.swift`. It needs the source's keyframe positions, which for MKV / MP4 come from a brief cue prewarm (a bounded seek that loads the Cues / `stss` index) and for MPEG-TS / M2TS come only from whatever `avformat_find_stream_info` plus that seek happened to scan. `keyframeIndexIsTrustworthy` gates the plan on three witnesses before trusting that index, falling back to a uniform-stride plan otherwise: the largest **gap** between consecutive keyframes must stay under a cap (a clustered TS index gaps by thousands of seconds; trusting it builds a multi-thousand-second first segment the `frag_custom` muxer buffers whole in RAM, #64), and the **coverage** from first to last indexed keyframe must span at least one `targetSegmentDuration`. The coverage check catches a remote MKV whose Cues tail read fails: the prewarm loads nothing, only the open-time keyframes survive bunched in the first few seconds, their gaps are tiny so the gap check passes, yet no keyframe reaches the first segment boundary, so the keyframe planner would degenerate to a single whole-file segment AVPlayer loads zero tracks from (`kFigAssetError_TrackNotFound`, #91). The **tail** check requires that coverage to reach within 60 s of the source duration: a dense index that stops minutes short is a partial scan (an MKV whose Cues are missing or point past EOF leaves only what the capped prewarm walked), and trusting it cuts a last segment from the final scanned keyframe to the end of the title that the producer can never finish (PR #703). The uniform fallback anchors segment 0 at the content start so a late-starting title doesn't advertise empty leading segments. At runtime the producer honors those boundaries with a keyframe-gated, decode-order cut (`VODSegmentCutter`): a segment opens only at the IRAP that reaches the next boundary, so the IRAP is the segment's first sample and its open-GOP leading pictures stay with it, matching the live path and the `hls` muxer. The earlier routing keyed each packet to a segment by its DTS against the PTS-valued boundaries, so under B-frame reorder a keyframe whose DTS trailed its PTS fell into the previous segment and the next one started mid-GOP, decode-dependent on its predecessor; a fresh decode at that boundary (rebuffer recovery) surfaced it as transient blocky corruption (#92). @@ -64,7 +64,7 @@ What a host observes about a seek comes in two shapes. `isSeeking` / `seekTarget Restart latency is self-localizing (#93 follow-up): the "producer restarted" line carries a phase split (`stopWait/reopen/seek/build`), a producer's FIRST source read is timed, and any single `AVIOReader` read exceeding 2 s emits one `slow read` summary naming where the time went (detour fetches with network time, `connStallTimeout` waits, reconnects, backoff sleeps, bytes dropped by the stale-generation guard, generation span). A slow read with all-zero counters means the wait was upstream of the read loop. -A restart-window request must also never leave AVPlayer waiting in silence (#93 round 3): AVPlayer's media watchdog logs `-12889 "No response for media file"` after ~3.5 s without response HEADERS (holding the connection open does not help), and three strikes fail the item. A VOD serve still running at 2 s (`SlowServeSignal` armed by `VideoSegmentProvider.mediaSegment(at:onSlow:)`) therefore emits an early `200` with `Transfer-Encoding: chunked`; the segment follows as a single chunk when it lands, and a serve that ultimately misses aborts the connection (truncated transfer, AVPlayer retries) instead of framing a cacheable empty 200. Fast serves keep the byte-identical `Content-Length` response. If the item dies anyway, `failedToPlayToEndTime` parks it at rate 0 / `timeControlStatus == .paused` (with `item.status` often still `readyToPlay`), which every pause-guarded recovery layer used to misread as user intent, making the session terminal. The host now counts loopback-path end failures (`endFailureCount`), and the engine confirms the death through the same deferred window as the `.failed` KVO, then reloads the item through the stage-2 chain with the pause guard bypassed, bounded by `ItemDeathReviveGate` (3 attempts per dead spot; playback progress or a user seek away restores the budget). The bypass only admits the dead item; it does not overrule the viewer. The host records when AVPlayer's own `rate` dropped to 0, so a pause made before the death counts whether it came from the engine, AVKit, Control Center or PiP, and an item whose transport had already stopped before it died (`endFailureFollowedPause`) is reloaded paused at its anchor; the next Play resumes there. An engine-routed pause is stamped directly; a pause from AVKit or Control Center is seen only through the rate change. During the three-second confirmation window, a Play or Pause through the engine decides and a Play from outside the engine (a non-zero rate) resumes. A pause from AVKit or Control Center on a player whose rate is already 0 (the dead item in that window, or an item left stopped by a recovery) is not observed, because AVPlayer reports no change. +A restart-window request must also never leave AVPlayer waiting in silence (#93 round 3): AVPlayer's media watchdog logs `-12889 "No response for media file"` after ~3.5 s without response HEADERS (holding the connection open does not help), and three strikes fail the item. A VOD serve still running at 2 s (`SlowServeSignal` armed by `VideoSegmentProvider.mediaSegment(at:onSlow:)`) therefore emits an early `200` with `Transfer-Encoding: chunked`; the segment follows as a single chunk when it lands, and a serve that ultimately misses aborts the connection (truncated transfer, AVPlayer retries) instead of framing a cacheable empty 200. Fast serves keep the byte-identical `Content-Length` response. If the item dies anyway, `failedToPlayToEndTime` parks it at rate 0 / `timeControlStatus == .paused` (with `item.status` often still `readyToPlay`), which every pause-guarded recovery layer used to misread as user intent, making the session terminal. The host now counts loopback-path end failures (`endFailureCount`), and the engine confirms the death through the same deferred window as the `.failed` KVO, then reloads the item through the stage-2 chain with the pause guard bypassed, bounded by `ItemDeathReviveGate` (3 attempts per dead spot; playback progress or a user seek away restores the budget). The fresh item resumes only if the host's play intent says the viewer was playing: a paused title whose item dies (a tvOS screensaver taking the display is one way) is reloaded and stays paused. When a restart does run, it must reproduce segments on the SAME media timeline the continuous run gave them: the loopback's contract with AVPlayer is "static VOD server", and AVPlayer anchors fMP4 segments by their `tfdt`. Each restart allocates a fresh mp4 muxer, and movenc zero-bases a new instance's timeline by default, so a restart-produced segment used to carry `tfdt=0` while the playlist placed it at its plan offset: an implicit timeline discontinuity on every restart, papered over for plain playback but fatal to ancillary consumers (AVKit's legible renderer detaches mid-PiP, Sodalite#32; playhead/loaded-range decoupling, #93). The muxer therefore sets `movflags +frag_discont` with `avoid_negative_ts=disabled` so `tfdt` carries the producer's absolute output timestamps, the restart audio gate inherits the session shift (video shift rescaled) instead of snapping audio onto the video seam, and leading head-of-stream audio that would map below 0 is dropped (the muxer no longer absorbs negative timestamps). A restarted segment is byte-identical to its continuous twin modulo the per-muxer `mfhd` sequence number (pinned by `RestartTimelineContinuityTests` on a committed A/V fixture); on matroska sources, per-sample DTS synthesis after a demuxer seek scatters the DTS decomposition and boundary-frame membership by a frame or two, but presentation timestamps and `tfdt` anchoring stay epoch-invariant. Because `tfdt` carries `unsigned int(64)`, that same setting makes a negative output axis unrepresentable rather than merely unusual, so the published first timestamp is clamped at zero (AE#509): libavformat serves an MPEG-TS whose first DTS sits within 60 s of the 33-bit PTS wrap with every timestamp `2^33` ticks low (`AV_PTS_WRAP_SUB_OFFSET`), and a live join there published `baseMediaDecodeTime = 2^64 - |dts|` against a playlist starting at 0, which AVPlayer answers by fetching the whole window and placing none of it, with no error and no stall of its own. @@ -90,7 +90,7 @@ A seek holds the last frame on screen rather than blanking it. `SampleBufferRend `AudioDecoder` stamps each `CMSampleBuffer` from a running sample count anchored to the first frame (`AudioClockAnchor`), not from the container-quantized per-packet PTS. Container timebases are coarse (1 ms in MKV), so when a frame's duration is not an integer number of ticks (a 1536-sample AC-3 frame is 34.83 ms at 44.1 kHz but exactly 32 ms at 48 kHz) the quantized PTS leave a sub-millisecond gap or overlap at every buffer boundary, and `AVSampleBufferAudioRenderer` reconciles a discontinuity at each one (~29 clicks/sec, a continuous crackle). Anchoring to the sample clock makes consecutive buffers abut exactly; a real source discontinuity (> 100 ms off the predicted clock, i.e. a seek or edit) re-anchors so genuine gaps are not papered over, and `flush()` drops the anchor. The clock advances only on a successfully emitted buffer, so a dropped buffer injects no phantom samples. -AV1+DV (Profile 10.0 / 10.1 / 10.4) routes through the native path on hardware-AV1 hosts via the `dav1` / `av01` track type plus the source's `dvvC` box. AV1+Atmos is genuinely rare in the wild (mastering still runs in HEVC overwhelmingly), so the SW pipeline's lack of Atmos passthrough is a theoretical limitation rather than a real one. The dispatch happens once at load time; hosts see a unified `@Published` state surface either way. +AV1+DV (Profile 10.0 / 10.1 / 10.4) routes through the native path on hardware-AV1 hosts via the `dav1` / `av01` track type plus the source's `dvvC` box. AV1+Atmos is genuinely rare in the wild (mastering still runs in HEVC overwhelmingly), so the SW pipeline's lack of Atmos passthrough is a theoretical limitation rather than a real one. AV1 capability registration is lazy and runs only when routing an AV1 source; loading H.264, HEVC or audio-only content does not consult it. The dispatch happens once at load time; hosts see a unified `@Published` state surface either way. **Background audio (iOS).** When the app backgrounds while playing, the engine keeps audio going rather than tearing the pipeline down. The decision is a pure, unit-tested policy, `backgroundAction(isAudioBackend:hasSoftwareHost:keepVideoAlive:state:)`, driven from the `UIApplication` lifecycle observers; `keepVideoAlive` comes from `shouldKeepVideoAlive(enabled:pipActive:state:)` and is gated to iOS (tvOS always tears down, wedge-safe: a frozen decode session crossing a multi-hour suspension wedged `mediaserverd`). On the native path "keep audio alive" is just declining to tear down: `AVPlayer` under the `.playback` session keeps decoding. The software path has no `AVPlayer`, and its combined demux loop normally paces the whole loop (audio and video) on the video renderer's `isReadyForMoreMediaData`; once `AVSampleBufferDisplayLayer` stops draining in the background that gate never reopens and audio would starve. So the host enters `backgroundAudioOnly`: the loop drops video packets and paces on the audio renderer (`AudioOutput.isReadyForMoreMediaData`) instead, keeping `AVSampleBufferAudioRenderer` fed and the synchronizer advancing. On foreground return the flag clears, the video decoder and renderer flush, and video resyncs at the next keyframe with audio uninterrupted. Scope is the combined VOD loop (and live-without-DVR, which shares it); the DVR feeder loop is unchanged. Exercise it headless with `aetherctl bgaudio` (see [cli.md](cli.md)). @@ -190,12 +190,17 @@ Runtime cache corruption is a reported playback failure, not silently skipped da The cache frontier is the intersection of selected audio and video presentation coverage containing the playhead. Unknown intervals remain unknown; byte counts -are not converted to guessed seconds. H.264 uses a bounded presentation reorder -queue and confirmed successor timestamps, because a VFR packet's decode duration -can be shorter than the picture's actual display hold. Larger discontinuities, +are not converted to guessed seconds. H.264 and HEVC use a bounded presentation +reorder queue and confirmed successor timestamps, because a VFR packet's decode +duration can be shorter than the picture's actual display hold, and a container +that rounds durations down (41 ms against 41/42 ms deltas at 23.976 fps) would +otherwise split the coverage at every frame (#613). Larger discontinuities, invalid or unexpectedly late timestamps invalidate or split coverage. Other -codecs retain strict packet-duration coverage. Without a proven compressed -frontier, the existing decoded-cushion fallback still applies. +codecs retain strict packet-duration coverage. Coverage keeps its history behind +the playhead for backward cached seeks, up to 4096 ranges per stream; a full +coverage forgets the ranges wholly behind the playhead rather than stop +describing new packets. Without a proven compressed frontier, the existing +decoded-cushion fallback still applies. A cached seek restores a retained keyframe cursor, including an earlier keyframe for available preroll, and keeps the producer at its existing source frontier. @@ -353,7 +358,7 @@ Sources/AetherEngine/ │ ├── ResidentSpan.swift Bytes already in memory that a read can be served from without a round trip: the open window parked across a parse seek, and the speculative tail fetch. Not a cache (no fetch-on-miss, no eviction, no block alignment); it either covers an offset or it does not (#281) │ ├── CustomIOReaderBridge.swift Bridges a host-supplied IOReader into avio_alloc_context read / seek callbacks, with the same read deadline and reversible read abort as AVIOReader so bounded seeks also bound (and unwedge) disc-adapter sources │ ├── Demuxer.swift libavformat wrapper; seek + bounded seek (deadline-capped) + byte-estimate positioning fallback for index-less containers (origin-corrected onto the file-relative time axis, fixed early bias in seconds, landing verified by a packet-PTS probe with proportional correction, and a sticky per-demuxer timestamp-seek lockout once the mechanism proves broken, #112). Per-open `DemuxerOpenProfile` budgets `find_stream_info` (probesize / max_analyze_duration), caller-overridable on the main playback open via `LoadOptions.probesize` / `maxAnalyzeDuration`. The native text readers' side demuxer sets `skipStreamInfo` to drop the `find_stream_info` pass entirely (codec_id / codec_type come from the container header / PMT at open); the reader runs a bounded `resolveStreamInfo()` on demand only if its target stream's codec is genuinely unresolved at open (#87) -│ ├── RedirectHeaderPolicy.swift Per-header redirect replay policy for AVIOReader's URLSession delegates: Range and non-credential extra headers survive cross-host redirects (header-dependent proxies, #8), credential headers (Authorization, Cookie, Emby/Jellyfin tokens) are replayed only to a same-host target with no TLS downgrade and scrubbed otherwise, so presigned redirect targets neither 400 on conflicting auth nor see the media-server token (#126) +│ ├── RedirectHeaderPolicy.swift Per-header redirect replay policy for AVIOReader's URLSession delegates and, through `EngineTLS`'s session-level delegate, for every other engine session (audit NET-108): Range and non-credential extra headers survive cross-host redirects (header-dependent proxies, #8), credential headers (Authorization, Cookie, Emby/Jellyfin tokens) are replayed only to a same-host target with no TLS downgrade and scrubbed otherwise, so presigned redirect targets neither 400 on conflicting auth nor see the media-server token (#126). `CredentialScope` applies the same rule to every URI a playlist names: the relay, the #316 subtitle proxy, the audio tap and the live rendition fetch send the credentials only to the URL the host handed over, and allowing a fetch never grants them (audit NET-109) │ ├── SlowReadDiagnostics.swift One-shot localization of a pathologically slow AVIOReader.read() (detour fetch / connStall / reconnect / backoff / dropped-generation bytes), #93 restart latency │ ├── SourceThrottle.swift Pure virtual-clock leaky-bucket rate limiter on the source read path (slow-CDN simulation for aetherctl --throttle); unit-testable without sleeping │ └── SourceContentLengthCache.swift Memoizes a source URL's resolved total byte length across demuxer opens, so a subtitle side demuxer opening the same origin later inherits the size the video producer already resolved (#112) @@ -379,7 +384,8 @@ Sources/AetherEngine/ │ ├── DiscReader.swift Disc detection + routing: local `.iso` URLs and custom ISO readers into the demux path; enumerates titles and threads the selected one (DVD vs Blu-ray) │ ├── DiscMetadata.swift Public `TitleInfo` / `ChapterInfo` plus the internal disc title + chapter model (45 kHz ticks, extent keys) │ ├── ISO9660Reader.swift Read-only ISO9660 bridge-filesystem reader (DVD-Video images) -│ ├── DVDIFOParser.swift DVD VMGI TT_SRPT title list + each VTS IFO program chain (per-title duration + chapters) and stream attribute tables (track languages) +│ ├── DVDIFOParser.swift DVD VMGI TT_SRPT title list + each VTS IFO program chain (per-title duration + chapters) and stream attribute tables (track languages, declared subpicture streams) +│ ├── DVDSubpictureAssembler.swift Joins a DVD subpicture's PES fragments for the streams the engine declares from the IFO (the `dvdsub` parser's job, which libavformat only does for streams it created) │ ├── DVDTitleSelector.swift Groups DVD title sets' content VOBs into selectable titles (whole-VTS, largest first) │ ├── ConcatIOReader.swift Synthetic seekable IOReader concatenating byte extents (DVD VOBs / Blu-ray M2TS clips) into one source │ ├── UDFReader.swift Read-only UDF 2.50 reader (Blu-ray BDMV, including the metadata partition and fragmented-file allocation descriptors) @@ -426,6 +432,7 @@ Sources/AetherEngine/ │ ├── AirPlayPlaylistDecision.swift Pure playlist choice for the wireless-AirPlay loopback rewrite: which of master / media a receiver is handed, kept separate and testable offline like `MasterFallbackDecision` (#86, #227) │ ├── NativeLegibleDeselectPin.swift Keeps AVKit's ready-time legible auto-select from engaging a rendition the host never asked for (Sodalite#38), and publishes the caption requests the system makes on its own as `systemCaptionRequest`, by BCP 47 language rather than by ordinal (Sodalite#65) │ ├── RemoteHLSFormatDetection.swift Dynamic-range classification for the `nativeRemoteHLS` bypass, read back from AVPlayer's own track format instead of a second origin probe, which that path exists to avoid (#168) +│ ├── RemoteHLSStreamDescription.swift The rest of that read-back for stats panels: delivered dimensions, colour description and avcC/hvcC profile as a `VideoStreamFormat`, and the item's audio tracks as `TrackInfo` (codec, channels, rate, dec3 JOC) │ ├── RemoteHLSIngestFallback.swift Reroutes a bypass session onto the live-ingest loopback when the master advertises HEVC / DV / AV1 but delivers MPEG-TS, carriage AVFoundation builds no video track for (#168) │ ├── RemoteHLSMasterRewrite.swift Declares host-supplied sidecar subtitles as legible renditions on the bypass by rewriting the origin master, since media selection on an HLS asset comes from the playlist and nowhere else (#316) │ ├── RemoteHLSSubtitleProxy.swift Stands a loopback origin in front of that remote master: two playlist GETs, a rewrite and a socket, and anything that does not line up keeps the plain origin URL (#316) @@ -438,7 +445,7 @@ Sources/AetherEngine/ │ ├── SoftwarePiPSource.swift Everything a host needs to build an `AVPictureInPictureController` content source around the software path without AVKit entering the engine: the layer plus the four transport answers behind the sample-buffer delegate │ └── StartupReadinessGate.swift Outcome of one startup-readiness attempt: the reloaded item reached a playable state, died, or ran out the settle window without doing either ├── Network/ -│ └── HLSLocalServer.swift Native path: local HTTP server (127.0.0.1) serving playlist + segments +│ └── HLSLocalServer.swift Native path: local HTTP server (all interfaces for AirPlay, every path behind a per-session token, loopback URLs handed to the local player) serving playlist + segments ├── Renderer/ │ ├── SampleBufferRenderer.swift SW path: AVSampleBufferDisplayLayer + B-frame reorder, HDR10+ attachments; `flush(removingDisplayedImage:)` holds the last frame through a seek (#90); the decode thread reaches the layer only through its `sampleBufferRenderer`, taken once on the main actor (#351) │ └── SubtitleFrameCompositor.swift Composites active cues into decoded software-path frames while PiP is active, since the system PiP window renders only the sample-buffer layer; playback wins, every failure path returns the original buffer untouched @@ -465,6 +472,7 @@ Sources/AetherEngine/ │ ├── HLSVideoEngine+SegmentPlanning.swift Native path: keyframe / uniform segment plans, extradata + AAC fixups │ ├── HLSVideoEngine+LiveReopen.swift Native path: live source-loss recovery (capped-backoff reopen on the same timeline); VOD backpressure-wedge re-anchor + consumer re-engage nudge, which re-reads the rendered position at nudge time so the zero-tolerance seek never lands behind the on-screen frame (#115) │ ├── CodecRoutePolicy.swift Native path: DV / HDR / codec routing decisions (track types, CODECS strings, VIDEO-RANGE) +│ ├── AnnexBSampleConverter.swift Native path: Annex-B HEVC samples rewritten to length-prefixed NALs by the session muxer, in-band VPS/SPS/PPS kept. movenc's own conversion under `hvc1` drops them, so a stream that sends a new PPS mid-title stopped decoding at the change (PR #703) │ ├── DoviRpuConverter.swift Native path: per-packet DV Profile 7 → 8.1 RPU conversion via libdovi (NAL surgery: convert type-62 RPU, drop type-63 EL) │ ├── DoviRpuConverter+Probe.swift Diagnostic DV-conversion probe (`doviConvertProbe` / `DoviConvertProbeResult`), backs `aetherctl dovitest` │ ├── Issue65LivelockBreakers.swift Pure backpressure-wedge detection (`BackpressureWedgeDetector`) breaking the VOD HLS scrub-burst livelock (#65); `seekIsWedged` starvation check + `SeekResumeGuard` single-resume latch for the deadline-bounded seek @@ -476,7 +484,7 @@ Sources/AetherEngine/ │ ├── PlanBoundaryAxis.swift Which axis a container stamps its index entries on, and therefore which packet timestamp the cutter gate may compare against a plan boundary: decode for a mov/mp4 sample table, presentation for a Matroska Cue (#358, AE#561) │ ├── VideoConfigRecord.swift The `hvcC` / `avcC` config record and the framing question hanging off it (#365): movenc decides whether to Annex-B-convert every sample from the EXTRADATA rather than the packet, so a source carrying Annex-B parameter sets while muxing length-prefixed NALs has each sample rewritten by a converter that finds no start codes. Mirrors movenc's own two tests instead of an equivalent-looking predicate, since predicting that decision wrong is the whole defect │ ├── H264SPS.swift Hand-rolled H.264 SPS parser (SSAI ad-creative coded dimensions / codec config) -│ ├── H264CompositionOffsetRepair.swift Rebuilds the presentation axis of an MP4 whose writer dropped `ctts` while the bitstream still reorders pictures: libavcodec's H.264 parser supplies each access unit's picture order count without decoding, a fail-closed head sample settles the ladder and the shift, and packets are rewritten to the timeline a correct muxer would have written, so the native path and hardware decode are kept (#409) +│ ├── H264CompositionOffsetRepair.swift Rebuilds the presentation axis of an MP4 whose writer dropped `ctts` while the bitstream still reorders pictures: libavcodec's H.264 or HEVC parser supplies each access unit's picture order count without decoding, a fail-closed head sample settles the ladder and the shift, and packets are rewritten to the timeline a correct muxer would have written, so the native path and hardware decode are kept (#409, HEVC #699) │ ├── OutputTimestampSanitizer.swift Final-stage DTS/PTS monotonicity guard before the fMP4 mux (SSAI splices, program restarts) │ ├── RestartCoalescer.swift Coalesces a burst of producer-restart requests into one in-flight + one settled target (rapid-seek, AetherEngine#35) │ ├── LiveWindow.swift Live path: session-relative DVR timeline (seconds since first frame), shared by the native and SW live paths @@ -484,11 +492,13 @@ Sources/AetherEngine/ │ ├── NALUnitChain.swift Final-stage payload guard before the fMP4 mux: a video sample whose length-prefixed NAL chain overruns it is cut at its last complete unit, because Apple's parser answers one such sample with -19602 for the whole segment (AE#561) │ ├── AudioLanguageMap.swift Native path: the ISO 639-2/T an audio track is declared with in the master's `EXT-X-MEDIA:TYPE=AUDIO` rendition and written into its `mdhd`; ICU in front of the twenty ISO 639-2/B bibliographic codes Matroska writes, then canonicalization for the ISO 639-3 members CLDR aliases and pass-through for the ones it does not, failing closed on anything ICU can neither resolve nor name (AE#458) │ ├── FragmentSplitter.swift Native path: routes mp4 muxer's avio output stream into init.mp4 (ftyp+moov) vs per-segment moof+mdat files -│ ├── PacketRingBuffer.swift Live path: keyframe-indexed, disk-spooled packet ring backing the SW-path DVR rewind +│ ├── PacketRingBuffer.swift Live path: keyframe-indexed, disk-spooled packet ring backing the SW-path DVR rewind (append-only chunk files, a 24-byte index record per packet, eviction by time and by the session retention budget, evict-and-retry when the volume or quota is full) +│ ├── SessionDirectoryLiveness.swift The flock marker a session directory under `aether-segments/` holds for its owner's lifetime, so the stale-session sweep spares a live SegmentCache or DVR ring (AE#451, SEG-105) │ ├── SegmentCache.swift Native path: producer/consumer segment store with backpressure, scrub-aware eviction + byte-budgeted VOD backward retention (restart-free back-seeks) │ ├── VTCapabilityProbe.swift AV1 system-decode probe + per-format `canHardwareDecode` HW-decodability check for H.264 / HEVC profiles (Intel Macs, older chips, #2); gates codec routing (VP9 / VP8 / MPEG-4 Part 2 / MPEG-2 / VC-1 and interlaced H.264 always route SW, see VideoRoutingPolicy) │ ├── LiveCadencePolicy.swift Turns the observed arrival cadence into the two LL-HLS decisions (served TARGETDURATION, blocking-reload eligibility), re-evaluated on every manifest render, instead of trusting a bursty origin's own number -│ └── PrefetchDiskBudget.swift Disk bound for an opt-in whole-source prefetch: the hard window is never evicted, so the budget bounds only what lives outside it (#207) +│ ├── PrefetchDiskBudget.swift Disk bound for an opt-in whole-source prefetch: the hard window is never evicted, so the budget bounds only what lives outside it (#207) +│ └── RetentionClaims.swift Process-wide ledger of the running sessions' disk allowances: a new session is sized from the free space minus what the others may still write, so concurrent engines cannot each claim a quarter of the same space (#687) └── View/ └── AetherPlayerView.swift Polymorphic surface: hosts either AVPlayerLayer (native) or AVSampleBufferDisplayLayer (SW) ``` diff --git a/docs/cli.md b/docs/cli.md index 4ffe2cae3..0d8b96a5a 100644 --- a/docs/cli.md +++ b/docs/cli.md @@ -33,25 +33,26 @@ Twenty-one subcommands plus the bare-URL `serve` alias. Opens the demuxer, prints the codec / resolution / frame rate of the video track, the audio track list (codec, channels, language, Atmos flag), the subtitle track list, the parsed container metadata (`MediaMetadata`: title / artist / album / albumArtist + embedded cover art presence), then exits. No HLS server is started. -`--detect-hdr10plus` and `--detect-atmos` add the opt-in detail passes of `AetherEngine.probe(url:detecting:)`, and both can be given at once (one open, one connection). HDR10+ is the interesting one to watch: the bare `probe` reads only what the container declares, and ST 2094-40 is declared nowhere, so a carrying source prints `format: hdr10` without the flag and `format: hdr10Plus` plus `HDR10+: ST 2094-40 metadata seen` with it. `not seen` means "not inside the scan budget", not "proven absent". +`--detect-hdr10plus` and `--detect-atmos` add the opt-in detail passes of `AetherEngine.probe(url:detecting:)`, and both can be given at once (one open, one connection). HDR10+ is the interesting one to watch: the bare `probe` reads only what the container declares, and ST 2094-40 is declared nowhere, so a carrying source prints `format: hdr10` without the flag and `format: hdr10Plus` plus `HDR10+: ST 2094-40 metadata seen` with it. `not seen` means "not inside the scan budget", not "proven absent". `--detect-hdr-vivid` adds the HDR Vivid (CUVA) check to the same packet pass and prints `HDR Vivid: CUVA metadata seen`; `format:` stays `hlg` / `hdr10`, since Vivid rides that base (#699). ```bash swift run aetherctl probe --detect-hdr10plus /path/to/hdr10plus.mkv ``` -`Scripts/make-hdr10plus-fixture.py ` builds a ~1 KB HEVC/PQ fixture that carries a real ST 2094-40 T.35 SEI (and prints it base64, which is how the two fixtures embedded in `HDR10PlusProbeIntegrationTests` were made). It verifies itself: it only emits the file when `ffprobe -show_frames` reports `HDR Dynamic Metadata SMPTE2094-40` on it, so the payload is one FFmpeg's own parser accepts rather than a byte pattern that resembles one. +`Scripts/make-hdr10plus-fixture.py ` builds a ~1 KB HEVC/PQ fixture that carries a real ST 2094-40 T.35 SEI (and prints it base64, which is how the two fixtures embedded in `HDR10PlusProbeIntegrationTests` were made). It verifies itself: it only emits the file when `ffprobe -show_frames` reports `HDR Dynamic Metadata SMPTE2094-40` on it, so the payload is one FFmpeg's own parser accepts rather than a byte pattern that resembles one. With `--vivid` it builds an HEVC/HLG fixture carrying a CUVA HDR Vivid SEI instead, verified the same way against ffprobe's `CUVA` side data (the fixture in `HDRVividProbeTests`). ## serve -The original behavior. The CLI prints the loopback URL and parks until Ctrl-C; from another terminal you can: +The original behavior. The CLI prints the loopback URL (`=== PLAYBACK URL ===`) and parks until Ctrl-C. The server listens on all interfaces (so an AirPlay receiver can reach it) and answers only paths that start with a per-session token, anything else is a 404, so the printed URL carries it: `http://127.0.0.1://...`. Take the `/` prefix from that line; from another terminal you can: ```bash -curl -i http://127.0.0.1:/master.m3u8 -curl -o /tmp/init.mp4 http://127.0.0.1:/init.mp4 -mediastreamvalidator http://127.0.0.1:/master.m3u8 +curl -i http://127.0.0.1://master.m3u8 +curl -o /tmp/init.mp4 http://127.0.0.1://init.mp4 +mediastreamvalidator http://127.0.0.1://master.m3u8 mp4dump --verbosity 1 /tmp/init.mp4 +curl -o /tmp/seg0.mp4 http://127.0.0.1://seg0.mp4 ffprobe -v debug /tmp/seg0.mp4 -open 'http://127.0.0.1:/master.m3u8' # macOS QuickTime +open 'http://127.0.0.1://master.m3u8' # macOS QuickTime ``` `--no-dv` forces the SDR / HDR10 route even for a Dolby Vision source (compare the two playlists). @@ -64,6 +65,8 @@ open 'http://127.0.0.1:/master.m3u8' # macOS QuickTime `--native-subs ` turns on the native WebVTT subtitle renditions (the `LoadOptions.prepareNativeSubtitles` path a full session uses): the engine calls `requestNativeSubtitleTrack()` before `start()`, then `attachAllNativeSubtitleStores()` after start. Every non-bitmap text track is served as a language-tagged `EXT-X-MEDIA:TYPE=SUBTITLES` rendition (`DEFAULT=NO,AUTOSELECT=NO`) in the master playlist, backed by a per-track `subs_N.m3u8` WebVTT media playlist. (An earlier design muxed `mov_text`/tx3g traks into the fMP4; in-band timed text is not HLS-conformant and AVPlayer rejected it, so the WebVTT rendition replaced it, see [formats.md › Native subtitle renditions](formats.md#native-subtitle-renditions-webvtt-for-pip-airplay-and-external-display).) The `` value is legacy and now ignored, kept only for CLI compatibility: every non-bitmap track is always declared, and actual track selection happens via the host API in a full session, not from this flag. `curl` the `master.m3u8` (or open it in QuickTime) to verify the `SUBTITLES` group + `subs_N.m3u8` endpoints enumerate every language as a legible `AVMediaSelection` group. Omit the flag to reproduce the default behavior (no renditions, output identical to before). +`--iframes` lists an I-frame rendition in the master (the `LoadOptions.serveIFramePlaylist` path a full session uses, AE#682): the engine calls `requestIFramePlaylist()` before `start()`. The master then carries `#EXT-X-I-FRAME-STREAM-INF` and the server answers `iframe.m3u8`, `iframe_init.mp4` and `iframe.mp4`, one keyframe per segment. The log line `[HLSVideoEngine] i-frame rendition: served segments=` or `absent reason=` says which it was. `curl` the playlist and a fragment next to the printed playback URL to look at them. + `--prewarm` / `--prewarm-bytes N` (AE#551, `play` only) warm the source before the load, the way a host warms the next episode, and print what was retained and how long it took. The load that follows adopts those bytes: its reads inside the warm head cost no request, and its data connection starts at the warm frontier instead of byte zero. Measure it against a REAL origin. On loopback the round trip it removes costs nothing to begin with, which is exactly the trap the #281 tail prefetch was built out of. `--throttle-kbps N` is a TEST-ONLY slow-CDN simulation: it caps source-IO delivery to N kbit/s. Set it below the stream bitrate to starve the producer below real-time and provoke AVPlayer rebuffers (for example the #92 open-GOP repro). Also available on `seektest` and `play`. @@ -80,7 +83,7 @@ open 'http://127.0.0.1:/master.m3u8' # macOS QuickTime ## swdecode -Opens `SoftwareVideoDecoder` for the source's video stream, feeds up to N packets (default 100, override with `--frames N`), and reports counters plus first-frame metadata (pixel format, dimensions). Tests the SW-pipeline decode path end-to-end without needing a render layer. Useful for legacy codecs (MPEG-4 Part 2, MPEG-2, VC-1) and AV1 / VP9 on platforms where the native AVPlayer path doesn't accept them. Verdict distinguishes three failure modes: +Opens `SoftwareVideoDecoder` for the source's video stream, feeds up to N packets (default 100, override with `--frames N`), and reports counters plus first-frame metadata (pixel format, dimensions, and `firstFrameColor`, the colour tags the picture reaches the display layer with, AE#654). Tests the SW-pipeline decode path end-to-end without needing a render layer. Useful for legacy codecs (MPEG-4 Part 2, MPEG-2, VC-1) and AV1 / VP9 on platforms where the native AVPlayer path doesn't accept them. Verdict distinguishes three failure modes: - decoder open failed (FFmpegBuild gate or malformed extradata) - decoder opened but no frames produced (pixel-format conversion, no IDR in window) @@ -142,7 +145,9 @@ the same clock as `FIRSTFRAME`. The 1 Hz tick samples the phase, which is far to signal apart from the moment the rate rolls; a healthy native join is exactly two edges, `loading` at the load and `playing` at the roll (AE#440). -`--subs ` matches against the track's libavcodec name or language and logs every overlay cue and cue trim as it lands. `--host-calls` replays host post-load behavior against the fresh session: `play`, `extractor` (`makeFrameExtractor`), `setrate` (`setRate(1.0)`), `pausestart` (Sodalite#104 round 4: `pause()` the instant load returns, before any frame exists, and `play()` at t=8; the shape of a host that holds a fresh load paused, and a software session used to answer it with eight ticks of `enq=+0 status=unknown r4d=n`, a black picture under a paused clock; now the `[SWHost] #104` lines show the first frame presented at a stopped clock and `startup 8/8 presenting` arrives while paused), `ratehold` (set 1.5, pause at tick 3, resume at tick 5, then read the rate back off the transport itself: the #436 drill, and it fails the run if the resume came back at 1.0), `reloadlive` (reload the URL on the live path when the probe flags it live, the AetherPlayer Open URL flow), `seekback` (rewind 20 s into the DVR window at t=15, return to the live edge at t=30), `overlapseek` (the #292 seek-window drills below), `pausehold` (Sodalite#104: pause at t=10 and HOLD until ten seconds before the end, printing the playhead, the edge and the resident depth every second, which is how a session paused for longer than its own DVR window is measured without waiting out a real one: pair it with `--dvr-window 30` and `--seconds 90` and the ninety minute question becomes a ninety second run), `still` (#544: asks for a scrub still at three aims, 20 s behind the playhead at t=15, 5 s behind at t=20 and at the edge at t=25, writing each to `/tmp/aetherctl-still-.png` and reporting hit or MISS with the decode time; pair it with `--sw --dvr-window N`, where the picture comes out of the DVR packet ring rather than a segment cache, and read the FILE as well as the count, because the bundled seed burns its own second into the frame so a still asked for 14.85 s showing `14` is the verdict that it decoded the right moment and not merely an image; exit 6 when nothing hit, 5 when the session ended before the first aim), `stallclock` (AE#549: stop the master clock at t=4 behind the host's back, the way an interrupted audio session does, then call a plain `play()` at t=7 and read whether the playhead moves again; the interruption itself cannot be staged on macOS, its outcome can, and without the `RendererClockResume` branch the run ends with the clock standing exactly where the stall left it, which is the field log's shape), and `pauseseek` (pause at t=12, seek at t=15 while paused, resume at t=20; with `--sw` the five paused ticks between landing and resume show what the `[SWDiag]` line reports while the pump is parked and has not heard of the seek, the AE#479 shape); this is how the pre-arming `setRate` wedge was isolated. +`--subs ` matches against the track's libavcodec name or language and logs every overlay cue and cue trim as it lands. `--host-calls` replays host post-load behavior against the fresh session: `play`, `extractor` (`makeFrameExtractor`), `setrate` (`setRate(1.0)`), `pausestart` (Sodalite#104 round 4: `pause()` the instant load returns, before any frame exists, and `play()` at t=8; the shape of a host that holds a fresh load paused, and a software session used to answer it with eight ticks of `enq=+0 status=unknown r4d=n`, a black picture under a paused clock; now the `[SWHost] #104` lines show the first frame presented at a stopped clock and `startup 8/8 presenting` arrives while paused), `ratehold` (set 1.5, pause at tick 3, resume at tick 5, then read the rate back off the transport itself: the #436 drill, and it fails the run if the resume came back at 1.0), `reloadlive` (reload the URL on the live path when the probe flags it live, the AetherPlayer Open URL flow), `reloadnext` (a host's episode seam: the same URL loaded again at t=4 on the still-playing engine with no `stop()` in between, so the native host is reused, and every published `state` printed as `STATE t=`; the second load must go `loading`, `playing` with no `paused` between them, which it did not while the reused host still carried the outgoing item's `.playing` status and every host raised its transport over the next episode), `seekback` (rewind 20 s into the DVR window at t=15, return to the live edge at t=30), `overlapseek` (the #292 seek-window drills below), `pausehold` (Sodalite#104: pause at t=10 and HOLD until ten seconds before the end, printing the playhead, the edge and the resident depth every second, which is how a session paused for longer than its own DVR window is measured without waiting out a real one: pair it with `--dvr-window 30` and `--seconds 90` and the ninety minute question becomes a ninety second run), `still` (#544: asks for a scrub still at three aims, 20 s behind the playhead at t=15, 5 s behind at t=20 and at the edge at t=25, writing each to `aetherctl-still-.png` in aetherctl's private per-user temporary directory (the path is printed) and reporting hit or MISS with the decode time; pair it with `--sw --dvr-window N`, where the picture comes out of the DVR packet ring rather than a segment cache, and read the FILE as well as the count, because the bundled seed burns its own second into the frame so a still asked for 14.85 s showing `14` is the verdict that it decoded the right moment and not merely an image; exit 6 when nothing hit, 5 when the session ended before the first aim; on a VOD session (AE#605) the three aims are 10.5 s behind the playhead, 4.5 s ahead of it and 600 s past it, through `scrubThumbnail`, and the third is EXPECTED to miss, because a frame the cache does not hold yet must not be answered with the one before it: `--sw` against an HTTP source is the software packet-cache case, and the testsrc fixture's counter in the PNG is the verdict that the still decoded to the target rather than snapping to its keyframe), `stallclock` (AE#549: stop the master clock at t=4 behind the host's back, the way an interrupted audio session does, then call a plain `play()` at t=7 and read whether the playhead moves again; the interruption itself cannot be staged on macOS, its outcome can, and without the `RendererClockResume` branch the run ends with the clock standing exactly where the stall left it, which is the field log's shape; the drill drives DEBUG-only engine hooks, so a Release `aetherctl` prints a notice and ends inconclusive, build it with `swift build --product aetherctl`), `pausereload`, `playreload` and `extplayreload` (#623: force the #93/#65 stage-2 item reload at t=10, on a session paused at t=6, on one left playing, and on one paused through the engine at t=6 and resumed at t=8 straight on the `AVPlayer` the way AVKit's transport does, which leaves the engine's intent reading paused while the player runs; the run fails with exit 4 if the clock moves after the paused reload or stands still after either playing one, and before #623 `pausereload` ended `+7.20 s, state=playing`), and `pauseseek` (pause at t=12, seek at t=15 while paused, resume at t=20; with `--sw` the five paused ticks between landing and resume show what the `[SWDiag]` line reports while the pump is parked and has not heard of the seek, the AE#479 shape); this is how the pre-arming `setRate` wedge was isolated. + +`--served-url` prints `SERVED ` whenever the session mounts a native item, with the loopback path token in clear. On the remote route with no loopback stand-in the item plays the origin URL itself, which is redacted like any other line. Every other line redacts the token, which is right for a log and leaves no way to look at what was actually served. With the URL, `init.mp4` is one `curl` away, and the fMP4 segments are already on disk under `$TMPDIR/aether-segments//seg-N-M.m4s`: `cat init.mp4 seg-26-27.m4s > s.mp4` is a file `ffprobe -show_packets` reads, which is how AE#684 measured the sound against the picture in the bytes rather than in the log. Fetch the init segment only: a second client asking for the playlist or for segments is a consumer the session then accounts for. `--seek-every N` seeks once every N ticks past tick 10, walking `--seek-pattern ` if one is given (a short backward hop otherwise), and `--seek-count K` stops after K seeks so a run can be a BURST and then play. Both halves are needed for anything about what a seek sequence leaves behind: the burst puts the store in the state under test, and only the playing half shows what the overlay carries through it. That pairing is what made AE#362's second mechanism reproducible (a hole between a restarted pump and the island the previous run left ahead of it, decoded across and then never re-read). @@ -150,7 +155,7 @@ load and `playing` at the roll (AE#440). `--live-ingest` loads the URL through `HLSLiveIngestReader` as a custom source, which is the shape a host uses for a live channel it ingests and re-serves itself (Sodalite's direct live path). Pair it with `--live`. It reaches the reader DIRECTLY, which is what a repro of the reader itself needs; since AE#363 plain `--live` also ends up there, but by way of the engine's own route (the raw live path detects the playlist and hands it to the ingest), so use `--live-ingest` when the reader is the subject and plain `--live` when the routing is. `hlslive` only serves local `.ts` files. AE#359 (the master's SUBTITLES renditions were parsed away) survived precisely because this path had no harness; `--live-ingest --subs ` reproduces and verifies it in 40 s against a public broadcaster URL. -`--fast-zap` sets `LoadOptions.liveJoinProfile = .fastZap` for the load. `live` has carried the flag for its own raw-TS fixture since AE#195, but that fixture has no upstream playlist, and the served `#EXT-X-TARGETDURATION` is floored by the UPSTREAM's observed arrival cadence (`LiveCadencePolicy`), which is what sizes the holdback the first serve waits for. So fastZap against an origin of one's own, the shape a downstream player actually ships, could not be driven from here at all. Measured on the same 1 s-GOP seed, `--preroll 0 --realtime`: raw TS with no playlist serves at 1.325 s on TARGETDURATION 1 (holdback 3 s, full cushion), the same content behind an `hlsfixture` origin cutting 2 s segments serves on TARGETDURATION **2** (holdback 6 s) although the engine re-cut it at 1 s. Pair it with `--live`, and read the first-serve line (AE#374) rather than a first-frame stopwatch. +`--fast-zap` sets `LoadOptions.liveJoinProfile = .fastZap` for the load. `live` has carried the flag for its own raw-TS fixture since AE#195, but that fixture has no upstream playlist, and the served `#EXT-X-TARGETDURATION` is floored by the UPSTREAM's observed arrival cadence (`LiveCadencePolicy`), which is what sizes the holdback the first serve waits for. So fastZap against an origin of one's own, the shape a downstream player actually ships, could not be driven from here at all. Measured on the same 1 s-GOP seed, `--preroll 0 --realtime`: raw TS with no playlist serves at 1.325 s on TARGETDURATION 1 (holdback 3 s, full cushion; since AE#670 the engine's own GOP-bound cut carries 1.5x headroom, so that row now seals at 2), the same content behind an `hlsfixture` origin cutting 2 s segments serves on TARGETDURATION **2** (holdback 6 s) although the engine re-cut it at 1 s. Pair it with `--live`, and read the first-serve line (AE#374) rather than a first-frame stopwatch. `--live-start-immediately` / `--no-live-start-immediately` set `LoadOptions.liveJoinStartsImmediately`, which cuts AVPlayer's stall-avoidance hold short once at the live join (AE#440). It is **on by default** @@ -228,6 +233,16 @@ rebuild window, where `videoRoute` is `.none` because there is no route to ask: cutting with the value, and now answers `set while the session is being rebuilt; the load in flight reads it from the options and delivers it`. +Round 5 is two presses a re-anchor apart rather than inside one (`--switch-audio-delay 50@15000 +--switch-audio-delay 100@15060`, the band is 50 to 90 ms on an M1 against a 300 s H.264 + AAC fixture). +The second press arrives after the first rebuild has returned and written `.playing` but before the new +host has published a position, so before the fix it read the zero `load()` had written and rebuilt at +the head: `#3 mount seek: item axis 0.00s`, `cutting seg0+`, on every run from 15050 to 15090. After +it, `#3 mount seek: item axis 14.90s` and `cutting seg3+` on every run. The control is +`--start-position 100 --seek-every 12 --seek-count 1 --seek-pattern 0 --switch-audio-delay 50@12450`: +a seek retires the parked position, so the rebuild after a genuine seek to 0 still mounts at `0.00s` +and not at the 100 s the load was handed. + `play --live` without `--dvr-window` is the live-only shape, and it is the one that shows the re-anchor gate: `AE#464: audio delay = +150 ms stands, but this session cannot re-anchor at the playhead (state=playing, live=true); it arrives at the next seam`. With `--dvr-window 1800` the same run takes the @@ -236,7 +251,9 @@ live-only session took a rebuild that rejoins at the edge. `--reload-applying =` (repeatable, with one shared `--reload-applying-at `, default +20 s) corrects a `LoadOption` on the playing session through `reloadAtCurrentPosition(applying:)` (#460). Keys: `header.`, `audio-bridge`, `preferred-audio`, `decode-path`, `is-live`, which is there to drive the refusal, since a field that names the session has to be observably refused rather than observably ignored, and `autoplay`, which drives the third answer: a field the session owns, neither refused nor applied. Both outcomes print, which is the pair a host's recovery ladder has to tell apart. Pair it with a header-logging origin to read the correction from the other end: with `--header "X-Auth: stale"` at load and `--reload-applying header.X-Auth=fresh`, the origin log shows three requests carrying the stale value, then three carrying the fresh one, and the transport telemetry carries straight through the rebuild (`resumed at 10.90s from 9.90s`). -`play --sw` sets `LoadOptions.preferredDecodePath = .software` (#461), the shipping per-session lever, rather than the process-global `setForceSoftwarePathForTesting` it drove before; that hook is still what `live --sw` and `dvr` use, since those harnesses run several sessions and want every one of them on the software host. `--reload-applying decode-path=software` is the same lever applied to a session that is already playing: on the 300 s H.264 fixture the run dispatches `codec=27 → native`, takes the correction at t=9.90 s and comes back `codec=27 → software` at 10.81 s, playing. On a live load the override reaches the same routing decision, which is the case with no alternative, since the #2 capability gate is VOD-only and a live session is never classified at all. +`play --sw` sets `LoadOptions.preferredDecodePath = .software` (#461), the shipping per-session lever, rather than the process-global `setForceSoftwarePathForTesting` it drove before; that hook is still what `live --sw` and `dvr` use, since those harnesses run several sessions and want every one of them on the software host. `--reload-applying decode-path=software` is the same lever applied to a session that is already playing: on the 300 s H.264 fixture the run dispatches `codec=27 → native`, takes the correction at t=9.90 s and comes back `codec=27 → software` at 10.81 s, playing. On a live load the override reaches the same routing decision, which is the case with no alternative, since the #2 capability gate is VOD-only for H.264 / HEVC and such a live session is never classified at all. + +`play --no-sw-escalation` sets `LoadOptions.escalatesToSoftwarePath = false` (AE#629), and every `play` run prints an `ESCALATION at duringStartup= absorbed=/ ` line when the AE#561 rebuild is taken. The pair is the A/B for a host's own fallback ladder. On a 40 s HEVC fixture from `Scripts/nal-overrun-fixture.py --after 20`, played with `AETHER_DISABLE_NAL_SANITIZER=1`, the default arm prints `ESCALATION at 12.00s duringStartup=false absorbed=CoreMediaErrorDomain/-19602 nativeItemFailed`, rebuilds on the software path and plays to the end (`VERDICT: OK`). The declining arm prints no escalation, goes to `.error` with `-19602` at 12.00 s and exits 2. `--drop-audio` forces every audio pipeline to fail, so the AE#462 video-only drop is observable without a source this build has no decoder for: the loopback cascade skips both the stream-copy probe and the bridge, and `SoftwarePlaybackHost` refuses its decoder open. The run then prints `audio delivery=droppedNoPipeline pipeline=none`, which is the pair a host reads (the typed fact plus the human label), against `delivery=streamCopy` / `bridged` / `decoded` on the same source without the flag. It is loud in the log on purpose, in both the CLI line and the engine's own, because a forced classification read as a real one would be worse than no harness. @@ -257,6 +274,10 @@ point is that there is no second ask to refuse. Two shapes are worth running del they are the two the flag has to survive: a long uninterrupted read, and a PAUSE, which must end the connection after five seconds and cost exactly one re-request at the frontier when playback resumes. +`--max-concurrent-requests N` sets `LoadOptions.maxConcurrentSourceRequests` (#377): the most requests the reader may have open against the origin at once, across every path it fetches on. `1` also switches off the speculative parallel paths, which is the shape of a connection-metered CDN. Count the requests in the origin's own log, or read the `[AVIOReader]` connection lines, with and without the flag. + +`--assert-dv` sets `LoadOptions.panelPresentsDolbyVision` (AE#493), the host's assertion that its display presents Dolby Vision. macOS has no per-mode display capability API, so a Mac run plays a Dolby Vision source as its HDR10 base layer (`effective-format=hdr10`) until the host says otherwise; the flag moves that label and the tvOS criteria request. It does not change the packaging of a Profile 5 / 8.1 / 8.4 source (those carry their `dvcC` and `SUPPLEMENTAL-CODECS` on every display since 6.72.0 / 6.73.0), while Profile 7 and AV1 Dolby Vision are still gated on it. A wrong claim costs one in-place media-playlist fallback, not the item. + `--picture-probe` attaches an `AVPlayerItemVideoOutput` to the running item and decodes the source time out of the picture itself, which is the one axis question nothing else here can answer: every other observable (`#260` frame times, `prodShift` / `hostShift`) describes what the engine WROTE, not @@ -420,11 +441,11 @@ A healthy run prints `contiguous` on every line. AE#561 is the counter-example i ## dovitest -Runs the Dolby Vision Profile 7 to 8.1 converter over every video packet of the source and writes the converted elementary stream (Annex B) to `/tmp/aetherctl-dovitest.hevc`, reporting packets processed, conversions, and failures. Lets you confirm the in-engine `DoviRpuConverter` (libdovi) output matches the `dovi_tool -m 2` ground truth offline, without a DV panel: +Runs the Dolby Vision Profile 7 to 8.1 converter over every video packet of the source and writes the converted elementary stream (Annex B) to `aetherctl-dovitest.hevc` in aetherctl's private per-user temporary directory (`$TMPDIR/aetherctl-`, mode 0700, the same directory on every run, so the next run overwrites the file instead of leaving a second copy of the film; the path is printed), or to the path given with `--out`, reporting packets processed, conversions, and failures. Lets you confirm the in-engine `DoviRpuConverter` (libdovi) output matches the `dovi_tool -m 2` ground truth offline, without a DV panel: ```bash -swift run aetherctl dovitest -dovi_tool extract-rpu -i /tmp/aetherctl-dovitest.hevc -o out.rpu +swift run aetherctl dovitest [--out PATH.hevc] +dovi_tool extract-rpu -i -o out.rpu dovi_tool info -i out.rpu -f 0 # expect dovi_profile 8, disable_residual_flag true ``` @@ -444,7 +465,7 @@ The profile differential is the diagnostic: a `video_delay=0` plus NOPTS or non- ## extract -Opens a `FrameExtractor` against the source and pulls a still frame. Thumbnail mode (default) snaps to the nearest keyframe and downscales to `--width` (default 320); `--snapshot` decodes frame-accurately at full resolution. `--at ` sets the seek position (default 60.0). The first frame is written to `/tmp/aetherctl-extract-.png`. `--loops N` repeats the extraction across eight cycling positions, which pairs with `leaks --atExit` to validate the decode-context teardown is clean: +Opens a `FrameExtractor` against the source and pulls a still frame. Thumbnail mode (default) snaps to the nearest keyframe and downscales to `--width` (default 320); `--snapshot` decodes frame-accurately at full resolution. `--at ` sets the seek position (default 60.0). The first frame is written to `aetherctl-extract-.png` in aetherctl's private per-user temporary directory (the path is printed). `--loops N` repeats the extraction across eight cycling positions, which pairs with `leaks --atExit` to validate the decode-context teardown is clean: ```bash swift run aetherctl extract --at 612 --snapshot # frame-accurate still @@ -460,7 +481,7 @@ Plays a source through the audio-only pipeline (default ten seconds, `--seconds aetherctl audiotap [--duration S] [--out PATH.wav] [--remote | --software] -Brings up the loopback session headless, decodes the audio tap (#95) as fast as segments are produced, writes mono Float32 48 kHz WAV (default `/tmp/audiotap.wav`), and prints buffer count, PCM seconds, discontinuity count, and the covered `sourceTime` span. A clean run reports exactly one discontinuity (the install itself). `--remote` drives the remote-HLS delivery path instead (direct AVPlayer ingest of an HLS url, no loopback): rendition/variant resolution, segment fetch + decrypt, playhead-follow decode. Verification tool for the PCM audio tap across the stream-copy and bridge audio paths. +Brings up the loopback session headless, decodes the audio tap (#95) as fast as segments are produced, writes mono Float32 48 kHz WAV (default `audiotap.wav` in aetherctl's private per-user temporary directory; `--out` sets the path), and prints buffer count, PCM seconds, discontinuity count, and the covered `sourceTime` span. A clean run reports exactly one discontinuity (the install itself). `--remote` drives the remote-HLS delivery path instead (direct AVPlayer ingest of an HLS url, no loopback): rendition/variant resolution, segment fetch + decrypt, playhead-follow decode. Verification tool for the PCM audio tap across the stream-copy and bridge audio paths. `--software` drives the third delivery path, the SW sink (`AudioTapPCMConverter`), which the other two modes cannot reach: they drive their readers directly, while the sink only exists inside a real session. This mode therefore loads the source through the whole engine, fails if it did not route to the software host, installs the tap through the public `installAudioTap()` and plays, so the sink runs exactly as it does in a host. It is bound to wall clock (the SW host decodes in real time), and it reports `peak` next to the buffer count because the two ways this path fails look identical in a report otherwise: **exit 3 covers both no buffers at all and buffers of digital silence**, which at a consumer is indistinguishable from a muted source. That gap is not hypothetical. With no harness here, a force unwrap that trapped on the FIRST buffer of any multichannel track shipped in 6.1.3 and survived to main (#400), and the silent-downmix defect underneath it only became visible once the trap was gone. Software routing needs a source the native path declines, e.g. `ffmpeg -f lavfi -i testsrc2 -f lavfi -i sine -c:v libvpx-vp9 -c:a aac -shortest clip.mkv`; add `-af "pan=5.1|c0=c0|c1=c0|c2=c0|c3=c0|c4=c0|c5=c0"` for the multichannel case and `-af "pan=quad|c0=c0|c1=c0|c2=c0|c3=c0"` for the layout AVAudioConverter refuses to mix. @@ -619,6 +640,8 @@ Activates two subtitle tracks simultaneously on one source (primary + secondary) Runs a live MPEG-TS session against a built-in fixture that serves an endless broadcast by looping a seed `.ts` with rewritten timestamps. Flags simulate the failure modes the live path hardens against: `--drop-after N` (mid-stream connection drop + reconnect), `--discontinuity-at N` (program-boundary PTS / PCR jump), `--realtime` (1x wall-clock pacing), `--preroll N` (backlog seconds the paced fixture bursts before 1x pacing; default 30, `0` models a strict-realtime origin with no backlog), `--origin-lead N` (how far ahead of the wall clock the paced origin is allowed to run, which is the standing distance a raw live client ends up reading behind it: the built-in 2 s sits exactly on the old live-edge tolerance and hides both sides of it, `--origin-lead 6` joins a session six seconds behind the reader's frontier and its post-rejoin distance then sawtooths 1.4 to 2.6 s, which is the Sodalite#104 round 2 measurement), `--realtime-rate X` (implies `--realtime`; holds the origin at X times wall clock for the whole run, which is the shape `--realtime` at 1x and an unpaced fixture bracket without covering: a restreamer serving from a standing buffer keeps handing over faster than the content happens, so the live edge outruns the client tracking it for as long as the session lasts rather than for the length of one burst), `--fast-zap` (loads with `LoadOptions.liveJoinProfile = .fastZap`; the first serve prefers the full holdback but is bounded after two finalized segments plus a 0.5...2.0 s observed-segment grace), `--dvr-window N` (timeshift), `--measure-rss` (sliding-window retention), `--reload-test` (live rejoin end to end, including the full-backlog replay shape some origins serve on reconnect). `--seed ` overrides the seed clip, `--sw` forces the software live path, `--report-cache-bytes` tracks on-disk DVR footprint, `--serve-only` parks the fixture without attaching an engine (raw `curl` / `ffprobe` inspection), `--rewind-test` runs the DVR rewind-and-return matrix variant, `--rewind-hold N` parks the playhead N seconds behind the edge and HOLDS it there for the rest of the run (the regime that separates a resident floor doing its job from a window outrunning the reader: it reports the floor-minus-playhead inversion, stalled ticks, and any `live window slid past the consumer` line), `--freeze-after N` freezes the upstream with the connection still open, `--rewind-before-freeze N` parks the playhead inside the DVR window first, `--unfreeze-after N` lets the frozen upstream deliver again after N seconds (the only way to drive the recovery half: a window closed with ENDLIST re-opening, and where the rejoin puts a timeshifted viewer). Since AE#446 round 7 the leg reports two different healthy outcomes for it, and the difference is the whole point of that round: `VERDICT: live-freeze position held` is a window that was closed and rejoined, while `VERDICT: live-freeze gap absorbed` is a gap short enough that the window was never closed at all, so no item was swapped and the viewer saw nothing. The per-tick line carries `item=` (the item's own playhead, which is the session clock less the session's shift, so the two disagree by a constant and only the first is the consumer's place) and `buf=` (how long that consumer can keep playing out of what it already holds). The pair is what makes an outage decision readable: the close spends a DEPTH, and the segments the window lists above the consumer's fetch point are only half of one. Since AE#523 round 2 what decides between them is the CONTENT: a freeze drives the second for as long as the consumer still holds more than one target duration of runway ahead of its own fetch point, whatever the clock says, and the clock keeps only the ceiling at which the producer gives a silent source up (`35 s` less a patience, so 26.0 s at the fixture's TARGETDURATION 6). With the built-in fixture that is, for example, `--freeze-after 60 --unfreeze-after 22 --rewind-before-freeze 30`, where 24.0 s of runway carries a 20.1 s silence and is absorbed, against a close and an item swap before that round. Drop the rewind to 0 and the same 12 s freeze is absorbed too since AE#520 round 2, because the edge viewer's own buffer is counted: 8.0 s listed plus 3.9 s in hand against a 6.0 s reserve, where the listed half alone closed the window 1.84 s before the source delivered again. A 30 s freeze at the same depth still closes, at 16.07 s of silence with 6.0 s of depth left and nothing at all still to fetch, which is the case the old gate could not close at all, `--live-only` loads with no DVR window at all (the shape of a client that keeps its rewind outside the engine, which is where AE#446 round 4 came from: the freeze leg then measures the only timeshift such a session can have, the backlog an outage puts between the closed window's end and the source's return, and the sliding 60 s live-only retention makes the fresh item's own axis observable), `--force-recovery-reload-at N` drives the stage-2 recovery reload without waiting for a real item death, and `--gen-highbitrate-seed` generates a ~22 Mbps 1080p H.264 MPEG-TS seed (for RSS-retention measurement) then exits. `--start-position S` loads the live session with a resume anchor, the same one `load(url:startPosition:)` takes. Live callers normally pass nil, so what the anchor does on a live join had never been drivable from here at all. Measured on a seed offset to 95173 s: the mount seek is spent on the ITEM axis while a host only ever sees the published clock (item + shift), and AVPlayer CLAMPS an anchor past the end of a 15 s item and joins normally (`item=0.74s`, playing), so an anchor on the wrong axis is not by itself a wedge. +`--no-blocking-reload` sets `LoadOptions.liveBlockingReload = false` (AE#446): the live playlist never advertises `CAN-BLOCK-RELOAD`, which separates "AVPlayer stopped fetching because the playlist stopped changing" from "AVPlayer stopped fetching because it is in low-latency mode". + `--sliding` is still accepted and does nothing: the sliding window is unconditional for live sessions now, and the flag stays only so an older script does not fail on it. The 1 Hz tick prints `item= ranges= status=` beside `t=` and `edge=` (AE#509). `t=` is the PUBLISHED clock, which carries `playlistShiftSeconds`; `item=` is `AVPlayerItem.currentTime()` on the item axis, which is the field a host's own diagnostic dumps. On a live source whose axis is hours into an encoder clock the two differ by the whole shift in a perfectly healthy session (`t=95173.70s item=0.75s ranges=1 status=1`), so a report quoting one of them cannot be read against a harness printing the other. `ranges=0` is the discriminating fact behind a live join that fetches a whole window and presents none of it: it says nothing has been PLACED, which `isPlaybackBufferEmpty` cannot say (AE#418, a fetch is not a placement). @@ -698,6 +721,85 @@ entered. The per-tick `edge=` and `behind=` are not usable as a check on that: ` its first value for the whole run and `behind` is derived from it, with or without `--dvr-window`. The stall half needs a field capture or an origin that reproduces the seek, not this table. +### An origin that delivers late (AE#684) + +`hlsfixture` publishes on a metronome, and a metronome is the one thing an IPTV origin is not. The defect AE#684 reported only exists when a delivery arrives later than the segment is long, so it could be read off a device log and reproduced nowhere. `Scripts/hls-burst-origin.py` serves a sliding playlist over pre-cut segments and publishes segment k at `end of k - prefill + delays[k % n]`: the MEAN cadence stays the segment duration (the origin never falls behind real time) while the gap between two deliveries follows the cycle. `--freeze-at K --freeze-seconds F` is an outage with the backlog landing at once, `--durs 6,4` alternates segment lengths, `--dur 6.3` over 6 s of media is a playlist that rounds its EXTINF up, `--window` / `--prefill` set how many segments the upstream lists, and `--rate-kbps` / `--latency-ms` are a link shared by every in-flight response. Its docstring carries the fixture command: 720x576 at 25 fps, 2 s GOPs, 6 s segments, and a white frame plus a 40 ms beep on every whole second, which is what makes the sound measurable against the picture afterwards. + +```bash +Scripts/hls-burst-origin.py --dir seg6 --delays 0,0.3,2.6,0.2,0.6,2.4,0.1,0.5 & +aetherctl play --live --live-ingest --fast-zap --seconds 180 http://127.0.0.1:8684/live.m3u8 +``` + +That cycle makes the origin's own gaps run 3.6 to 8.3 s. On the loopback they arrive 3.0 to 9.3 s apart, because the ingest reloads the upstream playlist every half segment and a delivery that just missed a reload waits for the next: **the served window changes once per UPSTREAM segment, up to one and a half of them apart, however finely the engine re-cuts it.** Under `.fastZap` the cut is the 2 s GOP, `max EXTINF` reads 2.000 s, and through 7.25.1 the upstream's 6 s entered only through the cadence floor, as `ceil(6 / 1.5)` = 4. That division is right for a MEASURED gap, which is a robust maximum; a segment duration is the other end of the distribution, the period the source delivers at when nothing is late. So the client's patience (`1.5 x 4` = 6.0 s) was exactly one period, and the 12 s holdback left 2.7 s at the low point of the sawtooth. + +What that costs is not the `-12888` itself but what AVPlayer does next: it skips a reload. Measured here and on an Apple TV alike, polls every 2.00 s, then 4.00 s (macOS) or 5.03 s (tvOS) of nothing after the error, and the stall lands inside that hole with the next delivery already listed 1.9 to 2.6 s earlier. + +The seal now asks for the upstream segment whole and takes as much of it as the join pays for. On an upstream of uniform segment length the join itself is not deepened (one more upstream segment would be one more download before the first picture on every zap, which AE#678 measured and declined), so three 6 s segments are 18 s joined, 16 s cut, and a seal of 5: + +``` +[HLSVideoEngine] live TARGETDURATION sealed at 5s (holdback 15.000s): max EXTINF 2.000s, + 1.5 x cut target 0.750s, measured floor 6.000s needs 4s of patience, + upstream segment 6.000s (one delivery, however finely it is cut here), of which the join + pays 5s of 6s (16.000s cut of the 18.000s it listed); upstream advertises 6.000s (reported, not used) +``` + +One 180 s run per arm on that origin, same deliveries (four gaps above 9.0 s in each): + +| | sealed | join | `-12888` | `playbackStalled` | `#524` running thin | +|---|---|---|---|---|---| +| 7.25.1 | TD 4, holdback 12 s | 3 segments | 5 | 1 | 4 | +| join-bound seal | TD 5, holdback 15 s | 3 segments | 3 | 0 | 0 | + +That is fewer, not none, and it is the honest size of what a seal one second higher buys: 7.5 s of patience still loses to a 9 s gap, but the 15 s holdback leaves 5.7 s at the low point where 12 s left 2.7 s, which is the difference between surviving AVPlayer's skipped reload and not. A join one segment deeper (TD 6, holdback 18 s) drew none of either in three such runs and was not shipped: it cost 3.81 s against 4.98 s to the first picture behind an 8 Mbit/s link. The device capture this came from had 10 of its 110 gaps above 9 s and 5 above 12 s, so the field expectation is fewer `-12888` and far fewer stalls, not zero. + +**`--durs 6,4` is the reporting channel, and its two phases have to read alike.** The join counts seconds back from the newest segment, so `--prefill 8` (a 4 s segment newest) joined four segments and sealed 6 while `--prefill 7` (a 6 s one newest) met the 16 s coverage at three, cut 14 s and sealed 4, 7.25.1's value, on about four tunes in ten. The join is now equalised over the phases of one listing: a tune never loads more than the deepest tune of the same channel already did (the deepest the coverage rule takes over the tune and the tunes one, two, ... segments earlier that its own join still reaches, at most one above its own depth). `--window 8`, mild jitter (`--delays 0.2,0.5,0.3,0.8,0.25,0.6`), three 60 s runs per arm and phase: + +| `--durs` | phase | | sealed | join | `-12888` | first picture behind 8 Mbit/s (median) | +|---|---|---|---|---|---|---| +| 6,4 | `--prefill 7` | 7.25.1 | TD 4, holdback 12 s | 3 segments, 16 s | 1 / 0 / 1 | 3.47 s | +| 6,4 | `--prefill 7` | now | TD 6, holdback 18 s | 4 segments, 20 s | 0 / 0 / 0 | 4.23 s | +| 6,4 | `--prefill 8` | 7.25.1 | TD 4, holdback 12 s | 4 segments, 20 s | 1 / 1 / 0 | 4.27 s | +| 6,4 | `--prefill 8` | now | TD 6, holdback 18 s | 4 segments, 20 s | 0 / 0 / 0 | 4.26 s | +| 10,8 | both | 7.25.1 | TD 7, holdback 21 s | 3 segments, 28 or 26 s | 0 / 0 / 0, 1 / 1 / 1 | 5.86 s, 5.52 s | +| 10,8 | both | now | TD 8, holdback 24 s | 3 segments, 28 or 26 s | 0 / 0 / 0, 1 / 1 / 1 | 5.88 s, 5.59 s | +| 6,6,4 | all three | 7.25.1 | TD 4, holdback 12 s | 3 segments, 16 s | 0 to 2 per run | 3.45 s, 3.43 s, 3.46 s | +| 6,6,4 | all three | now | TD 4, holdback 12 s | 3 segments, 16 s | 0 to 1 per run | 3.47 s, 3.47 s, 3.48 s | + +No run stalled in either arm. The extra segment on 6,4 is the short one, and the other phase already loads it. The equalising stops there on purpose: 10,8 and 6,6,4 join three in every phase, so nothing is added and their first picture is main's; 10,8 seals what its three segments pay (8), and 6,6,4 cuts 14 s in every phase and keeps 4, which is to say it is not helped. A first version took one segment more on any mixed-length join that could not pay its seal, and that cost those two shapes a segment on every tune (6.19 s to 8.12 s and 3.81 s to 4.98 s). Uniform upstreams keep their three segments (`--dur 6`: TD 5, `--dur 10`: TD 9), and a demuxed audio rendition is joined by the same rule (6,4 behind a master: 4 + 4 in both phases). One start does get slower: a host that fetches fewer segments at a time than the join has (`play --max-concurrent-requests 1 ... --rate-kbps 14000`) used to be served at 12 s of cut content and now waits for the join to be handed over or for the grace, 2.79 to 3.08 s before and 3.27 to 3.28 s after with `--prefill 8`, 2.80 to 2.83 s and 3.44 to 3.49 s with `--prefill 7`. + +**Whether a join is spent is a fact, not arithmetic.** The seal may only be taken short once the join has nothing more to give, and "cut content plus one segment reaches the summed EXTINF" is not that fact: `--dur 6.3` over 6 s of media never adds up, and a first version built on the sum sealed the full value there and took the bounded start (2.23 s to first picture against 0.20 s). The reader states it instead: the whole join batch is committed, its FIFO is empty, and the cutter is parked waiting for the next delivery. Three runs per arm on each inflated shape with three segments listed: served at once in all six (0.000 to 0.034 s of hold), TD 5 on 6 s media and TD 9 on 10 s. Two more shapes belong to the same fact. A demuxed audio rendition (run the origin twice, video-only and audio-only segments behind a master) makes the cutter park on the AUDIO reader when the audio ends a little before the video, so the fact is stated for the cutter and not for the video reader: both join batches committed and either reader dry. Read off the video reader alone it never became true, 2.2 s to first picture under `.fastZap` and 10.4 s under `.standard`; now 0.18 to 0.21 s in all six cells (6 s and 10 s `.fastZap`, 10 s `.standard`, audio aligned and 0.2 s short, three runs each), as on 7.25.1. And a join slower than the `.fastZap` grace (`play --max-concurrent-requests 1` behind `--rate-kbps 4000`) is served by the bounded start before it is spent, seals the full 6 or 10 over a window under its holdback and draws a `-16832` warning at the start: first picture 5.05 to 6.08 s against 7.05 to 8.26 s on 7.25.1, which still pays the second grace there, and no `-12888` against 1 to 3. + +**The gate held the second playlist request too.** AVPlayer opens with two `/media.m3u8` requests, each re-enters `waitForFirstLiveSegment`, and on a bounded start each waited its own grace (`GET /media.m3u8` at 2.81 s, `GET /init.mp4` at 4.83 s). On an ingest the gate now opens for good once a manifest has gone out. On a source the engine cuts itself it did not until 7.28.2 (next paragraph): that second grace is the "+2.07 s at the picture" of the AE#594 table above, and removing it there also leaves the session up to 2 s closer to the producing edge, which is that issue's open question. The raw-TS control (`live --fast-zap --realtime --preroll 0`, two seeds, two runs per arm) reads the same in both arms: first manifest at 3.84 to 3.96 s, `init.mp4` at 6.11 to 6.33 s, first picture at 6.20 to 6.58 s, playhead 6.3 to 6.8 s behind the wall clock. The AE#594 table stands as printed. + +**AE#686 applies the same latch to a source the engine cuts itself, by default since 7.28.2.** It started as a measurement arm (`AETHER_FIRST_SERVE_LATCH_ALL=1`, 7.26.3), because what it may cost (a session closer to the producing edge for its whole life) needs a device that seeks to edge-minus-holdback, which this harness does not. The reporter's Apple TV A/B answered that: on 1080p59.94 raw TS with 1.001 s segments, every bounded start reached its picture 1.003 to 1.039 s sooner with the latch, the session sat 0.5 to 1.5 s nearer the edge after 60 s, and 18 latched launches logged no stall, `-16832` or `-12888`. A rebuild from a backlog served a full cushion in both arms, so the latch had nothing to skip there. `AETHER_FIRST_SERVE_LATCH_ALL=0` restores the old gate, read once per process. Both arms print how long AVPlayer's second plain request waited, once per session: + +``` +[HLSVideoEngine] repeat live manifest request held 2.003s, fastZap bounded start after 2.000s grace +[HLSVideoEngine] repeat live manifest request held 0.000s, first-serve latch +``` + +`live --fast-zap --realtime --preroll 0 --seconds 40`, two runs per arm, interleaved: + +| seed | arm | first manifest | second request held | `GET /init.mp4` | sustained advance | `item=` at the end | stalls | +|---|---|---|---|---|---|---|---| +| `h264-ts-sample.ts` (2 s cuts) | off | 3.76 to 3.94 s, 2 segments | 2.003 s | 6.10 to 6.14 s | 7.4 to 7.7 s | 34.78 to 34.95 s | 0 | +| `h264-ts-sample.ts` | on | 3.94 s, 2 segments | 0.000 s | 4.08 s | 5.2 to 5.3 s | 36.57 to 37.20 s | 0 | +| 1080p59.94, 1 s GOP | off | 1.22 to 1.23 s, 3 segments | 1.011 s | 2.43 s | 3.2 s | 38.30 to 38.40 s | 0 | +| 1080p59.94, 1 s GOP | on | 1.24 s, 3 segments | 0.000 s | 1.41 to 1.44 s | 1.1 s | 39.19 to 39.30 s | 0 | + +The latch removes exactly the grace, and the session ends that much closer to the edge; on the device that cost no stall. + +The full matrix (6 s and 10 s segments, 3, 4 and 8 listed, both profiles, three runs per arm) is in `api.md` under the live join; `.standard` drops `--fast-zap` from the `play` line. + +**The same fixture measures sound against picture, in the bytes.** `play --served-url` hands over the init segment, the cache holds the rest, and decoding their concatenation gives the presentation time of every flash and every beep onset: 147 of 148 seconds of a served session read **-0.3 ms** (the very first beep reads +19.7 ms, the decoder starting), across a 12 s outage, with no audio frame of any duration but 1024 samples and no gap at any segment seam. A fresh item re-fetches the same cached segments, so a rebuild cannot change that. What a rebuild does change is which segment the item starts on and where in it, and a segment's sound does not begin where its picture does: the cut is taken on the video keyframe and a transport stream interleaves audio behind video, so on this fixture the sound opens between 11 ms after and 208 ms before the picture, segment by segment. Each item now says what it started on, + +``` +[AetherEngine] #684 item #1 starts at its own 4.000s: segment 2 + 0.000s, whose picture + opens at 4.000s and whose sound runs 3.979..5.963s (it opens 21 ms before the picture) +``` + +and each `live seg-N finalized` line carries `sound=first..last`. The PICTURE half of the question is measurable here: the same origin over a 24 fps twin that carries `Scripts/timecode-fixture.sh`'s block row, `play --picture-probe`, and an outage long enough to close the window (`--freeze-at 14 --freeze-seconds 14`). Four runs, each rebuilt with `#EXT-X-START:TIME-OFFSET=41.95,PRECISE=YES` at `segment 34 + 1.95s`, each fetching seg31 first, whose sound opens 208 ms before its picture: `axisErr` reads +12.000 on all 70 ticks of the first item and +40.000 on all 38 of the rebuilt one, whose playlist begins 28 s later. The picture sits on the item clock to the frame on both, so AVPlayer does not anchor an item on the earliest sample of the first segment it loads, and a mid-segment placement moves nothing on the video side. **What AVPlayer's audio renderer makes of it is not observable from a Mac**: an `MTAudioProcessingTap` on an HLS item is created without error and never called (0 callbacks in 12 s), and capturing the output needs a grant no headless run has. A report of sound and picture drifting apart after a rebuild therefore needs a device, this fixture (a beep that should land on a flash is the one sync test a viewer can read without instruments), and the three lines above it in the log: the `#454 placing` line, the `#684 item` line, and the AE#440 line that says whether the start was forced. + ### The header-enforcing origin (AE#363) A tokenized IPTV origin refuses anything that arrives without its per-request header, which is a shape none of the fixtures could produce, so neither live client could be driven against one: @@ -723,7 +825,9 @@ Note that the slicing is byte-based, not keyframe-aligned, so segments start mid Drives a real AVPlayer (native loopback-HLS path) through a burst of rapid seeks and reports the producer-restart coalescing behavior, the longest "wedge" (state `.playing` but the clock frozen), and final settle accuracy (AetherEngine#35). A concurrent sampler probe also checks the seek clock-bounce / `isSeeking` signal (AetherEngine#37 / #38): a single backward seek must not bounce the clock back through the pre-seek position, and `isSeeking` must span the real landing. The run ends with a `#38 SEEK EVENT LEDGER`: every `.began` must reach a terminal event (an unpaired one is a stranded in-flight window), and `.stalled` seeks are listed with any late `.landed` that followed them. The ledger also prints **seek latency** (`.began` to its first terminal event, per seek, with min / median / max), which is what an A/B about the cost of work ON the seek path is read off: quote the median and the max, never a mean, because the tail belongs to the source rather than to the change under test. Note the burst is serialised, so every seek here terminates before the next is issued; to put a seek on top of an unsettled one use `play --seek-every 1 --seek-count N` against an origin below media rate, where `phase=seeking` holds across the run and the ledger shows supersedes. `--seeks N`, `--gap-ms N`, `--settle N` shape the burst; needs `> 30 s` of seekable VOD. `--throttle-kbps N` caps source-IO delivery to simulate a slow CDN and force rebuffers during the burst (see `serve`). -The `#65` block of the results separates two parks that look alike in a log and mean opposite things (AE#528). A park whose `PARK` line reads `stuck=0s` is backpressure behind a consumer that is still declaring fetch targets, which is what a viewer scrubbing backwards through resident content looks like, and the wedge breaker staying quiet there is correct; the run says `PARKED BEHIND A LIVE CONSUMER (not a wedge)`. A wedge is a park whose `stuck=` climbs, and that one still breaks and re-anchors. +The exit code is 1 when a printed `<-- FAIL` line is among the `#37` bounce, the `#38` `isSeeking` window, the `#38` event pair and the unpaired-`began` ledger, or when the `#65` block ends in `PRODUCER WEDGE (UNRECOVERED)`, and the final `VERDICT` line then names them. The restart tallies stay informational: they are for comparing two builds, and a run that only prints them exits 0. + +The `#65` block of the results separates two parks that look alike in a log and mean opposite things (AE#528). A park whose `PARK` line reads `stuck=0s` is backpressure behind a consumer that is still declaring fetch targets, which is what a viewer scrubbing backwards through resident content looks like, and the wedge breaker staying quiet there is correct. `stuck=` counts the seconds since the consumer last declared a target and `idle=` only those in which it did not render either, and the breaker trips on `idle=` (AE#649): a park that reads `(consumer quiet, still playing)` with `stuck=` climbing and `idle=0s` is AVPlayer playing out a deep forward buffer between fetch bursts, which a cellular iPhone does for up to a minute at a time, and it does not break either. When every park of the run logged `idle=0s`, the run says `PARKED BEHIND A LIVE CONSUMER (not a wedge)`. A wedge is a park whose `idle=` climbs, and that one still breaks and re-anchors. ## hlslive @@ -731,7 +835,7 @@ Replays a synthetic SSAI ad-pod feed through the live-direct-play path to repro ## smbtest -Connects to an SMB2/3 share with `SMBConnection` (SMBClient backend), wraps the file in `SMBIOReader`, and runs a sequential-throughput pass plus a random-seek consistency check. macOS-only; needs the optional `AetherEngineSMB` product (`swift build --product aetherctl` pulls it in). Validates the SMB byte source without a device: +Connects to an SMB2/3 share with `SMBConnection` (SMBClient backend), wraps the file in `SMBIOReader`, and runs a sequential-throughput pass plus a random-seek check: the offsets are drawn first, the sequential pass records the 16 bytes at each, and every seeked read must come back whole and equal to those bytes (a failed or short read fails the run). macOS-only; needs the optional `AetherEngineSMB` product (`swift build --product aetherctl` pulls it in). Validates the SMB byte source without a device: ```bash swift run aetherctl smbtest "smb://user:pass@host/share/path/to/file.mkv" --reads 128 diff --git a/docs/formats.md b/docs/formats.md index fae44f750..36587d0b0 100644 --- a/docs/formats.md +++ b/docs/formats.md @@ -26,7 +26,7 @@ Interlaced sources (DVD-rip MPEG-2, SD / HD broadcast H.264) are deinterlaced th ### MP4 without composition offsets -Some writers emit a sample table with no `ctts` while the H.264 bitstream still reorders pictures. +Some writers emit a sample table with no `ctts` while the H.264 or HEVC bitstream still reorders pictures. Every sample then reports `PTS == DTS`, and since the native route stream-copies those timestamps into fMP4, AVPlayer is handed decode order as presentation order: each future reference picture is shown before the B pictures that precede it. Measured through AVFoundation's own decoder on a twin @@ -34,8 +34,8 @@ pair (one encode muxed twice, composition offsets removed from one), 45 of 66 pi time belonging to a different picture, with the content order stepping backwards 30 times (#409). The container lost the information, but the bitstream did not: every slice header carries a picture -order count, which is display order, and libavcodec's H.264 parser reads it without decoding a pixel -and takes MP4's length-prefixed payload directly. `H264CompositionOffsetRepair` samples the head +order count, which is display order, and libavcodec's H.264 and HEVC parsers read it without decoding +a pixel and take MP4's length-prefixed payload directly. `H264CompositionOffsetRepair` samples the head (twelve pictures at most, held rather than re-read, so no rewind and no second fetch) and repairs a confirmed source at the demuxer boundary: @@ -54,10 +54,12 @@ built from index entries and then filled with these packets. Detection is fail-closed and costs a healthy file almost nothing: the first real PTS-DTS offset ends the sample (usually on the first packet, since a reordered file's head sample sits one delay below zero). A source is only repaired when every sampled pair is equal, the decode ladder is uniform, the -picture order regresses, and the ranks it produces are distinct and fill the sampled window. Anything +picture order regresses, and the ranks it produces are distinct and fill the sampled window (or a +decode-order prefix of at least nine pictures fills its own range exactly, which covers a +hierarchical mini-GOP longer than the reorder delay, #699). Anything short of that (variable frame timing, a picture order that does not advance one rank per picture, a sample that starts nowhere it can be anchored) is delivered exactly as the container wrote it. -Reported by @orut34iop. +Reported by @orut34iop; the HEVC case (#699) by @ijuniorfu. ### Matroska with presentation slots in coding order @@ -214,6 +216,22 @@ sets `SourceProbe.carriesHDR10PlusMetadata` independently of the primary format, See [whole-probe limits and cancellation](api.md#whole-probe-limits-and-cancellation) for bounded source reads; absence of confirmation is not proof of absence or a statement about the connected display. +### HDR Vivid (CUVA) dynamic metadata + +HDR Vivid (CUVA T/UWA 005.1) is built to be backward compatible: the base layer is plain HLG or PQ in +BT.2020, and its dynamic metadata is an optional registered T.35 SEI on top (country 0x26, provider +0x0004, oriented code 0x0005). The engine plays that base exactly like any HLG or HDR10 source and the +SEI is stream-copied with the rest of the bitstream; no Apple platform applies it, and a display +without Vivid support shows the static base, which is the format's intended fallback. There is no +host-side tone mapping: the native route has no pixel stage, tvOS gives an app no EDR or panel-peak +reading to map against, and the TV maps the HDR signal itself (#699). + +`probe(url:detecting: .hdrVivid)` reports carriage as `SourceProbe.carriesHDRVividMetadata`, in the +same packet pass and budget as `.hdr10Plus`. libavcodec's CUVA parser is internal, so +`HDRVividMetadataScan` walks the body with the same field widths and only counts a message whose +`system_start_code` is one of the defined 1 to 7, whose fields are all present, and whose bits after +the last field are zero. HEVC only, as in libavcodec. `videoFormat` stays `.hlg` / `.hdr10`. + The label can also be taken back from the item itself, where the platform has no capability table to clamp it against (AE#515). A Dolby Vision source on macOS resolves to `.hdr10`, because `supportsDolbyVision` is unclaimable there without a host assertion, while AVFoundation goes on playing the `dvh1` sample entry the engine served. Measured with the assertion off on a 16" XDR, a Profile 5 and a Profile 8.1 grade of Dolby's reference content both strobe, so the RPU reaches the pixels with no claim set anywhere and the clamp was moving nothing but the label. When the item's sample entry reads `dvh1` / `dvhe` and the probe agrees the source is Dolby Vision, the label is upgraded from `.hdr10` to `.dolbyVision` at `readyToPlay`. It is an upgrade and not a mirror of what AVFoundation parsed, for two reasons that both matter: an `.sdr` label is the clamp being right about a display presenting no HDR at all, and on tvOS and iOS the per-mode table answers the capability question, so the label follows it rather than a sample entry that a Profile 5 master carries on every panel. Profile 8.1 keeps `.hdr10` on macOS: it reports `hvc1` with the DV configuration alongside it, it composes on that display all the same, and nothing in the stack reports that. ## Audio @@ -471,12 +489,16 @@ Decrypted disc images play through the normal decode path via a synthetic seekab Both: no decryption (CSS / AACS retail discs must be ripped decrypted first), no GPL nav libraries, no menus, BD-J, or multi-angle. +A remote `.iso` / `.img` / `.udf` needs an origin that honours HTTP `Range`, because a disc image is a filesystem that only random access can walk. The range probe is judged at the response head: an origin that answers it with a 200 fails the load with an error naming the cause (the body is never downloaded), and a range answered wider than asked is cut where the range ends. + **Title selection.** `engine.discTitles` (`@Published [TitleInfo]`) lists the disc's titles (id, name, duration, chapter count) and `engine.selectedDiscTitle` is the active one; `engine.selectTitle(id:)` switches title, rebuilding the pipeline from the new title's head. The selection survives audio-track switches and background-resume reloads, and a fresh `load` defaults to the main title (an out-of-range id clamps to it). Blu-ray enumerates all playlists; a DVD enumerates its title sets (the VMGI TT_SRPT title list, resolved whole-VTS, with the duration read from each VTS's main program chain; per-cell / episodic splitting is deferred). **Chapters.** `engine.discChapters` (`@Published [ChapterInfo]`) carries the selected title's chapters; `engine.selectChapter(id:)` seeks to one (a thin `seek` wrapper, no pipeline rebuild). For Blu-ray they come from the playlist's PlayListMark entries (entry marks only; link points dropped), each mark's timestamp on its clip's STC offset by the clip's in_time and the cumulative duration of preceding play items. For DVD they come from the main program chain's program map plus the cumulative cell playback times. Chapter starts are title-relative (0-based); `selectChapter` adds the title's content-start base (the native playlist shift, or the software path's container start PTS) so the seek lands on the source-PTS playback axis. **Track languages.** Neither disc format carries a track language in the stream, so a title demuxed on its own reports every audio and subtitle track as undetermined and `preferredAudioLanguages` / `preferredSubtitleLanguages` have nothing to match on. The languages are read out of the disc's own navigation data instead and backfilled onto the tracks by stream id: on Blu-ray from every PlayItem's STN table (the ISO 639-2 codes beside each stream's PID, first declaration winning), on DVD from the VTS IFO audio and subpicture attribute tables, with the title's main program chain naming the substream each attribute is actually carried as (a stream the chain marks absent is dropped, and without a readable chain the attribute's position is used, which is how the great majority of discs are authored). Only an undetermined track is filled in: a language the container really declares stays authoritative. `aetherctl disc-inspect` prints what a disc declares per title, so a disc whose tracks stay undetermined can be told apart from a disc that declares nothing (#527). +**Subpicture streams and the probe.** An MPEG-PS demuxer can meet a new stream at any packet, so libavformat never lets `find_stream_info` stop early on a DVD. With the playback budget that meant 50 MB read on every open, and a subtitle whose first packet comes after it was not a track at all. For a title whose VTS IFO is readable, the engine creates every subpicture stream the IFO declares before the probe. `mpegps` routes packets to an existing stream by id, and the engine joins each subpicture's PES fragments itself, because libavformat's `dvdsub` parser only attaches to streams the demuxer created. Such a title then probes 8 MB / 5 s (#651). Stream indices on such a title start with those subpicture streams. A disc without a readable VTS IFO keeps the full budget. + **Container chapters.** `engine.mediaChapters` (`@Published [ChapterInfo]`) carries the chapters a Matroska or MP4 container declares, read off the probe demuxer at load. It is empty for disc sources, which publish `discChapters` instead, so exactly one of the two is populated. Unlike disc chapters these need no base: a non-disc source plays on the container's own PTS axis on both backends, so `startSeconds` is a timestamp a host hands straight to `seek(to:)`. `selectChapter(id:)` resolves against `discChapters` only and no-ops for a container chapter id. Ids are assigned sequentially in start order, so they stay usable as list indices, and untitled entries are numbered "Chapter N". A chapter's duration runs to the next chapter's start rather than to its declared end, because muxers routinely write `end == start`; the last entry falls back to its declared end, then to the container duration. ## Custom byte sources (`IOReader`)