diff --git a/CHANGELOG.md b/CHANGELOG.md index 7df7ea50d..fe0268bdc 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -17,6 +17,7 @@ the public-API contract. ### Added +- The session log places missing TrueHD Atmos heights or LFE. With `objectAudioRendering` the bridge logs each bed channel's RMS and peak level, the window's timeline position and the active and elevated object counts, after the first 5 s, every 30 s, and when the stream seeks, ends or stops; the engine logs the master's audio `CODECS` and rendition `CHANNELS`; and the audio route line adds `AVAudioSession.renderingMode`, multichannel-content support, each port's channel labels with a height count and its spatial-audio flag, and is also written when the rendering mode or spatial capability changes. Route lines name ports by type and no longer carry `portName`, which for AirPods, Bluetooth and AirPlay is the user's own name for the device. A TrueHD Atmos track loaded with object rendering off says so. The readyToPlay track dump adds the item's `allowedAudioSpatializationFormats`. See [formats.md › TrueHD Atmos (object rendering)](docs/formats.md#truehd-atmos-object-rendering). - `LoadOptions.objectAudioRendering` keeps the height channels and object positioning of TrueHD Atmos by rendering its objects into a speaker bed; it is a lossy conversion, not passthrough of the original stream. With `.apac(SpatialSpeakerLayout)` a TrueHD track FFmpeg marks as Atmos is decoded (beds, objects and their metadata), rendered into the chosen 5.1.2 to 9.1.6 speaker bed and delivered as Apple Positional Audio (`CODECS="apac.31.LL"`), which tvOS sends to an Atmos receiver as Dolby MAT. Lossy (320 kbps per bed channel) in place of the lossless 7.1 channel presentation, so it is opt-in; requires OS 26 and falls back to `audioBridgeMode` otherwise. See [formats.md › TrueHD Atmos (object rendering)](docs/formats.md#truehd-atmos-object-rendering). - `aetherctl serve --atmos-bed ` and `serve --audio-index ` serve a TrueHD Atmos track through that path. - Live HTTPS-to-HTTP redirect tests cover provider refusal, anonymous redirects, static-header stripping and the downgrade credential filter. @@ -36,7 +37,7 @@ the public-API contract. - Sidecar decoding runs on a dispatch worker instead of a cooperative-pool thread, so concurrent authorized decodes cannot starve the async resolvers their requests wait on. - A pause made through the engine while the premature-end recovery re-seeks (AE#287) is kept. The recovery called `play()` after its seek whatever the play intent said, so the title started again. - An external ASS sidecar the host draws itself (`preserveASSMarkup`) on the `nativeRemoteHLS` bypass keeps its plain rendition selected behind a suppressing legible output instead of deselecting it. AE#616 measures `sourceTime` off the selected injected rendition, so the deselect left the clock a styled-ASS overlay draws from on item time, 1 to 8 s ahead of the picture after a seek on a transcode restarted at an earlier keyframe. The hidden rendition is selected only once its whole-program WebVTT has been extracted and playback is running, because AVPlayer fetches and buffers a selected rendition and AE#616 measures only presented lines. Turning subtitles off takes it down immediately. -- TrueHD Atmos rendered to APAC reports a decoder that produces nothing (`.audioBridgeProducedNoOutput` on VOD), as the channel bridges do since AE#641, instead of playing the picture over silence. +- TrueHD Atmos rendered to APAC reports a decoder that produces nothing (`.audioBridgeProducedNoOutput` on VOD), as the channel bridges do since AE#641, instead of playing the picture over silence. Its threshold is 1200 packets, a second of TrueHD, not the channel bridge's 64: after a start or resume mid-stream the object decoder waits for the next major sync, measured at up to 102 access units on Dolby's Unfold demo, and at 64 a resumed session was reported dead and went to `.error` while it played, ignoring every later seek. - Playlist preflight through a refreshable `HTTPRequestAuthorization` keeps the 2 MB playlist cap the static-header preflight has (audit NAT-5), and relayed fetches use upstream's per-fetch pending cap (audit NET-107) in place of the fork's high-water wait. - A 206 whose `Content-Range` total is zero or negative no longer crashes an HTTP source as it opens. `Int64` parses a signed total, so `bytes 0-0/-9223372036854775808` reached `total - 1` with `Int64.min` and trapped. - Below 1x, or after a pause in the last quarter second, the audio-only host parks its clock when it reaches the last sample. It parked after the tail's length in wall time, which cut up to 0.125 s at 0.5x. diff --git a/Sources/AetherEngine/AetherEngine.swift b/Sources/AetherEngine/AetherEngine.swift index 2ed1bb446..42b11b2e1 100644 --- a/Sources/AetherEngine/AetherEngine.swift +++ b/Sources/AetherEngine/AetherEngine.swift @@ -3477,7 +3477,11 @@ public final class AetherEngine: ObservableObject { /// `sessionRebuildResumesPlaying`. See `rebuildResumesPlaying`. var transportIntentUnderReconstruction: Bool? private(set) var audioSelectionTask: Task? - private var pendingAudioSelection: Int? + /// A `selectAudioTrack` pick not yet started, and the one whose rebuild is running. A reload that + /// lands meanwhile restores these before `activeAudioTrackIndex`, which the switch's own + /// `stopInternal` has just cleared (see `liveSelection`). + var pendingAudioSelection: Int? + var audioSwitchInFlight: Int? private var audioSelectionEpoch = UUID() private func cancelPendingAudioSelection() { @@ -3485,6 +3489,9 @@ public final class AetherEngine: ObservableObject { audioSelectionTask?.cancel() audioSelectionTask = nil pendingAudioSelection = nil + // The cancelled switch is no longer anyone's intent: a reload that snapshots the selection + // before the cancelled task unwinds must not restore it. + audioSwitchInFlight = nil } /// AE#464 round 3: true while a re-anchor raised by `setAudioDelay` is running, so the presses @@ -6988,8 +6995,13 @@ public final class AetherEngine: ObservableObject { guard !Task.isCancelled, self.audioSelectionEpoch == epoch else { return } if self.activeAudioTrackIndex == selected { continue } let beforeReload = self.loadGeneration + self.audioSwitchInFlight = selected let failure = await self.reloadWithAudioOverride( url: url, audioStreamIndex: Int32(selected), expectedGeneration: beforeReload) + // A cancelled task returns late: the marker may now belong to a newer switch. + if self.audioSelectionEpoch == epoch, self.audioSwitchInFlight == selected { + self.audioSwitchInFlight = nil + } // This rebuild owns one stopInternal generation. Background teardown or // another SDK recovery cannot lend its successor to queued audio work. guard !Task.isCancelled, self.audioSelectionEpoch == epoch, failure == nil, diff --git a/Sources/AetherEngine/Audio/AudioRouteDescription.swift b/Sources/AetherEngine/Audio/AudioRouteDescription.swift index b1307a20c..b5f77d21d 100644 --- a/Sources/AetherEngine/Audio/AudioRouteDescription.swift +++ b/Sources/AetherEngine/Audio/AudioRouteDescription.swift @@ -1,23 +1,35 @@ import Foundation import AVFoundation +import CoreAudioTypes /// The audio route as one log fragment, shared by every host that writes a route line, so the native and /// the software path describe the same route in the same words and a diff between their logs is a diff /// of routes, not of formats. enum AudioRouteDescription { - /// `output=… preferred=… max=… ports=[name[type, ch=n], …] latency=…ms io=…ms`, or nil where there is - /// no `AVAudioSession`. The output latency is the field AE#395 needs: a buffered AirPlay route delays - /// sound by seconds where HDMI delays it by milliseconds, and a feed that is fine on one is late on - /// the other. + /// `output=… preferred=… max=… rendering=… multichannelContent=… ports=[type[ch=n, labels=…, + /// heights=n], …] latency=…ms io=…ms`, or nil where there is no `AVAudioSession`. The output latency is + /// the field AE#395 needs: a buffered AirPlay route delays sound by seconds where HDMI delays it by + /// milliseconds, and a feed that is fine on one is late on the other. + /// + /// `rendering` and the port's channel labels are what a missing-heights report turns on: the + /// rendering mode is the system's own word for what it does with the audio (`dolbyAtmos` against + /// `surround` or `spatialAudio`), and the labels are the speakers the route reports. Read the two + /// together: a Dolby MAT carrier can report fewer channels than it carries (AE#520), so `heights=0` + /// next to `rendering=dolbyAtmos` proves nothing on its own, while `heights=0` next to `surround` + /// is a route with no height channels at all. + /// + /// Ports are named by type, never by `portName`: for AirPods, Bluetooth and AirPlay that is + /// whatever the user called the device, and an HDMI sink's name can be a renamed CEC name. The + /// line reaches the host's handler and from there diagnostics reports, and no host-side scrub of + /// a free-text name inside it can be made reliable. static func current() -> String? { #if os(iOS) || os(tvOS) let session = AVAudioSession.sharedInstance() - let ports = session.currentRoute.outputs.map { port in - "\(port.portName)[\(port.portType.rawValue), ch=\(port.channels?.count ?? -1)]" - }.joined(separator: ", ") + let ports = session.currentRoute.outputs.map(portDescription).joined(separator: ", ") return "output=\(session.outputNumberOfChannels) preferred=\(session.preferredOutputNumberOfChannels) " - + "max=\(session.maximumOutputNumberOfChannels) ports=[\(ports)] " + + "max=\(session.maximumOutputNumberOfChannels) rendering=\(renderingModeName(session.renderingMode)) " + + "multichannelContent=\(session.supportsMultichannelContent) ports=[\(ports)] " + "latency=\(String(format: "%.0f", session.outputLatency * 1000))ms " + "io=\(String(format: "%.1f", session.ioBufferDuration * 1000))ms" #else @@ -26,18 +38,141 @@ enum AudioRouteDescription { } #if os(iOS) || os(tvOS) + private static func portDescription(_ port: AVAudioSessionPortDescription) -> String { + var fields = ["ch=\(port.channels?.count ?? -1)"] + if let labels = port.channels?.map(\.channelLabel), !labels.isEmpty { + fields.append("labels=\(channelLabelSummary(labels))") + fields.append("heights=\(heightChannelCount(labels))") + } + if port.isSpatialAudioEnabled { fields.append("spatial") } + return "\(port.portType.rawValue)[\(fields.joined(separator: ", "))]" + } + + static func renderingModeName(_ mode: AVAudioSession.RenderingMode) -> String { + switch mode { + case .notApplicable: return "notApplicable" + case .monoStereo: return "monoStereo" + case .surround: return "surround" + case .spatialAudio: return "spatialAudio" + case .dolbyAudio: return "dolbyAudio" + case .dolbyAtmos: return "dolbyAtmos" + @unknown default: return "raw\(mode.rawValue)" + } + } + /// AE#395: one line per route change, for the process rather than per engine (a multiview runs several). /// A change mid-session leaves the session's start line describing a route that is gone, and a user /// switching output to test a theory is exactly that case. Installed on first touch, never removed. + /// + /// A rendering-mode, rendering-capability or spatial-playback change gets the same line: an output + /// can switch between multichannel PCM and Dolby Atmos, or change the layouts it supports, without + /// the route itself changing. static let changeLogger: Void = { + let session = AVAudioSession.sharedInstance() _ = NotificationCenter.default.addObserver( - forName: AVAudioSession.routeChangeNotification, - object: AVAudioSession.sharedInstance(), queue: nil + forName: AVAudioSession.routeChangeNotification, object: session, queue: nil ) { note in let reason = (note.userInfo?[AVAudioSessionRouteChangeReasonKey] as? UInt).map(String.init) ?? "?" guard let route = current() else { return } EngineLog.emit("[AetherEngine] audioRoute changed reason=\(reason) \(route)", category: .engine) } + for (name, what) in [(AVAudioSession.renderingModeChangeNotification, "rendering mode"), + (AVAudioSession.renderingCapabilitiesChangeNotification, "rendering capabilities"), + (AVAudioSession.spatialPlaybackCapabilitiesChangedNotification, "spatial playback")] { + _ = NotificationCenter.default.addObserver(forName: name, object: session, queue: nil) { _ in + guard let route = current() else { return } + EngineLog.emit("[AetherEngine] audioRoute \(what) changed \(route)", category: .engine) + } + } }() #endif + + // MARK: - Channel labels + + /// Height labels in CoreAudio's numbering: top centre, the three vertical-height (top front) + /// positions, the three top-back positions, the top-middle and top-rear rows Atmos layouts use, + /// and the top-surround pair. + private static let heightLabels: Set = [ + kAudioChannelLabel_TopCenterSurround, + kAudioChannelLabel_VerticalHeightLeft, kAudioChannelLabel_VerticalHeightCenter, + kAudioChannelLabel_VerticalHeightRight, + kAudioChannelLabel_TopBackLeft, kAudioChannelLabel_TopBackCenter, kAudioChannelLabel_TopBackRight, + kAudioChannelLabel_LeftTopMiddle, kAudioChannelLabel_RightTopMiddle, + kAudioChannelLabel_LeftTopRear, kAudioChannelLabel_CenterTopRear, kAudioChannelLabel_RightTopRear, + kAudioChannelLabel_LeftTopSurround, kAudioChannelLabel_RightTopSurround, + ] + + /// The abbreviations CoreAudioBaseTypes.h uses in its layout comments. + private static let labelNames: [AudioChannelLabel: String] = [ + kAudioChannelLabel_Left: "L", kAudioChannelLabel_Right: "R", kAudioChannelLabel_Center: "C", + kAudioChannelLabel_LFEScreen: "LFE", kAudioChannelLabel_LFE2: "LFE2", + kAudioChannelLabel_LeftSurround: "Ls", kAudioChannelLabel_RightSurround: "Rs", + kAudioChannelLabel_LeftCenter: "Lc", kAudioChannelLabel_RightCenter: "Rc", + kAudioChannelLabel_CenterSurround: "Cs", + kAudioChannelLabel_LeftSurroundDirect: "Lsd", kAudioChannelLabel_RightSurroundDirect: "Rsd", + kAudioChannelLabel_RearSurroundLeft: "Rls", kAudioChannelLabel_RearSurroundRight: "Rrs", + kAudioChannelLabel_LeftWide: "Lw", kAudioChannelLabel_RightWide: "Rw", + kAudioChannelLabel_TopCenterSurround: "Ts", + kAudioChannelLabel_VerticalHeightLeft: "Vhl", kAudioChannelLabel_VerticalHeightCenter: "Vhc", + kAudioChannelLabel_VerticalHeightRight: "Vhr", + kAudioChannelLabel_TopBackLeft: "Tbl", kAudioChannelLabel_TopBackCenter: "Tbc", + kAudioChannelLabel_TopBackRight: "Tbr", + kAudioChannelLabel_LeftTopMiddle: "Ltm", kAudioChannelLabel_RightTopMiddle: "Rtm", + kAudioChannelLabel_LeftTopRear: "Ltr", kAudioChannelLabel_CenterTopRear: "Ctr", + kAudioChannelLabel_RightTopRear: "Rtr", + kAudioChannelLabel_LeftSideSurround: "Lss", kAudioChannelLabel_RightSideSurround: "Rss", + kAudioChannelLabel_LeftBottom: "Lb", kAudioChannelLabel_RightBottom: "Rb", + kAudioChannelLabel_CenterBottom: "Cb", + kAudioChannelLabel_LeftTopSurround: "Lts", kAudioChannelLabel_RightTopSurround: "Rts", + kAudioChannelLabel_LFE3: "LFE3", + kAudioChannelLabel_LeftBackSurround: "Lbs", kAudioChannelLabel_RightBackSurround: "Rbs", + kAudioChannelLabel_LeftEdgeOfScreen: "Les", kAudioChannelLabel_RightEdgeOfScreen: "Res", + kAudioChannelLabel_Mono: "M", + kAudioChannelLabel_HeadphonesLeft: "HpL", kAudioChannelLabel_HeadphonesRight: "HpR", + kAudioChannelLabel_Unused: "-", kAudioChannelLabel_Unknown: "?", + ] + + static func heightChannelCount(_ labels: [AudioChannelLabel]) -> Int { + labels.filter(heightLabels.contains).count + } + + /// Labels in route order, with runs folded so a 32-channel route stays one short field: + /// consecutive discrete channels as `D0-31`, repeats of one label as `?x32`. + /// `L R C LFE Ls Rs Rls Rrs Vhl Vhr Ltr Rtr` is a 7.1.4 route. + static func channelLabelSummary(_ labels: [AudioChannelLabel]) -> String { + var tokens: [String] = [] + var index = 0 + while index < labels.count { + let label = labels[index] + var end = index + 1 + if let discrete = discreteIndex(label) { + while end < labels.count, discreteIndex(labels[end]) == discrete + UInt32(end - index) { end += 1 } + let run = end - index + tokens.append(run > 1 ? "D\(discrete)-\(discrete + UInt32(run - 1))" : "D\(discrete)") + } else { + while end < labels.count, labels[end] == label { end += 1 } + let run = end - index + let name = labelName(label) + tokens.append(run > 1 ? "\(name)x\(run)" : name) + } + index = end + } + return tokens.joined(separator: " ") + } + + /// One label's short name: CoreAudio's abbreviation, `D` for a discrete channel, `A` for an + /// ambisonic component, `#` for anything else. + static func labelName(_ label: AudioChannelLabel) -> String { + if let name = labelNames[label] { return name } + if let discrete = discreteIndex(label) { return "D\(discrete)" } + return hoaName(label) ?? "#\(label)" + } + + private static func discreteIndex(_ label: AudioChannelLabel) -> UInt32? { + label >> 16 == 1 ? label & 0xFFFF : nil + } + + private static func hoaName(_ label: AudioChannelLabel) -> String? { + label >> 16 == 2 ? "A\(label & 0xFFFF)" : nil + } } diff --git a/Sources/AetherEngine/Audio/Spatial/BedLevelMeter.swift b/Sources/AetherEngine/Audio/Spatial/BedLevelMeter.swift new file mode 100644 index 000000000..d85d893af --- /dev/null +++ b/Sources/AetherEngine/Audio/Spatial/BedLevelMeter.swift @@ -0,0 +1,185 @@ +import Accelerate +import Foundation + +/// Per-channel levels of the rendered bed, for the session log. +/// +/// A report that the heights or the LFE are missing can only be placed if the log says what the +/// engine handed AVPlayer. Level here means the encoder was given that channel's sound, so a height +/// heard from the floor speakers lost it after this point. Silence here is a lead, not a verdict: the +/// object counts come from the metadata (gain and position), and an elevated object can still be +/// silent or snap to a floor speaker. +/// +/// Measured on the planes the encoder is fed, gap-fill silence included, because that is what was +/// sent. The first window after a start or a seek closes after `firstWindowSeconds`, so a short test +/// session still logs one; later windows close every `windowSeconds`. Channels are named with +/// CoreAudio's labels, as on the route line. +struct BedLevelMeter { + + static let firstWindowSeconds = 5 + static let windowSeconds = 30 + + private static let heightSpeakers = Set(SpatialSpeaker.allCases.filter(\.isHeight)) + + let layout: SpatialSpeakerLayout + let sampleRate: Int + + private var sumSquares: [Double] + private var peaks: [Float] + private var frames = 0 + private var windowsClosed = 0 + /// Where the window's first frame sits on the output timeline, in seconds, when the caller knows. + private var windowStart: Double? + + /// Latest metadata per element, since an update restates only the elements that changed. + private var elementStates: [ObjectAudioElementState?] = [] + /// Object counts as of the latest update, which hold until the next one: a window with no update + /// at all still reports what is playing. + private var activeObjects = 0 + private var elevatedObjects = 0 + private var maxActiveObjects = 0 + private var maxElevatedObjects = 0 + + /// The decoder's LFE input, before the renderer applies its metadata gain. With the bed's LFE level + /// it splits a silent LFE into a silent source, a zero gain, or no LFE element at all. + /// Every LFE element: TrueHD can carry LFE2 beside LFE, and the renderer sums both. + private var lfeIndices: [Int] = [] + private var lfeInputSquares: Double = 0 + private var lfeInputFrames = 0 + + init(layout: SpatialSpeakerLayout, sampleRate: Int) { + self.layout = layout + self.sampleRate = sampleRate + sumSquares = Array(repeating: 0, count: layout.channelCount) + peaks = Array(repeating: 0, count: layout.channelCount) + } + + /// Count the objects the metadata has playing, and how many of them sit above ear level with + /// elevation allowed. A new object configuration forgets the old elements' states; its first + /// update restates every element. + mutating func observe(roles: [ObjectAudioRole], updates: [ObjectAudioMetadataUpdate], + configurationChanged: Bool = false) { + if configurationChanged || elementStates.count != roles.count { + elementStates = Array(repeating: nil, count: roles.count) + lfeIndices = roles.indices.filter { roles[$0] == .lfe } + // A configuration with no objects sends no object metadata; the old counts must not hold. + activeObjects = 0 + elevatedObjects = 0 + } + for update in updates { + for (index, state) in update.states.enumerated() where index < elementStates.count { + if let state { elementStates[index] = state } + } + activeObjects = 0 + elevatedObjects = 0 + for (index, role) in roles.enumerated() where role == .object { + guard let state = elementStates[index], state.gain > 0 else { continue } + activeObjects += 1 + if state.position.z > 0, !Self.heightSpeakers.isSubset(of: state.excluded) { elevatedObjects += 1 } + } + maxActiveObjects = max(maxActiveObjects, activeObjects) + maxElevatedObjects = max(maxElevatedObjects, elevatedObjects) + } + } + + /// Measure the LFE elements' combined input for one decoded block (one plane per role). + mutating func measureInput(roles: [ObjectAudioRole], planes: [UnsafePointer], frameCount: Int) { + let indices = roles.indices.filter { roles[$0] == .lfe && $0 < planes.count } + guard frameCount > 0, !indices.isEmpty else { return } + for index in indices { + var squares: Float = 0 + vDSP_svesq(planes[index], 1, &squares, vDSP_Length(frameCount)) + lfeInputSquares += Double(squares) + } + lfeInputFrames += frameCount + } + + /// Measure `frameCount` frames of the bed (one plane per layout channel) starting at `startSeconds` + /// on the output timeline. Returns the log line when this closes a window. + mutating func add(_ planes: [UnsafePointer], frameCount: Int, startSeconds: Double? = nil) -> String? { + guard frameCount > 0 else { return nil } + if frames == 0 { windowStart = startSeconds } + let n = vDSP_Length(frameCount) + for c in 0..= due ? closeWindow() : nil + } + + /// The open window, if it holds at least a second, for a seek, end of stream or teardown. The + /// next window is a first window again, and the objects start over with the next update. + mutating func restart() -> String? { + let line = frames >= sampleRate ? closeWindow() : nil + elementStates = [] + lfeIndices = [] + activeObjects = 0 + elevatedObjects = 0 + clearWindow() + windowsClosed = 0 + return line + } + + /// `bed levels 7.1.4 at 3725.4 s over 30.0 s, rms/peak dBFS: L -22.4/-3.1, …; objects: up to 12 + /// active, 5 elevated; LFE input -18.2 dBFS, metadata gain 0.0 dB` + private mutating func closeWindow() -> String { + let channels = layout.speakers.enumerated().map { c, speaker in + let rms = (sumSquares[c] / Double(max(frames, 1))).squareRoot() + return "\(AudioRouteDescription.labelName(speaker.channelLabel)) " + + "\(Self.dBFS(rms))/\(Self.dBFS(Double(peaks[c])))" + } + let at = windowStart.map { String(format: " at %.1f s", $0) } ?? "" + let seconds = String(format: "%.1f", Double(frames) / Double(sampleRate)) + let line = "bed levels \(layout.rawValue)\(at) over \(seconds) s, rms/peak dBFS: " + + channels.joined(separator: ", ") + + "; objects: up to \(maxActiveObjects) active, \(maxElevatedObjects) elevated" + + "; " + lfeSummary() + windowsClosed += 1 + clearWindow() + return line + } + + private func lfeSummary() -> String { + guard !lfeIndices.isEmpty else { return "no LFE element" } + let input = Self.dBFS((lfeInputSquares / Double(max(lfeInputFrames, 1))).squareRoot()) + let gains = lfeIndices.compactMap { elementStates.indices.contains($0) ? elementStates[$0]?.gain : nil } + guard !gains.isEmpty else { return "LFE input \(input) dBFS, no metadata yet" } + let gainText = gains.map { $0 > 0 ? String(format: "%.1f dB", 20 * log10($0)) : "-inf dB" } + .joined(separator: "/") + return "LFE input \(input) dBFS, metadata gain \(gainText)" + } + + /// The decoder's elements, for the configuration line: `LFE + 15 objects`, or the bed channels it + /// names first. + static func configurationSummary(_ roles: [ObjectAudioRole]) -> String { + var parts: [String] = [] + let beds = roles.compactMap { role -> String? in + if case .bed(let speaker) = role { return AudioRouteDescription.labelName(speaker.channelLabel) } + return nil + } + if !beds.isEmpty { parts.append("beds " + beds.joined(separator: " ")) } + let lfe = roles.filter { $0 == .lfe }.count + parts.append(lfe == 0 ? "no LFE" : (lfe == 1 ? "LFE" : "\(lfe) LFE")) + parts.append("\(roles.filter { $0 == .object }.count) objects") + return parts.joined(separator: " + ") + } + + private mutating func clearWindow() { + for c in sumSquares.indices { sumSquares[c] = 0; peaks[c] = 0 } + lfeInputSquares = 0 + lfeInputFrames = 0 + frames = 0 + windowStart = nil + maxActiveObjects = activeObjects + maxElevatedObjects = elevatedObjects + } + + static func dBFS(_ amplitude: Double) -> String { + amplitude > 0 ? String(format: "%.1f", 20 * log10(amplitude)) : "-inf" + } +} diff --git a/Sources/AetherEngine/Audio/Spatial/SpatialAudioBridge.swift b/Sources/AetherEngine/Audio/Spatial/SpatialAudioBridge.swift index d5b344abd..f31ccd758 100644 --- a/Sources/AetherEngine/Audio/Spatial/SpatialAudioBridge.swift +++ b/Sources/AetherEngine/Audio/Spatial/SpatialAudioBridge.swift @@ -46,6 +46,8 @@ final class SpatialAudioBridge: AudioTranscodingBridge, @unchecked Sendable { private let encoder: APACEncoder private var renderer: ObjectAudioRenderer? private var rendererGeneration = -1 + /// What the encoder is fed, per bed channel, for the session log (see `BedLevelMeter`). + private var levelMeter: BedLevelMeter /// Rendered bed awaiting the encoder. TrueHD decodes one 40-frame access unit per block, 1200 /// a second; feeding the converter that often costs more in call overhead than the encode, so @@ -73,7 +75,7 @@ final class SpatialAudioBridge: AudioTranscodingBridge, @unchecked Sendable { private(set) var outputBytesLifetime: Int64 = 0 /// AE#641: called once, on the pump thread, when the object decoder has been fed - /// `AudioBridge.silentFeedPacketThreshold` packets in a row without returning a block. The APAC + /// `silentFeedPacketThreshold` packets in a row without returning a block. The APAC /// sample entry is written up front, like FLAC's, so nothing downstream fails on its own and the /// session would otherwise play the picture over silence. Set before the producer starts feeding. var onDecoderProducedNothing: (@Sendable (AudioBridge.FeedStats) -> Void)? @@ -81,6 +83,15 @@ final class SpatialAudioBridge: AudioTranscodingBridge, @unchecked Sendable { private static let avNoPTS: Int64 = -0x7FFFFFFFFFFFFFFF - 1 + /// Packets the object decoder may take without a block before it counts as producing nothing. + /// Not the channel bridge's 64: a TrueHD packet is one 40-frame access unit, and after a start or + /// resume mid-stream the decoder waits for the next major sync before its first block. Measured + /// on Dolby's Unfold demo, that wait was up to 102 access units; at 64 every resume past the + /// opening seconds was reported as a dead decoder and the session went to `.error` while it + /// played, ignoring every seek after it. 1200 is a second of TrueHD, still a fraction of a second + /// of the producer's time. + static let silentFeedPacketThreshold = 1200 + init( srcTimeBase: AVRational, layout: SpatialSpeakerLayout, @@ -90,6 +101,7 @@ final class SpatialAudioBridge: AudioTranscodingBridge, @unchecked Sendable { self.srcTimeBase = srcTimeBase self.layout = layout self.decoder = decoder + levelMeter = BedLevelMeter(layout: layout, sampleRate: Self.sampleRate) let rate = bitRate ?? layout.channelCount * Self.bitRatePerChannel encoder = try APACEncoder(layout: layout, sampleRate: Double(Self.sampleRate), bitRate: rate) soundSampleEntryOverride = APACSampleEntry.sampleEntry( @@ -178,7 +190,7 @@ final class SpatialAudioBridge: AudioTranscodingBridge, @unchecked Sendable { /// Caller holds `opLock`. private func noteDecodedNothingIfNeeded() { guard !decodedNothingReported, stats.decodedNothing, - stats.packetsFedSinceLastEnqueue >= AudioBridge.silentFeedPacketThreshold else { return } + stats.packetsFedSinceLastEnqueue >= Self.silentFeedPacketThreshold else { return } decodedNothingReported = true EngineLog.emit( "[SpatialAudioBridge] ERROR: AE#641 the TrueHD object decoder has produced nothing: " @@ -193,6 +205,7 @@ final class SpatialAudioBridge: AudioTranscodingBridge, @unchecked Sendable { drainedAtEOF = true var results: [UnsafeMutablePointer] = [] try? encodePending(into: &results) + logLevels(levelMeter.restart()) if let packets = try? encoder.flush() { emit(packets, into: &results) } if !results.isEmpty { EngineLog.emit("[SpatialAudioBridge] EOF flush emitted \(results.count) tail packet(s)", category: .session) @@ -206,6 +219,7 @@ final class SpatialAudioBridge: AudioTranscodingBridge, @unchecked Sendable { decoder.reset() renderer?.reset() encoder.reset() + logLevels(levelMeter.restart()) pendingFrames = 0 originFrame = nil encoderStartFrame = nil @@ -225,6 +239,8 @@ final class SpatialAudioBridge: AudioTranscodingBridge, @unchecked Sendable { func close() { opLock.lock() defer { opLock.unlock() } + // A viewer who hears no heights stops playback: the window they stopped in is the one to log. + logLevels(levelMeter.restart()) cleanup() } @@ -241,18 +257,22 @@ final class SpatialAudioBridge: AudioTranscodingBridge, @unchecked Sendable { guard block.sampleRate == Self.sampleRate else { throw BridgeError.unsupportedSampleRate(block.sampleRate) } guard block.frameCount > 0 else { return } - if renderer == nil || block.configurationGeneration != rendererGeneration { - if renderer != nil { - EngineLog.emit( - "[SpatialAudioBridge] object configuration changed (\(block.roles.count) elements), renderer rebuilt", - category: .session - ) - } + let rebuilt = renderer == nil || block.configurationGeneration != rendererGeneration + if rebuilt { + // Whether the presentation has an LFE element at all is the first question a missing-LFE + // report asks. + EngineLog.emit( + "[SpatialAudioBridge] object configuration\(renderer == nil ? "" : " changed, renderer rebuilt"): " + + BedLevelMeter.configurationSummary(block.roles), + category: .session + ) renderer = ObjectAudioRenderer(layout: layout, roles: block.roles) rendererGeneration = block.configurationGeneration } else if block.isDiscontinuity { renderer?.markDiscontinuity() } + levelMeter.observe(roles: block.roles, updates: block.updates, configurationChanged: rebuilt) + levelMeter.measureInput(roles: block.roles, planes: block.planes, frameCount: block.frameCount) // Anchor the encoder on the first block after a reset, and keep the encoder's input // contiguous with the source afterwards: a gap the decoder skipped (corrupt access units) is // filled with silence so everything after it stays on its timestamp. @@ -286,9 +306,12 @@ final class SpatialAudioBridge: AudioTranscodingBridge, @unchecked Sendable { guard pendingFrames > 0 else { return } let frames = pendingFrames pendingFrames = 0 + let planes = outputPlanes.map { UnsafePointer($0) } + // Where this chunk plays, for the level line: the anchor is a source PTS in 48 kHz frames. + let chunkStart = encoderStartFrame.map { Double($0 + encoderFramesIn) / Double(Self.sampleRate) } let packets: [APACEncoder.Packet] do { - packets = try encoder.encode(planes: outputPlanes.map { UnsafePointer($0) }, frameCount: frames) + packets = try encoder.encode(planes: planes, frameCount: frames) } catch { // Packet timestamps count packets since the encoder started, so a failed call that // swallowed packets would leave every later one stamped early. Start the encoder over @@ -304,9 +327,14 @@ final class SpatialAudioBridge: AudioTranscodingBridge, @unchecked Sendable { throw error } encoderFramesIn += Int64(frames) + logLevels(levelMeter.add(planes, frameCount: frames, startSeconds: chunkStart)) emit(packets, into: &results) } + private func logLevels(_ line: String?) { + if let line { EngineLog.emit("[SpatialAudioBridge] \(line)", category: .session) } + } + /// Stamp encoder packets onto the output timeline and wrap them for the muxer. Packet k is /// stamped `encoderStartFrame + k·1024`, the encoder's priming packets included, because /// AVFoundation presents an APAC packet's audio `leadingFrames` (2048) before its timestamp: it diff --git a/Sources/AetherEngine/Audio/Spatial/SpatialSpeakerLayout.swift b/Sources/AetherEngine/Audio/Spatial/SpatialSpeakerLayout.swift index ff913530f..dc81c81d0 100644 --- a/Sources/AetherEngine/Audio/Spatial/SpatialSpeakerLayout.swift +++ b/Sources/AetherEngine/Audio/Spatial/SpatialSpeakerLayout.swift @@ -96,4 +96,28 @@ public enum SpatialSpeaker: String, Sendable, CaseIterable { var isLFE: Bool { self == .lfe } var isHeight: Bool { position.z > 0 } + + /// The CoreAudio label for this position, so the bed's log line names channels the way the + /// route line and CoreAudioBaseTypes.h do. 5.1-style surrounds and 7.1 sides share + /// `LeftSurround`/`RightSurround`, as CoreAudio's own Atmos tags do; no layout has both. + var channelLabel: AudioChannelLabel { + switch self { + case .left: return kAudioChannelLabel_Left + case .right: return kAudioChannelLabel_Right + case .center: return kAudioChannelLabel_Center + case .lfe: return kAudioChannelLabel_LFEScreen + case .surroundLeft, .sideLeft: return kAudioChannelLabel_LeftSurround + case .surroundRight, .sideRight: return kAudioChannelLabel_RightSurround + case .rearLeft: return kAudioChannelLabel_RearSurroundLeft + case .rearRight: return kAudioChannelLabel_RearSurroundRight + case .wideLeft: return kAudioChannelLabel_LeftWide + case .wideRight: return kAudioChannelLabel_RightWide + case .topFrontLeft: return kAudioChannelLabel_VerticalHeightLeft + case .topFrontRight: return kAudioChannelLabel_VerticalHeightRight + case .topMiddleLeft: return kAudioChannelLabel_LeftTopMiddle + case .topMiddleRight: return kAudioChannelLabel_RightTopMiddle + case .topRearLeft: return kAudioChannelLabel_LeftTopRear + case .topRearRight: return kAudioChannelLabel_RightTopRear + } + } } diff --git a/Sources/AetherEngine/BackgroundTeardownSelection.swift b/Sources/AetherEngine/BackgroundTeardownSelection.swift index 314e54ffb..93fdf6306 100644 --- a/Sources/AetherEngine/BackgroundTeardownSelection.swift +++ b/Sources/AetherEngine/BackgroundTeardownSelection.swift @@ -39,7 +39,13 @@ extension AetherEngine { BackgroundTeardownSelection( subtitles: Self.mergedSubtitleCarryover( live: captureSubtitleSessionCarryover(), snapshot: parked?.subtitles), - audioTrackIndex: activeAudioTrackIndex ?? parked?.audioTrackIndex, + // Newest audio intent first. A host that picks a track and then rebuilds the session + // (a transport restore, a foreground return) before the switch has landed would + // otherwise read the active track the switch's own teardown had just cleared, and come + // back on the auto pick, superseding the switch: a TrueHD pick on a file with an E-AC-3 + // Atmos track returned to the E-AC-3 track on an Apple TV. + audioTrackIndex: pendingAudioSelection ?? audioSwitchInFlight + ?? activeAudioTrackIndex ?? parked?.audioTrackIndex, discTitleID: activeDiscTitleID ?? parked?.discTitleID ) } diff --git a/Sources/AetherEngine/Native/NativeAVPlayerHost.swift b/Sources/AetherEngine/Native/NativeAVPlayerHost.swift index 75cc958bc..5d6f1b71c 100644 --- a/Sources/AetherEngine/Native/NativeAVPlayerHost.swift +++ b/Sources/AetherEngine/Native/NativeAVPlayerHost.swift @@ -2282,6 +2282,13 @@ final class NativeAVPlayerHost { /// Dump item.tracks at readyToPlay (HLS: asset.tracks is empty; item.tracks has the resolved list after playlist+init.mp4 parse). Channel layout tag diagnoses multichannel-routing path. private static func dumpPlayerItemTracks(_ item: AVPlayerItem, sid: Int) async { + // Whether AVPlayer may spatialize (virtualize) multichannel audio on this item: one of the + // candidates when a bed's heights come out of the floor speakers. + EngineLog.emit( + "[NativeAVPlayerHost] #\(sid) item.allowedAudioSpatializationFormats=" + + "\(spatializationFormatsName(item.allowedAudioSpatializationFormats)) (readyToPlay)", + category: .engine + ) let tracks = item.tracks if tracks.isEmpty { EngineLog.emit("[NativeAVPlayerHost] #\(sid) item.tracks empty (readyToPlay)", category: .engine) @@ -2756,6 +2763,16 @@ final class NativeAVPlayerHost { EngineLog.emit("[NativeAVPlayerHost] #\(sid) audioRoute \(route) (\(phase))", category: .engine) } + nonisolated static func spatializationFormatsName(_ formats: AVAudioSpatializationFormats) -> String { + switch formats { + case []: return "none" + case .monoAndStereo: return "monoAndStereo" + case .multichannel: return "multichannel" + case .monoStereoAndMultichannel: return "monoStereoAndMultichannel" + default: return "raw\(formats.rawValue)" + } + } + /// Read sr/ch/bits/layoutTag from CMAudioFormatDescription. Layout tag diagnoses where downmix occurs: unknown/stereo tag = AVPlayer parse layer; correct 7.1 tag = route/soundbar layer. nonisolated private static func audioFormatDescription(_ fmt: CMFormatDescription) -> String { var parts: [String] = [] diff --git a/Sources/AetherEngine/Network/HLSLocalServer.swift b/Sources/AetherEngine/Network/HLSLocalServer.swift index cf55ae87a..1a39ac534 100644 --- a/Sources/AetherEngine/Network/HLSLocalServer.swift +++ b/Sources/AetherEngine/Network/HLSLocalServer.swift @@ -1635,6 +1635,22 @@ final class HLSLocalServer: @unchecked Sendable { return nil } + /// What a master tells AVPlayer about the audio, read back from the text itself so the log line cannot + /// drift from what is served: the variant's `CODECS`, whether an `EXT-X-MEDIA` audio rendition is + /// declared, and its `CHANNELS`, which is where HLS states a channel count and spatial or object + /// audio. `codecs="avc1.640029,apac.31.03" audioRendition=yes channels=none`. + static func masterAudioSummary(_ master: String) -> String { + func quoted(_ value: String?) -> String { value.map { "\"\($0)\"" } ?? "none" } + let lines = master.split(whereSeparator: \.isNewline).map(String.init) + let codecs = lines.first { $0.hasPrefix("#EXT-X-STREAM-INF:") } + .flatMap { HLSPlaylistParser.attribute("CODECS", in: $0) } + let rendition = lines.first { + $0.hasPrefix("#EXT-X-MEDIA:") && HLSPlaylistParser.attribute("TYPE", in: $0) == "AUDIO" + } + let channels = rendition.flatMap { HLSPlaylistParser.attribute("CHANNELS", in: $0) } + return "codecs=\(quoted(codecs)) audioRendition=\(rendition == nil ? "no" : "yes") channels=\(quoted(channels))" + } + /// Pure playlist builders callable without a live server instance. subResourceBaseURL emits absolute URIs for AVAssetResourceLoader; nil emits relative URIs for the HTTP workflow. static func buildMasterPlaylistText(provider: HLSSegmentProvider, subResourceBaseURL: URL? = nil, diff --git a/Sources/AetherEngine/Video/HLSVideoEngine+SpatialAudio.swift b/Sources/AetherEngine/Video/HLSVideoEngine+SpatialAudio.swift index 9c9d110c8..d8216d836 100644 --- a/Sources/AetherEngine/Video/HLSVideoEngine+SpatialAudio.swift +++ b/Sources/AetherEngine/Video/HLSVideoEngine+SpatialAudio.swift @@ -42,12 +42,23 @@ extension HLSVideoEngine { rendering: objectAudioRendering, codecID: par.codec_id, profile: par.profile, sampleRate: par.sample_rate) else { - if objectAudioRendering != .off, par.codec_id == AV_CODEC_ID_TRUEHD { - EngineLog.emit( - "[HLSVideoEngine] object rendering requested but TrueHD is not Atmos " - + "(profile=\(par.profile) rate=\(par.sample_rate)); keeping the channel bridge", - category: .session - ) + if par.codec_id == AV_CODEC_ID_TRUEHD { + if objectAudioRendering != .off { + EngineLog.emit( + "[HLSVideoEngine] object rendering requested but TrueHD is not 48 kHz Atmos " + + "(profile=\(par.profile) rate=\(par.sample_rate)); keeping the channel bridge", + category: .session + ) + } else if par.profile == AV_PROFILE_TRUEHD_ATMOS { + // The host's choice, not a failure, but the one a missing-heights report turns on: + // the channel bridge plays the 7.1 presentation, where the objects are mixed into + // the floor channels. + EngineLog.emit( + "[HLSVideoEngine] TrueHD Atmos with objectAudioRendering off; the channel bridge " + + "carries the 7.1 presentation, with the objects mixed into the floor channels", + category: .session + ) + } } return nil } diff --git a/Sources/AetherEngine/Video/HLSVideoEngine.swift b/Sources/AetherEngine/Video/HLSVideoEngine.swift index 84c126870..c00b0d179 100644 --- a/Sources/AetherEngine/Video/HLSVideoEngine.swift +++ b/Sources/AetherEngine/Video/HLSVideoEngine.swift @@ -2128,6 +2128,12 @@ public final class HLSVideoEngine: @unchecked Sendable { self.servedSourceIsHDR = videoRange != .sdr self.servedDolbyVisionConversion = convertP7ToProfile81 ? .profile7ToProfile81 : nil EngineLog.emit("[HLSVideoEngine] serving on \(url.absoluteString) (dvModeAvailable=\(dvModeAvailable) effectiveDvMode=\(effectiveDvMode) panelIsHDR=\(panelIsInHDRMode) displaySupportsHDR=\(displaySupportsHDR) matchContent=\(matchContentEnabled) sourceIsHDR=\(videoRange != .sdr || effectiveDvMode) useMaster=\(useMasterPlaylist) videoRange=\(videoRange) dvVariant=\(dvVariant) audioLang=\(servedAudioLanguage ?? "none"))") + // Everything the playlist says about the audio, and a media-direct session says nothing: AVPlayer + // then reads only the init segment. + let masterAudio = useMasterPlaylist + ? HLSLocalServer.masterAudioSummary(HLSLocalServer.buildMasterPlaylistText(provider: prov)) + : "not served (media playlist only)" + EngineLog.emit("[HLSVideoEngine] master audio: \(masterAudio)", category: .session) return url } diff --git a/Tests/AetherEngineTests/AudioRouteDescriptionTests.swift b/Tests/AetherEngineTests/AudioRouteDescriptionTests.swift new file mode 100644 index 000000000..6bf1941f7 --- /dev/null +++ b/Tests/AetherEngineTests/AudioRouteDescriptionTests.swift @@ -0,0 +1,74 @@ +import Testing +import CoreAudioTypes +@testable import AetherEngine + +/// The route line names the speakers the output offers, so a report can say whether an HDMI route had +/// height channels to put a bed's heights on. +struct AudioRouteDescriptionTests { + + @Test("a 7.1.4 route is spelled with CoreAudio's abbreviations and counts four heights") + func atmosRoute() { + let labels: [AudioChannelLabel] = [ + kAudioChannelLabel_Left, kAudioChannelLabel_Right, kAudioChannelLabel_Center, + kAudioChannelLabel_LFEScreen, kAudioChannelLabel_LeftSurround, kAudioChannelLabel_RightSurround, + kAudioChannelLabel_RearSurroundLeft, kAudioChannelLabel_RearSurroundRight, + kAudioChannelLabel_VerticalHeightLeft, kAudioChannelLabel_VerticalHeightRight, + kAudioChannelLabel_LeftTopRear, kAudioChannelLabel_RightTopRear, + ] + #expect(AudioRouteDescription.channelLabelSummary(labels) == "L R C LFE Ls Rs Rls Rrs Vhl Vhr Ltr Rtr") + #expect(AudioRouteDescription.heightChannelCount(labels) == 4) + } + + @Test("the top-middle row of a 9.1.6 route counts as heights") + func topMiddle() { + let labels = [kAudioChannelLabel_LeftTopMiddle, kAudioChannelLabel_RightTopMiddle, kAudioChannelLabel_Left] + #expect(AudioRouteDescription.heightChannelCount(labels) == 2) + } + + @Test("the top-surround pair counts as heights and every label in the SDK's 55-66 range has a name") + func topSurroundAndLaterLabels() { + let labels = (55...66).map { AudioChannelLabel($0) } + #expect(AudioRouteDescription.heightChannelCount(labels) == 2) + #expect(!AudioRouteDescription.channelLabelSummary(labels).contains("#")) + } + + @Test("a bed speaker and the route name the same channel the same way") + func bedSpeakersUseRouteNames() { + let names = SpatialSpeakerLayout.l714.speakers.map { AudioRouteDescription.labelName($0.channelLabel) } + #expect(names.joined(separator: " ") == "L R C LFE Ls Rs Rls Rrs Vhl Vhr Ltr Rtr") + let ninePointOneSix = SpatialSpeakerLayout.l916.speakers.map { AudioRouteDescription.labelName($0.channelLabel) } + #expect(ninePointOneSix.joined(separator: " ") == "L R C LFE Ls Rs Rls Rrs Lw Rw Vhl Vhr Ltm Rtm Ltr Rtr") + } + + @Test("runs fold so a 32-channel route stays one short field, and no label is a height") + func runsFold() { + let discrete = (0..<32).map { kAudioChannelLabel_Discrete_0 | AudioChannelLabel($0) } + #expect(AudioRouteDescription.channelLabelSummary(discrete) == "D0-31") + #expect(AudioRouteDescription.heightChannelCount(discrete) == 0) + let unknown = [AudioChannelLabel](repeating: kAudioChannelLabel_Unknown, count: 8) + #expect(AudioRouteDescription.channelLabelSummary(unknown) == "?x8") + let mixed = [kAudioChannelLabel_Left, kAudioChannelLabel_Right, + kAudioChannelLabel_Discrete_0 | 2, kAudioChannelLabel_Discrete_0 | 3, + kAudioChannelLabel_Discrete_0 | 5, kAudioChannelLabel_HOA_ACN_0 | 1, 9999] + #expect(AudioRouteDescription.channelLabelSummary(mixed) == "L R D2-3 D5 A1 #9999") + } +} + +#if os(iOS) || os(tvOS) +/// The live line, read off whatever route the device or simulator has: the fields a missing-heights +/// report reads are always there, in order, whatever the values. +struct AudioRouteDescriptionLiveTests { + @Test("the route line carries the rendering mode, multichannel support and per-port labels") + func liveLine() throws { + let line = try #require(AudioRouteDescription.current()) + print("[AudioRouteDescriptionLiveTests] \(line)") + let rendering = try #require(line.range(of: " rendering=")) + let multichannel = try #require(line.range(of: " multichannelContent=")) + let ports = try #require(line.range(of: " ports=[")) + #expect(rendering.lowerBound < multichannel.lowerBound && multichannel.lowerBound < ports.lowerBound) + #expect(!line.contains("rendering=raw"), "every mode the SDK defines has a name") + // Each port is its type and its channel fields, never the device's name. + #expect(line.range(of: #"ports=\[(\]|[A-Za-z0-9]+\[ch=-?\d+)"#, options: .regularExpression) != nil) + } +} +#endif diff --git a/Tests/AetherEngineTests/DocumentedConstantsTests.swift b/Tests/AetherEngineTests/DocumentedConstantsTests.swift index f1f9b520e..abe3fc582 100644 --- a/Tests/AetherEngineTests/DocumentedConstantsTests.swift +++ b/Tests/AetherEngineTests/DocumentedConstantsTests.swift @@ -239,6 +239,22 @@ final class DocumentedConstantsTests: XCTestCase { "docs/formats.md and cli.md quote apac.31.04 for 16 channels") } + /// formats.md "TrueHD Atmos (object rendering)" and the CHANGELOG quote the bed level windows. + func testBedLevelWindowsAreWhatTheDocsSay() throws { + let docs = try documentation() + XCTAssertEqual(BedLevelMeter.firstWindowSeconds, 5) + XCTAssertEqual(BedLevelMeter.windowSeconds, 30) + assertDocumented("after the first 5 s of audio (and again after every seek), then every 30 s", docs) + } + + /// formats.md "TrueHD Atmos (object rendering)" and the CHANGELOG quote the APAC silence threshold. + func testAPACSilenceThresholdIsWhatTheDocsSay() throws { + guard #available(macOS 26.0, iOS 26.0, tvOS 26.0, visionOS 26.0, *) else { return } + let docs = try documentation() + XCTAssertEqual(SpatialAudioBridge.silentFeedPacketThreshold, 1200) + assertDocumented("returns nothing for 1200 packets in a row, a second of TrueHD", docs) + } + func testExternalSubtitleTrackIDBaseIsDocumented() throws { let docs = try documentation() XCTAssertEqual(AetherEngine.externalSubtitleTrackIDBase, 100_000) diff --git a/Tests/AetherEngineTests/Issue357BackgroundTeardownSelectionTests.swift b/Tests/AetherEngineTests/Issue357BackgroundTeardownSelectionTests.swift index 535a754c5..d84687d0a 100644 --- a/Tests/AetherEngineTests/Issue357BackgroundTeardownSelectionTests.swift +++ b/Tests/AetherEngineTests/Issue357BackgroundTeardownSelectionTests.swift @@ -117,6 +117,36 @@ struct Issue357BackgroundTeardownSelectionTests { #expect(selection.discTitleID == 7) } + /// A host picked TrueHD on a file whose auto pick is E-AC-3 Atmos, then rebuilt the session while + /// the switch was running: the rebuild read the active track the switch's teardown had cleared + /// and came back on E-AC-3, superseding the switch. + @Test("a reload taken while an audio switch is pending or running restores the switch's track") + func reloadKeepsAudioSwitch() throws { + let engine = try AetherEngine() + engine.activeAudioTrackIndex = 3 + engine.pendingAudioSelection = 1 + #expect(engine.consumeReloadSelection().audioTrackIndex == 1) + engine.pendingAudioSelection = nil + engine.activeAudioTrackIndex = nil + engine.audioSwitchInFlight = 1 + #expect(engine.consumeReloadSelection().audioTrackIndex == 1) + engine.audioSwitchInFlight = nil + engine.activeAudioTrackIndex = 3 + #expect(engine.consumeReloadSelection().audioTrackIndex == 3) + } + + /// A `stop()` or `load()` cancels the switch; its track is then nobody's intent. + @Test("a cancelled audio switch is not restored by a later reload") + func cancelledSwitchIsForgotten() throws { + let engine = try AetherEngine() + engine.pendingAudioSelection = 2 + engine.audioSwitchInFlight = 1 + engine.stop() + #expect(engine.pendingAudioSelection == nil) + #expect(engine.audioSwitchInFlight == nil) + #expect(engine.consumeReloadSelection().audioTrackIndex == nil) + } + @Test("a pick made after the teardown is newer intent and wins over the snapshot") func newerIntentWins() throws { let engine = try AetherEngine() diff --git a/Tests/AetherEngineTests/Issue458AudioLanguageMetadataTests.swift b/Tests/AetherEngineTests/Issue458AudioLanguageMetadataTests.swift index 0b62008b0..6ddedfc32 100644 --- a/Tests/AetherEngineTests/Issue458AudioLanguageMetadataTests.swift +++ b/Tests/AetherEngineTests/Issue458AudioLanguageMetadataTests.swift @@ -207,6 +207,20 @@ struct Issue458AudioRenditionPlaylistTests { #expect(!master.contains("AUDIO=\"aud\"")) } + /// The session log's `master audio:` line, read back from the served text so it cannot drift from it. + @Test("the master audio summary reads codecs, the rendition and its CHANNELS off the master text") + func masterAudioSummary() { + let labelled = HLSLocalServer.buildMasterPlaylistText( + provider: AudioRenditionMockProvider(audio: (language: "eng", name: "English"))) + #expect(HLSLocalServer.masterAudioSummary(labelled) + == "codecs=\"hvc1.1.6.L120.90,mp4a.40.2\" audioRendition=yes channels=none") + let unlabelled = HLSLocalServer.buildMasterPlaylistText(provider: AudioRenditionMockProvider(audio: nil)) + #expect(HLSLocalServer.masterAudioSummary(unlabelled) + == "codecs=\"hvc1.1.6.L120.90,mp4a.40.2\" audioRendition=no channels=none") + let declared = labelled.replacingOccurrences(of: "AUTOSELECT=YES", with: "AUTOSELECT=YES,CHANNELS=\"12/-/BED-12\"") + #expect(HLSLocalServer.masterAudioSummary(declared).hasSuffix("channels=\"12/-/BED-12\"")) + } + /// A group referenced by a variant has to exist, or AVPlayer fails the master outright. @Test("the variant's AUDIO attribute and the group it names appear together or not at all") func groupReferenceIsConsistent() { diff --git a/Tests/AetherEngineTests/SpatialAudioBridgeTests.swift b/Tests/AetherEngineTests/SpatialAudioBridgeTests.swift index b553ab377..f03f6fc53 100644 --- a/Tests/AetherEngineTests/SpatialAudioBridgeTests.swift +++ b/Tests/AetherEngineTests/SpatialAudioBridgeTests.swift @@ -181,24 +181,26 @@ struct SpatialAudioBridgeTests { /// The APAC sample entry is written before any audio, so a decoder that never finds a major sync /// plays the picture over silence unless the bridge says so itself (AE#641). + final class Reports: @unchecked Sendable { + private let lock = NSLock() + private var count = 0 + func add() { lock.lock(); count += 1; lock.unlock() } + var value: Int { lock.lock(); defer { lock.unlock() }; return count } + } + @Test("a decoder that returns or accepts nothing is reported once, at the threshold, and a healthy one never") func decodedNothingIsReportedOnce() throws { guard #available(macOS 26.0, iOS 26.0, tvOS 26.0, visionOS 26.0, *) else { return } - final class Reports: @unchecked Sendable { - private let lock = NSLock() - private var count = 0 - func add() { lock.lock(); count += 1; lock.unlock() } - var value: Int { lock.lock(); defer { lock.unlock() }; return count } - } let silent = try SpatialAudioBridge( srcTimeBase: AVRational(num: 1, den: 1000), layout: .l714, decoder: SyntheticDecoder(skipAfterReset: .max)) defer { silent.close() } let reports = Reports() silent.onDecoderProducedNothing = { _ in reports.add() } - _ = try feed(silent, startMs: 0, count: AudioBridge.silentFeedPacketThreshold - 1) + let threshold = SpatialAudioBridge.silentFeedPacketThreshold + _ = try feed(silent, startMs: 0, count: threshold - 1) #expect(reports.value == 0) - _ = try feed(silent, startMs: 1260, count: 40) + _ = try feed(silent, startMs: Int64(threshold - 1) * 20, count: 40) #expect(reports.value == 1) let rejecting = try SpatialAudioBridge( @@ -206,7 +208,7 @@ struct SpatialAudioBridgeTests { defer { rejecting.close() } let rejected = Reports() rejecting.onDecoderProducedNothing = { _ in rejected.add() } - _ = try feed(rejecting, startMs: 0, count: AudioBridge.silentFeedPacketThreshold + 10) + _ = try feed(rejecting, startMs: 0, count: SpatialAudioBridge.silentFeedPacketThreshold + 10) #expect(rejected.value == 1) let healthy = try SpatialAudioBridge( @@ -219,6 +221,35 @@ struct SpatialAudioBridgeTests { #expect(quiet.value == 0) } + /// The resume that put a tvOS session into `.error`: TrueHD in Matroska or M2TS is one 40-frame + /// access unit per packet, and after a mid-stream start the decoder skips to the next major sync, + /// 102 access units on Dolby's Unfold demo. That wait is not a dead decoder. + @Test("a mid-stream start that waits 102 access units for a major sync is not reported as silence") + func majorSyncWaitIsNotSilence() throws { + guard #available(macOS 26.0, iOS 26.0, tvOS 26.0, visionOS 26.0, *) else { return } + let bridge = try SpatialAudioBridge( + srcTimeBase: AVRational(num: 1, den: 48_000), layout: .l714, + decoder: SyntheticDecoder(skipAfterReset: 102)) + defer { bridge.close() } + let reports = Reports() + bridge.onDecoderProducedNothing = { _ in reports.add() } + var emitted = 0 + for i in 0..<300 { + guard let pkt = av_packet_alloc() else { continue } + defer { var p: UnsafeMutablePointer? = pkt; av_packet_free(&p) } + _ = av_new_packet(pkt, 1) + pkt.pointee.pts = Int64(i) * 40 + pkt.pointee.dts = pkt.pointee.pts + for fp in try bridge.feed(packet: pkt) { + emitted += 1 + var p: UnsafeMutablePointer? = fp + trackedPacketFree(&p) + } + } + #expect(reports.value == 0) + #expect(emitted > 0, "the decoder found its sync and the bridge encoded") + } + @Test("a producer restart re-anchors on the new position") func restartReanchors() throws { guard #available(macOS 26.0, iOS 26.0, tvOS 26.0, visionOS 26.0, *) else { return } @@ -400,3 +431,133 @@ struct SpatialAudioBridgeTests { return nil } } + +/// The bed level line is the engine's half of a missing-heights report: it says which channels of the +/// rendered bed carried sound in what AVPlayer was handed, so a silent height here and a silent height +/// at the receiver are told apart. +@Suite("TrueHD Atmos bed level meter") +struct BedLevelMeterTests { + + /// `chunks` encode chunks (2048 frames, as the bridge feeds it) of a half-scale sine on one channel + /// of a 7.1.4 bed, the rest silent. At 48 kHz the 5 s first window closes on chunk 118 and a 30 s + /// window on chunk 704. + private func feed(_ meter: inout BedLevelMeter, chunks: Int, channel: Int = 0, + startSeconds: Double? = nil) -> [String] { + let layout = SpatialSpeakerLayout.l714 + let chunk = 2048 + let planes = (0...allocate(capacity: chunk) } + defer { planes.forEach { $0.deallocate() } } + for c in planes.indices { planes[c].update(repeating: 0, count: chunk) } + for i in 0...allocate(capacity: frames) + let object = UnsafeMutablePointer.allocate(capacity: frames) + defer { lfe.deallocate(); object.deallocate() } + for i in 0..