Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 35 additions & 0 deletions .github/workflows/tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,41 @@ jobs:
run: swift test --parallel --num-workers $(sysctl -n hw.ncpu)
timeout-minutes: 20

build-without-nemo-text-processing:
name: Build without NemoTextProcessing trait (macOS)
runs-on: macos-15
steps:
- name: Checkout code
uses: actions/checkout@v5

- name: Check versions
run: |
swift --version
xcodebuild -version

# #880/#888: consumers on Swift 6.2+ can resolve with `traits: []` to keep the
# ~18 MB Rust engine out of ASR-only apps. Prove the package still builds and
# that no engine symbol reaches the linked product. SwiftPM 6.1 (Xcode 16.4,
# the image default) accepts the flag but still links the binary target, so
# this job must run on an Xcode 26 toolchain.
- name: Select Xcode 26
run: |
sudo xcode-select -s /Applications/Xcode_26.3.app
swift --version

- name: Build with the trait disabled
run: swift build --disable-default-traits --product fluidaudiocli

- name: Assert no engine symbols linked
run: |
count=$(nm .build/debug/fluidaudiocli | grep -c 'text_processing_rs\|_nemo_' || true)
echo "engine symbols in fluidaudiocli: $count"
test "$count" -eq 0

- name: Run tests with the trait disabled
run: swift test --disable-default-traits --filter 'TextNormalizerUnavailableTests|NemoTextNormalizerUnavailableTests|TextNormalizerTests|NemoTextNormalizerTests'
timeout-minutes: 20

build-macos-x86_64:
name: Build Swift Package (macOS x86_64 cross-compile)
runs-on: macos-15
Expand Down
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -132,3 +132,4 @@ fluidaudio_cli/*
# scripts + trials live in mobius repo, see models/tts/supertonic_3/).
.venv-supertonic3/
build/supertonic-3-coreml/
.build-*/
50 changes: 48 additions & 2 deletions Documentation/ASR/PostProcessing.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ TN converts written-form text to spoken form — useful for TTS preprocessing:

## Using with FluidAudio

FluidAudio supports text-processing-rs through the `TextNormalizer` class. The native engine ships with the package as the `NemoTextProcessing` binary target and is linked directly — no setup required, it works out of the box for every SwiftPM consumer.
FluidAudio supports text-processing-rs through the `TextNormalizer` class. The native engine ships with the package as the `NemoTextProcessing` binary target and is linked directly — no setup required, it works out of the box for every SwiftPM consumer. Apps that don't use TTS or ITN can opt out of the engine (about 8 MB per architecture slice) with a package trait; see [Opting out](#opting-out-of-the-engine).

### ITN (Spoken to Written)

Expand Down Expand Up @@ -70,4 +70,50 @@ print(normalizedResult.text) // Written form

### Native Library

The engine is bundled: `Package.swift` declares a `NemoTextProcessing` binary target (a prebuilt xcframework from [text-processing-rs](https://github.com/FluidInference/text-processing-rs) releases) that SwiftPM downloads and links automatically. `TextNormalizer.isNativeAvailable` always returns `true`; it is kept only for source compatibility with releases ≤ 0.15.6, which resolved the library at runtime and silently returned input unchanged when it was absent.
The engine is bundled: `Package.swift` declares a `NemoTextProcessing` binary target (a prebuilt xcframework from [text-processing-rs](https://github.com/FluidInference/text-processing-rs) releases) that SwiftPM downloads and links automatically. It is linked at build time, so `TextNormalizer.isNativeAvailable` is a compile-time constant: `true` whenever the engine is part of the build, `false` only when a consumer opts out (below). Releases ≤ 0.15.6 resolved the library at runtime and silently returned input unchanged when it was absent.

### Opting out of the engine

The engine is a prebuilt Rust static library (about 8 MB per architecture slice once linked and stripped, measured on `fluidaudiocli`; the xcframework itself is ~29 MB per iOS slice). ASR/VAD/diarization-only apps, and apps that ship their own Rust runtime (a second copy of the Rust std symbols fails to link), can leave it out with the `NemoTextProcessing` package trait. Requires Swift 6.2 / Xcode 26 or later; older toolchains read `Package.swift` and always link the engine. (SwiftPM 6.1 in Xcode 16.3–16.4 accepts `traits: []` but still links the binary target, so it gives no size benefit there.)

```swift
// Package.swift of the consuming package / app
.package(url: "https://github.com/FluidInference/FluidAudio.git", from: "0.15.7", traits: [])
```

With the trait disabled:

- `TextNormalizer` and `NemoTextNormalizer` remain in the API. `isNativeAvailable`, `isTnAvailable`, and `NemoTextNormalizer.isAvailable` report `false`.
- Every normalization call returns its input unchanged; `version` is `nil`; custom rules are ignored (a warning is logged).
- TTS frontends run without NeMo normalization: Kokoro English falls back to the built-in `EnglishTextNormalizer` rules, and Kokoro Mandarin verbalizes numerals with `MandarinNumberNormalizer`. Keep the trait enabled for byte-exact NeMo readings.

**Xcode projects.** Xcode 26.3 has no UI or pbxproj key for package traits (support appears in 26.4). Until then, wrap the dependency in a one-target local package that sets the trait and re-exports the module, and link the app against that instead of FluidAudio directly:

```swift
// FluidAudioShim/Package.swift
// swift-tools-version: 6.2
import PackageDescription

let package = Package(
name: "FluidAudioShim",
platforms: [.macOS(.v14), .iOS(.v17)],
products: [.library(name: "FluidAudioShim", targets: ["FluidAudioShim"])],
dependencies: [
.package(url: "https://github.com/FluidInference/FluidAudio.git", from: "0.15.7", traits: [])
],
targets: [
.target(name: "FluidAudioShim", dependencies: [.product(name: "FluidAudio", package: "FluidAudio")])
]
)
```

```swift
// FluidAudioShim/Sources/FluidAudioShim/Reexport.swift
@_exported import FluidAudio
```

Existing `import FluidAudio` lines keep compiling. Measured on a universal macOS app this way (Xcode 26.3): 16.85 MB off the executable, 12.7%, zero engine symbols, ASR/diarization symbols unchanged.

**The xcframework still downloads.** The binary target is declared unconditionally and only the dependency edge is trait-conditioned, so a clean resolve still fetches the 49 MB `NemoTextProcessing.xcframework.zip` even with the trait off. Ship size is unaffected; CI and cold checkouts pay the download. That is a SwiftPM limitation, not something the package can change.

To build the package itself without the engine: `swift build --disable-default-traits`.
2 changes: 2 additions & 0 deletions Package.swift
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,8 @@ let package = Package(
),
// Byte-exact NeMo text normalization (FST engine, all 7 languages).
// Prebuilt xcframework from FluidInference/text-processing-rs v0.3.0.
// Always linked on tools < 6.2; Package@swift-6.2.swift exposes it as
// the opt-out `NemoTextProcessing` trait (#880, #888).
.binaryTarget(
name: "NemoTextProcessing",
url:
Expand Down
95 changes: 95 additions & 0 deletions Package@swift-6.2.swift
Original file line number Diff line number Diff line change
@@ -0,0 +1,95 @@
// swift-tools-version: 6.2
import PackageDescription
import Foundation

// Tools 6.2+ manifest: identical to Package.swift plus the `NemoTextProcessing`
// trait. Keep the two in sync; Package.swift serves toolchains < 6.2, which
// always link the engine. (SwiftPM 6.1 accepts the trait syntax but still
// links a trait-conditioned binary target — verified on Xcode 16.4 — so the
// opt-out is gated at 6.2.)

let package = Package(
name: "FluidAudio",
platforms: [
.macOS(.v14),
.iOS(.v17),
],
products: [
.library(
name: "FluidAudio",
targets: ["FluidAudio"]
),
.executable(
name: "fluidaudiocli",
targets: ["FluidAudioCLI"]
),
],
traits: [
// Opt out of the NeMo text-normalization engine (~8 MB per slice, a prebuilt
// Rust staticlib) for ASR/VAD/diarization-only apps, or when the app
// links its own Rust runtime (#880, #888):
// .package(url: ..., traits: [])
// TTS frontends and `TextNormalizer` then pass text through unchanged
// and report `isNativeAvailable == false`.
.trait(
name: "NemoTextProcessing",
description: "Link the bundled NeMo text-normalization engine (TTS frontends, ITN)."
),
.default(enabledTraits: ["NemoTextProcessing"]),
],
dependencies: [],
targets: [
.target(
name: "FluidAudio",
dependencies: [
"FastClusterWrapper",
"MachTaskSelfWrapper",
.target(name: "NemoTextProcessing", condition: .when(traits: ["NemoTextProcessing"])),
],
path: "Sources/FluidAudio",
exclude: ["ASR/Parakeet/Unified/benchmark.md"],
resources: [
// Keep .process: .copy of a Resources-named directory breaks Apple code signing on iOS.
.process("TTS/LuxTts/G2p/Resources")
]
),
// Byte-exact NeMo text normalization (FST engine, all 7 languages).
// Prebuilt xcframework from FluidInference/text-processing-rs v0.3.0.
.binaryTarget(
name: "NemoTextProcessing",
url:
"https://github.com/FluidInference/text-processing-rs/releases/download/v0.3.0/NemoTextProcessing.xcframework.zip",
checksum: "76d0ee9a32b1ee2193231299180ca9bc4fc7e98794e771b3d55d66498352d85f"
),
.target(
name: "FastClusterWrapper",
path: "Sources/FastClusterWrapper",
publicHeadersPath: "include"
),
.target(
name: "MachTaskSelfWrapper",
path: "Sources/MachTaskSelfWrapper",
publicHeadersPath: "include"
),
.executableTarget(
name: "FluidAudioCLI",
dependencies: ["FluidAudio"],
path: "Sources/FluidAudioCLI",
exclude: ["README.md"],
resources: [
.process("Utils/english.json")
]
),
.testTarget(
name: "FluidAudioTests",
dependencies: [
"FluidAudio",
"FluidAudioCLI",
],
resources: [
.process("TTS/LuxTts/Resources")
]
),
],
cxxLanguageStandard: .cxx17
)
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ Want to convert your own model? Check [möbius](https://github.com/FluidInferenc
## Highlights

- **Automatic Speech Recognition (ASR)**: [Parakeet TDT v3](Documentation/Models.md#batch-transcription-near-real-time) (0.6b) and other TDT/CTC models for batch transcription supporting 25 European languages and Japanese, plus SenseVoice and Paraformer for Mandarin Chinese; [Parakeet EOU](Documentation/Models.md#streaming-transcription-true-real-time) (120m) for streaming ASR with end-of-utterance detection (English only). See all [ASR models](Documentation/Models.md#asr-models).
- **Inverse Text Normalization (ITN)**: Post-process ASR output to convert spoken-form to written-form ("two hundred" → "200"). See [text-processing-rs](https://github.com/FluidInference/text-processing-rs)
- **Inverse Text Normalization (ITN)**: Post-process ASR output to convert spoken-form to written-form ("two hundred" → "200"). See [text-processing-rs](https://github.com/FluidInference/text-processing-rs). Optional: ASR-only apps can drop the engine (~8 MB per slice) with `traits: []` (Swift 6.2+), see [PostProcessing.md](Documentation/ASR/PostProcessing.md#opting-out-of-the-engine)
- **Text-to-Speech (TTS)**: Kokoro (82m) for parallel synthesis with SSML and pronunciation control across 9 languages (EN, ES, FR, HI, IT, JA, PT, ZH); PocketTTS for streaming TTS with voice cloning support (EN, DE, ES, FR, IT, PT — 6L and 24L variants)
- **Speaker Diarization (Online + Offline)**: Speaker separation and identification across audio streams. Streaming pipeline for real-time processing and offline batch pipeline with advanced clustering.
- **Speaker Embedding Extraction**: Generate speaker embeddings for voice comparison and clustering, you can use this for speaker identification
Expand Down
Loading
Loading