Kokoro-ANE: 5 of hexgrad's 9 supported languages have voice data but no real G2P frontend (Spanish, French, Hindi, Italian, Brazilian Portuguese)

[Abusayid693]

What problem does this solve?

hexgrad/Kokoro-82M documents 9 supported language/locale groups in its own VOICES.md: American English, British English, Japanese, Mandarin Chinese, Spanish, French, Hindi, Italian, and Brazilian Portuguese.

KokoroAneVariant currently has real support for only 3: english, mandarin, japanese — each with its own dedicated text-to-phoneme frontend (English's built-in, NemoTextNormalizer+MandarinG2P for Mandarin, the MeCab+Cutlet port from #915 for Japanese).

The voice tensors for the other 5 language groups are already present, though — they're just folded into englishVoices, with this SDK's own doc comment admitting the real behavior:

non-English-prefixed packs (zf_*, jf_*, …) still speak English phonemes here, just with that voice's timbre.

Concretely, these voice codes exist in the SDK today but can only be used as English-phoneme playback with that voice's timbre, not real speech in their actual language:

Language	hexgrad's VOICES.md section	Voice codes already in englishVoices
Spanish	Spanish	ef_dora, em_alex, em_santa
French	French	ff_siwis
Hindi	Hindi	hf_alpha, hf_beta, hm_omega, hm_psi
Italian	Italian	if_sara, im_nicola
Brazilian Portuguese	Brazilian Portuguese	pf_dora, pm_alex, pm_santa

[paologrimaldi]

One data point that may shrink the scope here, at least for Spanish.

FluidAudio still ships a CoreML multilingual G2P model that covers Spanish: MultilingualG2PModel (Sources/FluidAudio/TTS/G2P/), backed by MultilingualG2PEncoder.mlmodelc / MultilingualG2PDecoder.mlmodelc at the root of FluidInference/kokoro-82m-coreml. MultilingualG2PLanguage has .spanish (spa) alongside French, Italian, Hindi and Brazilian Portuguese, and it returns IPA. The removed mono KokoroTtsManager used exactly this model for ef_* / em_* voices, so Spanish on Kokoro was working in this SDK up to v0.14.5 without espeak-ng.

Today the only caller left is the CLI G2P benchmark, but the model is still public. So a .spanish variant for KokoroAneVariant looks like wiring rather than a new frontend: phonemize per word with MultilingualG2PModel, join with punctuation, and feed the result into the existing synthesizeFromPhonemes path. The same should hold for French, Italian and Portuguese.

We tried exactly that today against v0.15.7 in a small macOS harness, and it works. A 28 s Spanish sample (ef_dora, ~30x realtime on Apple Silicon) transcribed back verbatim with Parakeet v3 (--language es). Things worth knowing if someone picks this up:

• The G2P output needs a small mapping onto ANE/vocab.json: ASCII g → ɡ (U+0261), tie-bar affricates → ʧ/ʤ, and combining marks stripped (KokoroAneVocab.encode iterates by grapheme cluster, so e + U+0303 never matches). Otherwise characters are dropped silently.
• The model marks stress inconsistently (most polysyllables come back unmarked). Spanish stress is fully predictable from spelling, so we recompute it. Placement matters: ˈ must sit immediately before the stressed vowel, after onset consonants and glides (kapˈitulo, tjˈempo), the same convention the English frontend emits. Putting it before the syllable onset measurably hurt intelligibility.
• Nothing in 0.15.7 downloads the two MultilingualG2P bundles anymore; ModelHub.download(.kokoro, subdirectory:to:) into <TtsCacheDirectory>/Models/kokoro/ puts them where the model looks.
• Known G2P quirks so far: word-initial ll gets a spurious leading i (llegó → iʎeɣo), río → zio.

Context: we ship Local Narrator (Spanish + English audiobooks on iOS) on the legacy Kokoro path and are pinned below 0.14.6 because of this gap. Happy to test a branch on real devices, and to contribute the stress rules and mapper if useful.

[Abusayid693]

@Alex-Wengg any idea on this ? can we get additional languages ?

[Alex-Wengg]

@Abusayid693 which specific one do you have in mind

[Abusayid693]

@Alex-Wengg Hindi (priority), French would be great if possible both or else Hindi as priority ? I tried other multilingual modals for these, but could not get the voice quality that kokoro delivers

[paologrimaldi]

I am focused on Spanish and improving Spanish as far as I can

[Alex-Wengg]

i will focus on french & spanish. the hindi version last time i had tried kept failing with the phoneme mapping

[Alex-Wengg]

@paologrimaldi @Abusayid693 aby reason why you guys don't use supertonic?

[Abusayid693]

@Alex-Wengg For my use case, Kokoro is a better fit because it natively outputs word-level timestamps, allowing for highly accurate word highlighting right out of the box. With Supertonic 3, I would have to implement a separate forced aligner since it only outputs the final audio file.

• Also did not find supertonic audio quality as good as of kokoro

[paologrimaldi]

@Alex-Wengg mostly timing, we built on Kokoro before Supertonic-3 came out.

The real dealbreaker for us though is the 70 char cap. We read whole books, so a normal paragraph gets split into 5-10 chunks and you can hear the seams, same thing as #736. Kokoro fits a full paragraph in one pass so the flow carries across sentences. Over a 10 hour audiobook that choppiness really adds up.

Smaller stuff: Supertonic's Spanish hasn't been benchmarked yet, and the voices are shared across all languages instead of being Spanish-trained.

I'll still run it through my Spanish harness and post results here, but unless long passages sound smooth, Kokoro + G2P is still the better path for us.