Polish audio TN by jimregan · Pull Request #470 · NVIDIA/NeMo-text-processing

jimregan · 2026-08-07T23:04:18Z

What does this PR do ?

Audio TN for Polish. Also TN, but that has too many non-determinisms to care about

Before your PR is "Ready for review"

Pre checks:

PR Type:

New Feature
Bugfix
Documentation
Test

If you haven't finished some of the above items you can still open "Draft" PR.

Signed-off-by: Jim O'Regan <joregan@kth.se>

for more information, see https://pre-commit.ci

Signed-off-by: Jim O'Regan <joregan@kth.se>

for more information, see https://pre-commit.ci

Signed-off-by: Jim O'Regan <joregan@kth.se>

…into polish-tn

for more information, see https://pre-commit.ci

Signed-off-by: Jim O'Regan <joregan@kth.se>

…into polish-tn Signed-off-by: Jim O'Regan <joregan@kth.se>

Signed-off-by: Jim O'Regan <joregan@kth.se>

for more information, see https://pre-commit.ci

Signed-off-by: Jim O'Regan <joregan@kth.se>

…into polish-tn

for more information, see https://pre-commit.ci

Signed-off-by: Jim O’Regan <joregan@kth.se>

github-actions · 2026-08-31T01:49:09Z

This PR is stale because it has been open for 14 days with no activity. Remove stale label or comment or update or this will be closed in 7 days.

Note that Google Translate's TTS can manage 0,00000000000000001 ("zero i jedna stubiliardowa") but higher than that it becomes stupid: 0,000000000000000001 comes out as "zero jeden przez zero...(18 times)" ("zero one divided by zero zero etc.") Signed-off-by: Jim O'Regan <joregan@kth.se>

toleniecki · 2026-09-14T15:52:29Z

+pl_acc	ny
+pl_ins	nami
+pl_loc	nach
+pl_voc	ny


there is something about new line different that in other files, i am not sure if that is important.

That's likely just an artifact of his editor. Not a problem for the parser.

toleniecki · 2026-09-14T15:57:56Z

+pl_nom	le
+pl_gen	l
+pl_dat	lom
+pl_acc	li


that does not generalize by ending, i think there is some other word feature other than ending and genus in play. e.g.
sg_nom "fala" (wave) pl_acc is "fale", not "fali" as this file suggests.

but it works for names:
sg_nom "Ala" produces pl_acc "Ali".

It's only used with names at the moment (specifically, names of queens or saints to disambiguate a trailing roman numeral or a leading 'św.' respectively)

toleniecki · 2026-09-14T16:15:02Z

+sg_nom	<eps>
+sg_gen	a
+sg_dat	owi
+sg_acc	<eps>


here again the rule is more complex than ending alone, but this time i happen to know it has something to do with the object being inanimate vs animate (person, animal, plant). e.g.:
"kolec" (spike) sg_acc is "kolec", while animate thing like "golec" in sg acc becomes "golca".

if "mi" in the file name stands for "male inanimate", the it dismisses the comment, but then there is no "mp_c" file

This is intended for [^e]c, I would have had noun_mi_ec.tsv for the case you're describing, and sg_nom would be 'ec' because the e is deleted. Same with -iec, same with -zec for marzec.

It would probably be easier to follow if I were to name everything in terms of a specific noun but I don't honestly imagine that anybody else will be working on this.

toleniecki · 2026-09-14T16:38:03Z

@@ -0,0 +1,14 @@
+sg_nom	m
+sg_gen	ma


not sure if "mi" is "male inanimate", but assuming that -- this is wrong, e.g.:
sg nom "szlam" (sludge) -> sg gen "szlamu"
sg nom "szlem" (slem) -> sg gen "szlemu"
sg nom "dżem" (jam) -> sg gen "dżemu"

so i think it should be "mu"

gram -> grama

toleniecki · 2026-09-14T16:42:19Z

@@ -0,0 +1,14 @@
+sg_nom	r
+sg_gen	ra


"ru" if it's for inanimate, otherwise "ra" is fine.
e.g.
"katar" (a cold, inanimate) -> "kataru"
"Tatar" (Tatar nation, animate) -> "Tatara"

In the types of words that tend to come up in normalisation, it tends to be -a for inanimate (cetnar -> cetnara)

toleniecki · 2026-09-14T19:50:45Z

@@ -0,0 +1,24 @@
+1	dziesiąty


"dziesiętny", if it is about denominators.

Maybe the file is poorly named, but it's decimal denominators purely, so for decimal I,D if len(D) == 1, the denominator is "tenth", len(D) == 2, "hundredth", etc.

toleniecki · 2026-09-14T19:54:55Z

the title says "denominators", but then list all the words suggesting that they are male. we typically refer to fractions as females, so "0,1" reads "zero i jedna dziesiąta" (zero and one tenth, where "thenth" is female, thus "-a", not "-y").
i am not sure what the purpose of this file is, but if its to produce TTS of "0,001" and so one, that should all end with -a instead of -y

but then it becomes plural if its anything other than "1", so e.g. "0,2" is "0 i 2 dziesiętne", "0,3" is "0 i 3 dziesiętne" etc. + above 5 it becomes "0 i 5 dziesiętnych".

note that i am not bothering you here with how to read and write numbers on purpose, because that thing alone is a huge decision tree.

Adjectives are relatively regular in terms of inflection, this is reusing the same mechanism that's used for ordinals.

bpawulski · 2026-09-15T07:58:20Z

+o 4.25~o czwartej dwadzieścia pięć
+o godz. 21:14~o godzinie dwudziestej pierwszej czternaście
+4:25~czwarta dwadzieścia pięć
+04:25:07~czwarta dwadzieścia pięć siedem


This sounds too much machine style. The human would say:
"czwarta dwadzieścia pięć i siedem sekund"

bpawulski · 2026-09-15T07:59:36Z

+o godz. 21:14~o godzinie dwudziestej pierwszej czternaście
+4:25~czwarta dwadzieścia pięć
+04:25:07~czwarta dwadzieścia pięć siedem
+00:00:00~zero zero zero


similar here, human would say: "północ" -> midnight

bpawulski · 2026-09-15T08:02:27Z

+zero przecinek zero zero zero zero zero zero zero zero zero zero zero zero zero zero zero zero zero zero zero zero zero zero zero dwa
+~1,5
+jeden i pięć dziesiątych
+jeden przecinek pięć


"jeden i pół" is perfectly fine here and to me that would be the most obvious one

Yes, reading the decimal as a fraction would be more natural, but that's a specific set of exceptions to normal decimal handling. This is a draft, I'll get to it when I have time.

Maybe having a simple example here is misleading? 1,5 looks quite clear cut, but 1,56734 less so.

bpawulski · 2026-09-15T08:07:20Z

+~1,5 kg
+jeden i pięć dziesiątych kilograma
+jeden przecinek pięć kilograma
+półtora kilograma


"jeden i pół kilograma" is perfectly fine here as well

Signed-off-by: Jim O'Regan <joregan@kth.se>

…absent from the test) Signed-off-by: Jim O'Regan <joregan@kth.se>

toleniecki · 2026-09-15T12:41:22Z

@@ -0,0 +1,14 @@
+sg_nom	t
+sg_gen	ta


if its for "male inanimate", it should be "-tu", as in:
"blat"(couneter) -> sg gen "blatu"

for animate things its fine, e.g. "kot" (cat) -> sg gen "kota"

wat -> wata
Again, the point here is not to provide general inflection tables, but to target the kind of words that appear in normalisation, which overwhelmingly tend to have genitives in -a, not -u

Signed-off-by: Jim O'Regan <joregan@kth.se>

jimregan · 2026-09-15T18:47:29Z

+mm	milimetr	mi	noun_mi_r.tsv
+l	litr	mi	noun_mi_r.tsv
+ml	mililitr	mi	noun_mi_r.tsv
 g	gram	mi	noun_mi_m.tsv


@toleniecki does this clear things up?

Signed-off-by: Jim O'Regan <joregan@kth.se>

for more information, see https://pre-commit.ci

jimregan and others added 2 commits August 8, 2026 00:02

Polish audio text normalisation

0f1be7e

Signed-off-by: Jim O'Regan <joregan@kth.se>

[pre-commit.ci] auto fixes from pre-commit.com hooks

4114f96

for more information, see https://pre-commit.ci

jimregan marked this pull request as draft August 7, 2026 23:09

jimregan and others added 22 commits August 8, 2026 08:52

make measure suitable for audio TN, needs more work

fadb93d

Signed-off-by: Jim O'Regan <joregan@kth.se>

decimals

75572f2

Signed-off-by: Jim O'Regan <joregan@kth.se>

[pre-commit.ci] auto fixes from pre-commit.com hooks

7412020

for more information, see https://pre-commit.ci

convert more names

26eec32

Signed-off-by: Jim O'Regan <joregan@kth.se>

convert more names

e915ac5

Signed-off-by: Jim O'Regan <joregan@kth.se>

convert more names

838a960

Signed-off-by: Jim O'Regan <joregan@kth.se>

św.

f5b7c87

Signed-off-by: Jim O'Regan <joregan@kth.se>

Merge branch 'polish-tn' of github.com:jimregan/NeMo-text-processing …

99776b5

…into polish-tn

[pre-commit.ci] auto fixes from pre-commit.com hooks

cda0665

for more information, see https://pre-commit.ci

extended time tests

8787290

Signed-off-by: Jim O'Regan <joregan@kth.se>

time

9b8dec5

Signed-off-by: Jim O'Regan <joregan@kth.se>

rework

c53f241

Signed-off-by: Jim O'Regan <joregan@kth.se>

Merge branch 'polish-tn' of github.com:jimregan/NeMo-text-processing …

a1badb6

…into polish-tn Signed-off-by: Jim O'Regan <joregan@kth.se>

Move ordinal equivalences to data

c539566

Signed-off-by: Jim O'Regan <joregan@kth.se>

number nouns (2-ka -> dwójka)

1dd0e73

Signed-off-by: Jim O'Regan <joregan@kth.se>

Add Polish standalone decimal normalization

6ac4a59

Signed-off-by: Jim O'Regan <joregan@kth.se>

[pre-commit.ci] auto fixes from pre-commit.com hooks

1966f0f

for more information, see https://pre-commit.ci

Add Polish fraction normalization

cd40129

Signed-off-by: Jim O'Regan <joregan@kth.se>

Reuse decimal and fraction graphs in Polish measures

1bf592a

Signed-off-by: Jim O'Regan <joregan@kth.se>

Merge branch 'polish-tn' of github.com:jimregan/NeMo-text-processing …

21dfb8f

…into polish-tn

[pre-commit.ci] auto fixes from pre-commit.com hooks

b6b6f3f

for more information, see https://pre-commit.ci

Add new denominators for large numbers in Polish

76b25dc

Signed-off-by: Jim O’Regan <joregan@kth.se>

github-actions Bot added the Stale label Aug 31, 2026

github-actions Bot removed the Stale label Sep 2, 2026

tbartley94 self-assigned this Sep 11, 2026

toleniecki reviewed Sep 14, 2026

View reviewed changes

bpawulski reviewed Sep 15, 2026

View reviewed changes

Comment thread tests/nemo_text_processing/pl/data_text_normalization/test_cases_normalize_with_audio.txt

bpawulski reviewed Sep 15, 2026

View reviewed changes

Comment thread tests/nemo_text_processing/pl/data_text_normalization/test_cases_normalize_with_audio.txt Outdated

bpawulski reviewed Sep 15, 2026

View reviewed changes

Comment thread tests/nemo_text_processing/pl/data_text_normalization/test_cases_normalize_with_audio.txt Outdated

bpawulski reviewed Sep 15, 2026

View reviewed changes

Comment thread tests/nemo_text_processing/pl/data_text_normalization/test_cases_normalize_with_audio.txt Outdated

bpawulski reviewed Sep 15, 2026

View reviewed changes

Comment thread tests/nemo_text_processing/pl/data_text_normalization/test_cases_normalize_with_audio.txt Outdated

bpawulski reviewed Sep 15, 2026

View reviewed changes

Comment thread tests/nemo_text_processing/pl/data_text_normalization/test_cases_normalize_with_audio.txt Outdated

jimregan added 2 commits September 15, 2026 11:47

make tests less confusing

5f22618

Signed-off-by: Jim O'Regan <joregan@kth.se>

push preferred form to top of test (this was already generated, just …

fec3532

…absent from the test) Signed-off-by: Jim O'Regan <joregan@kth.se>

toleniecki reviewed Sep 15, 2026

View reviewed changes

Comment thread nemo_text_processing/text_normalization/pl/data/grammar/noun_mi_tr.tsv Outdated

toleniecki reviewed Sep 15, 2026

View reviewed changes

Comment thread nemo_text_processing/text_normalization/pl/data/grammar/noun_mi_tr.tsv Outdated

jimregan added 2 commits September 15, 2026 20:34

remove, overly specific

7326516

Signed-off-by: Jim O'Regan <joregan@kth.se>

replace

daeccab

Signed-off-by: Jim O'Regan <joregan@kth.se>

jimregan commented Sep 15, 2026

View reviewed changes

jimregan and others added 2 commits September 16, 2026 21:05

read small decimals as fractions

b6d3b3b

Signed-off-by: Jim O'Regan <joregan@kth.se>

[pre-commit.ci] auto fixes from pre-commit.com hooks

bb4674c

for more information, see https://pre-commit.ci

Uh oh!

Conversation

jimregan commented Aug 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What does this PR do ?

Before your PR is "Ready for review"

Uh oh!

github-actions Bot commented Aug 31, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

toleniecki Sep 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

toleniecki Sep 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

toleniecki Sep 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

toleniecki Sep 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

jimregan commented Aug 7, 2026 •

edited

Loading

toleniecki Sep 14, 2026 •

edited

Loading

toleniecki Sep 14, 2026 •

edited

Loading

toleniecki Sep 14, 2026 •

edited

Loading

toleniecki Sep 15, 2026 •

edited

Loading