Skip to content

Fix parsing of ".E0" input#43

Open
n-peugnet wants to merge 2 commits into
bzick:masterfrom
n-peugnet:issue-42-fuzz-input
Open

Fix parsing of ".E0" input#43
n-peugnet wants to merge 2 commits into
bzick:masterfrom
n-peugnet:issue-42-fuzz-input

Conversation

@n-peugnet

@n-peugnet n-peugnet commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

I'm not absolutely sure that this is the expected output for this specific input, but it seems to make sense. Also not sure that the fix is correct, but to me it does not make sense to allow an exponent just after a dot (at least Go does not allow it).

Fixes #42

@n-peugnet n-peugnet force-pushed the issue-42-fuzz-input branch from 25b2fa8 to fad1932 Compare July 7, 2026 11:31
@n-peugnet n-peugnet marked this pull request as ready for review July 8, 2026 08:10
@bzick

bzick commented Jul 12, 2026

Copy link
Copy Markdown
Owner

Thanks for the fix! The .E0 case is handled correctly, but the current change is a bit too broad: it also breaks valid float literals where an exponent follows the dot after a leading digit, e.g. 1.e5 or 10.E3. These are valid floats (Go accepts them too: 1.e5 == 100000, https://go.dev/play/p/qxj-fplRtm0), and today the tokenizer parses each as a single TokenFloat.

With this PR they get shattered into separate tokens:

input before (master) with this PR
1.e5 Float("1.e5") Int("1") Unknown(".") Keyword("e") Int("5")
10.E3 Float("10.E3") Int("10") Unknown(".") Keyword("E") Int("3")
.E0 Keyword("E") Int("0") Unknown(".") Keyword("E") Int("0") ✅ (the intended fix)

The root cause is that the new condition looks only at nextByte and no longer distinguishes "dot without a leading digit" (.E0, which we want to reject) from "dot after digits" (1.e5, which is a valid mantissa). The distinguishing signal is hasNumber.

I added regression cases for 1.e5/10.E3 to the floats subtest — they pass on master and fail with this PR's change, which is how I noticed the regression.

Proposed alternative — keep allowing the exponent right after the dot only when a digit already preceded it:

} else if !((nextByte == 'e' || nextByte == 'E') && hasNumber) && nextByte != 0 {
    break
}

With this, .E0 (no leading digit → hasNumber == false) is still rejected as intended, while 1.e5 / 10.E3 stay single floats. I verified it passes both the .E0 edge case and the 1.e5/10.E3 float cases.

@n-peugnet n-peugnet force-pushed the issue-42-fuzz-input branch from fad1932 to d236d62 Compare July 13, 2026 09:23
@n-peugnet

Copy link
Copy Markdown
Contributor Author

These are valid floats (Go accepts them too: 1.e5 == 100000)

Right I was not aware that these were valid float, it's good that you double checked. I just rebased and applied your suggestion instead.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

New issue detected by fuzzing: input: ".E0" expected: ".E0" actual: "E0"

2 participants