Repository navigation
Integer map keys: JSON integer syntax, i128/u128, and from a Value - #485
Open
yuxi-liu-wired wants to merge 3 commits into
Open
yuxi-liu-wired wants to merge 3 commits into
yuxi-liu-wired wants to merge 3 commits into
Conversation
yuxi-liu-wired
force-pushed
the
pr/integer-map-keys
branch
from
October 4, 2026 18:56
41a21a7 to
cb44bda
Compare
Licenser
reviewed
Oct 4, 2026
Licenser
left a comment
Member
There was a problem hiding this comment.
some little pieces of polish
Integer map keys (i8..u64, and i128/u128 with `128bit`) were parsed with
`str::parse`, which also accepts a leading `+` and leading zeros:
from_slice::<BTreeMap<u32, u8>>(r#"{"+1":2}"#) Ok({1: 2})
from_slice::<BTreeMap<i64, u8>>(r#"{"01":2}"#) Ok({1: 2})
Neither is JSON number syntax (RFC 8259 §6), and serde_json and
sonic-rs reject both as integer keys. `-0` is rejected as well: as a
number it is the key `0`, as a string it isn't, so accepting it would
let `"0"` and `"-0"` collide.
One function, `serde::parse_integer_key`, now does this for both the
text deserializer (`MapKey`) and the Value one (`MapKeyDeserializer`):
it checks the leading characters and leaves the rest to `str::parse`.
Test: `integer_keys_follow_json_number_syntax`, through `from_slice` and
the Value functions (fails before this change).
Found by a three-way differential fuzzer (serde_json, simd-json,
sonic-rs).
`i128` and `u128` values deserialize in the default build (through
`as_i128`/`as_u128`), but the map-key implementations were behind
`#[cfg(feature = "128bit")]`, in the text deserializer (`MapKey`) and in
the Value one (`MapKeyDeserializer`), so without the feature every
128-bit key fell back to serde's default and failed:
from_slice::<BTreeMap<i128, u8>>(r#"{"7":1}"#)
Err(Serde("i128 is not supported"))
Keys are parsed from the key text, so they need no tape support for
128-bit numbers; the cfg is dropped in both places. The 128-bit key
round trip in `serde::test::maps` (also behind the feature, so the
default run never exercised it) now runs in every build. serde_json and
sonic-rs accept these keys.
Test: `i128_map_keys`, through `from_slice` and the Value functions
(fails before this change).
Found by a three-way differential fuzzer (serde_json, simd-json,
sonic-rs).
Deserializing from an owned or borrowed Value by value
(`from_owned_value`, `from_borrowed_value`, `Value::deserialize_*`)
passed each object key as `Value::String`, so integer keys never parsed:
let v = to_owned_value(br#"{"1":2}"#)?;
from_owned_value::<BTreeMap<i64, u8>>(v)
Err(invalid type: string "1", expected i64)
The by-reference path (`from_refowned_value`) and the text path accept
it, and so do serde_json and sonic-rs. Keys now go through
`MapKeyDeserializer` in all four Value paths; its constructor takes
`impl Into<Cow<str>>`, so one `new` serves borrowed and owned keys.
Test: `value_map_integer_keys`; `integer_keys_follow_json_number_syntax`
and `i128_map_keys` now also check the by-value paths (all through the
same `map_key_results` helper: `from_slice` and the four Value
functions). Fails before this change.
Found by a differential fuzzer (serde_json, simd-json, sonic-rs, jiter).
yuxi-liu-wired
force-pushed
the
pr/integer-map-keys
branch
from
October 5, 2026 03:27
cb44bda to
2560d0b
Compare
Contributor
Author
|
Thanks for the review. Rebased on main (it conflicted with #484's test) and addressed all four points; each commit still covers one root cause:
The tests run through |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Related fixes in one area, 3 commits (each with its own test).
fix: integer map keys must be JSON integers
Integer map keys (i8..u64, and i128/u128 with
128bit) were parsed withstr::parse, which also accepts a leading+and leading zeros:Neither is JSON number syntax (RFC 8259 §6), and serde_json and
sonic-rs reject both as integer keys.
-0is rejected as well: as anumber it is the key
0, as a string it isn't, so accepting it would let"0"and"-0"collide.One function,
serde::parse_integer_key, does this for both the textdeserializer (
MapKey) and the Value one (MapKeyDeserializer): itchecks the leading characters and leaves the rest to
str::parse.Test:
integer_keys_follow_json_number_syntax, throughfrom_sliceandthe Value functions (fails before this change).
fix: i128/u128 map keys without the 128bit feature
i128andu128values deserialize in the default build (throughas_i128/as_u128), but the map-key implementations were behind#[cfg(feature = "128bit")], in bothMapKeyandMapKeyDeserializer,so without the feature every 128-bit key fell back to serde's default
and failed:
Keys are parsed from the key text, so they need no tape support for
128-bit numbers; the cfg is dropped in both places, and the 128-bit
round trip in
serde::test::maps(also behind the feature) now runs inevery build. serde_json and sonic-rs accept these keys.
Test:
i128_map_keys(fails before this change).fix: integer map keys when deserializing from a Value
Deserializing from an owned or borrowed Value by value
(
from_owned_value,from_borrowed_value,Value::deserialize_*)passed each object key as
Value::String, so integer keys never parsed:The by-reference path (
from_refowned_value) and the text path acceptit, and so do serde_json and sonic-rs. Keys now go through
MapKeyDeserializerin all four Value paths; its constructor takesimpl Into<Cow<str>>, so onenewserves borrowed and owned keys.Test:
value_map_integer_keys, through all four Value functions (failsbefore this change).
Testing
The key tests share a
map_key_resultshelper that runsfrom_sliceand the four Value functionson the same input. Each commit's test fails before the change. Locally: fmt and clippy on 1.88, and
cargo testwith default features, each of128bit,alloc,approx-number-parsing,arraybackend,beef,known-key,value-no-dup-keys,--no-default-features(with and withoutserde_impl) andtarget-cpu=native.Found with a differential fuzzer comparing serde_json, simd-json, sonic-rs and jiter.