Skip to content

Integer map keys: JSON integer syntax, i128/u128, and from a Value - #485

Open
yuxi-liu-wired wants to merge 3 commits into
simd-lite:mainfrom
yuxi-liu-wired:pr/integer-map-keys
Open

yuxi-liu-wired wants to merge 3 commits into
simd-lite:mainfrom
yuxi-liu-wired:pr/integer-map-keys

Conversation

@yuxi-liu-wired

@yuxi-liu-wired yuxi-liu-wired commented Oct 1, 2026 •

Copy link
Copy Markdown
Contributor

Related fixes in one area, 3 commits (each with its own test).

fix: integer map keys must be JSON integers

Integer map keys (i8..u64, and i128/u128 with 128bit) were parsed with
str::parse, which also accepts a leading + and leading zeros:

from_slice::<BTreeMap<u32, u8>>(r#"{"+1":2}"#)   Ok({1: 2})
from_slice::<BTreeMap<i64, u8>>(r#"{"01":2}"#)   Ok({1: 2})

Neither is JSON number syntax (RFC 8259 §6), and serde_json and
sonic-rs reject both as integer keys. -0 is rejected as well: as a
number it is the key 0, as a string it isn't, so accepting it would let
"0" and "-0" collide.

One function, serde::parse_integer_key, does this for both the text
deserializer (MapKey) and the Value one (MapKeyDeserializer): it
checks the leading characters and leaves the rest to str::parse.

Test: integer_keys_follow_json_number_syntax, through from_slice and
the Value functions (fails before this change).

fix: i128/u128 map keys without the 128bit feature

i128 and u128 values deserialize in the default build (through
as_i128/as_u128), but the map-key implementations were behind
#[cfg(feature = "128bit")], in both MapKey and MapKeyDeserializer,
so without the feature every 128-bit key fell back to serde's default
and failed:

from_slice::<BTreeMap<i128, u8>>(r#"{"7":1}"#)
    Err(Serde("i128 is not supported"))

Keys are parsed from the key text, so they need no tape support for
128-bit numbers; the cfg is dropped in both places, and the 128-bit
round trip in serde::test::maps (also behind the feature) now runs in
every build. serde_json and sonic-rs accept these keys.

Test: i128_map_keys (fails before this change).

fix: integer map keys when deserializing from a Value

Deserializing from an owned or borrowed Value by value
(from_owned_value, from_borrowed_value, Value::deserialize_*)
passed each object key as Value::String, so integer keys never parsed:

let v = to_owned_value(br#"{"1":2}"#)?;
from_owned_value::<BTreeMap<i64, u8>>(v)
    Err(invalid type: string "1", expected i64)

The by-reference path (from_refowned_value) and the text path accept
it, and so do serde_json and sonic-rs. Keys now go through
MapKeyDeserializer in all four Value paths; its constructor takes
impl Into<Cow<str>>, so one new serves borrowed and owned keys.

Test: value_map_integer_keys, through all four Value functions (fails
before this change).

Testing

The key tests share a map_key_results helper that runs from_slice and the four Value functions
on the same input. Each commit's test fails before the change. Locally: fmt and clippy on 1.88, and
cargo test with default features, each of 128bit, alloc, approx-number-parsing,
arraybackend, beef, known-key, value-no-dup-keys, --no-default-features (with and without
serde_impl) and target-cpu=native.

Found with a differential fuzzer comparing serde_json, simd-json, sonic-rs and jiter.

@Licenser Licenser left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

some little pieces of polish

Comment thread src/serde/de.rs Outdated
Comment thread src/serde/de.rs
Comment thread src/tests/serde.rs
Comment thread src/serde/value/shared.rs Outdated
claude added 3 commits October 5, 2026 03:11
Integer map keys (i8..u64, and i128/u128 with `128bit`) were parsed with
`str::parse`, which also accepts a leading `+` and leading zeros:

    from_slice::<BTreeMap<u32, u8>>(r#"{"+1":2}"#)   Ok({1: 2})
    from_slice::<BTreeMap<i64, u8>>(r#"{"01":2}"#)   Ok({1: 2})

Neither is JSON number syntax (RFC 8259 §6), and serde_json and
sonic-rs reject both as integer keys. `-0` is rejected as well: as a
number it is the key `0`, as a string it isn't, so accepting it would
let `"0"` and `"-0"` collide.

One function, `serde::parse_integer_key`, now does this for both the
text deserializer (`MapKey`) and the Value one (`MapKeyDeserializer`):
it checks the leading characters and leaves the rest to `str::parse`.

Test: `integer_keys_follow_json_number_syntax`, through `from_slice` and
the Value functions (fails before this change).

Found by a three-way differential fuzzer (serde_json, simd-json,
sonic-rs).
`i128` and `u128` values deserialize in the default build (through
`as_i128`/`as_u128`), but the map-key implementations were behind
`#[cfg(feature = "128bit")]`, in the text deserializer (`MapKey`) and in
the Value one (`MapKeyDeserializer`), so without the feature every
128-bit key fell back to serde's default and failed:

    from_slice::<BTreeMap<i128, u8>>(r#"{"7":1}"#)
        Err(Serde("i128 is not supported"))

Keys are parsed from the key text, so they need no tape support for
128-bit numbers; the cfg is dropped in both places. The 128-bit key
round trip in `serde::test::maps` (also behind the feature, so the
default run never exercised it) now runs in every build. serde_json and
sonic-rs accept these keys.

Test: `i128_map_keys`, through `from_slice` and the Value functions
(fails before this change).

Found by a three-way differential fuzzer (serde_json, simd-json,
sonic-rs).
Deserializing from an owned or borrowed Value by value
(`from_owned_value`, `from_borrowed_value`, `Value::deserialize_*`)
passed each object key as `Value::String`, so integer keys never parsed:

    let v = to_owned_value(br#"{"1":2}"#)?;
    from_owned_value::<BTreeMap<i64, u8>>(v)
        Err(invalid type: string "1", expected i64)

The by-reference path (`from_refowned_value`) and the text path accept
it, and so do serde_json and sonic-rs. Keys now go through
`MapKeyDeserializer` in all four Value paths; its constructor takes
`impl Into<Cow<str>>`, so one `new` serves borrowed and owned keys.

Test: `value_map_integer_keys`; `integer_keys_follow_json_number_syntax`
and `i128_map_keys` now also check the by-value paths (all through the
same `map_key_results` helper: `from_slice` and the four Value
functions). Fails before this change.

Found by a differential fuzzer (serde_json, simd-json, sonic-rs, jiter).
@yuxi-liu-wired

Copy link
Copy Markdown
Contributor Author

Thanks for the review. Rebased on main (it conflicted with #484's test) and addressed all four points; each commit still covers one root cause:

  • integer keys must be JSON integers: one parse_integer_key for text and Value, -0 rejected
  • i128/u128 keys without 128bit: both deserializers, and the round-trip test is no longer behind the feature
  • integer keys from a by-value Value: MapKeyDeserializer::new(impl Into<Cow<str>>)

The tests run through from_slice and all four Value functions. Locally: fmt and clippy on 1.88, and cargo test with default features, each of 128bit, alloc, approx-number-parsing, arraybackend, beef, known-key, value-no-dup-keys, --no-default-features (with and without serde_impl) and target-cpu=native, all pass.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants