fix(sys): spread Map and Set keys across hash buckets (backport quickjs-ng #1815)

[mwatts]

Issue # (if available)

Backport of quickjs-ng/quickjs#1815 by @pixelpax, which is open upstream and not in a quickjs-ng release yet. The C change is theirs, unchanged except for the baseline it applies to.

Description of changes

map_hash_key hashes an object or symbol key as its address times 3163, and a number as (lo ^ hi) * 3163 over the bits of its double. The bucket is the low bits of that hash (h & (hash_size - 1)). Two consequences on the pinned baseline (quickjs-ng fd0a021):

• Heap pointers are 16-byte aligned, so the low four bits of an object or symbol hash are constant. Object-keyed Maps, Sets, WeakMaps and WeakSets use at most 1 bucket in 16, so chains are about 16 times longer than the load factor intends.
• An integer below 2048 has a zero low word and zero low bits in its high word, so every such key lands in one bucket.

This PR adds integration patch 0033-map-hash-mix.patch. It runs both hashes through the lowbias32 finalizer, a bijection on uint32_t, before the tag is mixed in. That is the same function and the same two call sites as #1815. Equality is still decided by js_same_value_zero, and iteration still follows the record list, so only bucket placement changes. String and BigInt keys are untouched.

The patch should be dropped when the baseline moves to a quickjs-ng release that contains #1815. sys/patches/README.md says so.

How it was found. gpui-qs runs Quickshell QML on gpui-shell (through gpui-kit's rquickjs). Its reactive runtime keys Maps, Sets and WeakMaps by objects on every property read and change. In a native profile of a 600-item panel updating at 5 Hz, map_find_record was 18% of busy main-thread time.

Measured effect. Lookup cost in ns per lookup, interpreter, rquickjs-cli release build of this branch against main (96593cd). Best of 5 rounds inside each run, then the three runs shown. aarch64 (Apple M-series under Asahi Linux), load average about 6. The script is below.

Keys	n	main (ns)	this PR (ns)	Speed vs main
objects { i }	1,000	143, 145, 145	88, 88, 88	1.62x
objects { i }	20,000	215, 248, 318	90, 90, 93	2.39x
Symbol(i)	1,000	115, 115, 118	88, 90, 90	1.31x
Symbol(i)	20,000	125, 130, 143	90, 90, 93	1.39x
Set of objects, has	20,000	125, 135, 148	95, 98, 98	1.32x
WeakMap of objects	20,000	130, 138, 150	90, 90, 93	1.44x
integers 0..n	1,000	700, 703, 708	95, 95, 95	7.37x
integers 0..n	20,000	573, 598, 605	95, 98, 98	6.03x
i + 0.5	1,000	853, 858, 860	93, 93, 93	9.17x
strings "key" + i	20,000	93, 93, 95	93, 93, 95	1.00x
2**53 + 2i	20,000	73, 73, 75	95, 95, 95	0.77x

2**53 + 2i gets slower. Above 2^53, consecutive doubles differ only in their low word, and the old multiplication happened to send such a run to every bucket in turn. After the patch those keys cost what every other key family costs. #1815 reports the same family and the same direction.

In gpui-qs, the same idea (measured with a MurmurHash3 finalizer rather than lowbias32) lowered whole-process CPU on that panel by about 5% and its render time by about 12%.

This is an interpreter fix, not a JIT optimization, so the three-engine README matrix in AGENTS.md does not apply.

Where the remaining time goes. After the patch a lookup is still 10 to 20 times slower than in Bun, so I split the cost into layers. One script, best of 7 rounds of 2,000,000 operations, performance.now(), same machine. Every engine was run three times, interleaved; the rows below varied by under 1%. quickjs-jit is this repository at 96593cd with jit-abi, as gpui-shell builds it, run through a file runner over Context::full; the stock engines are their own qjs built with CMake Release.

ns per operation	Bellard bcb9ad1	quickjs-ng 9b06171 (master)	quickjs-ng fd0a021 (this baseline)	quickjs-jit 96593cd	this PR	Bun 1.4.3	Node 26.11
empty loop iteration	13.9	17.3	22.9	27.0	24.8	3.5	10.2
o.x	13.9	20.1	24.6	30.9	28.5	2.3	8.7
closure call f(i)	23.6	28.9	34.7	45.8	38.5	3.3	9.9
m.get(intKey)	37.5	686.3	689.1	695.8	66.2	4.2	10.4
m.get(objKey)	43.1	65.7	96.7	104.8	69.6	4.7	14.0
m.get(stringKey)	49.9	62.4	70.8	78.3	76.6	6.5	15.2
set.has(objKey)	47.4	68.0	97.5	108.8	72.6	5.3	13.0

(The "this PR" column is the default build without jit-abi; the jit-abi build of 96593cd adds 2 to 7 ns per row over the default build.)

What this says:

• With the patch, a Map lookup costs about what one loop iteration, one array read and one native call cost in the interpreter. The rest of the gap to Bun and Node is interpretation, which only a JIT closes; it is not in the hash table any more.
• Bellard's current QuickJS hashes every key family through map_hash64/map_hash32 into the high bits, so it already does not have this defect. quickjs-ng master still has it (hence #1815), which is why its integer row is still near 690 ns.
• Independent of this PR: quickjs-ng master runs the base rows 1.2 to 1.3 times as fast as the fd0a021 baseline, and this repository's build of that baseline is a further 1.1 to 1.3 times slower than stock qjs (the jit-abi probes, assertions that NDEBUG would remove, and build flags each contribute; I did not separate them fully). For host-heavy embedders like gpui-shell, where the JIT does not engage (benchmarks/results/gpui-shell-jit-v1.md reports 0.99x to 1.00x on the realistic 443-node panel), moving the baseline forward looks like a larger lever than any single patch. That is outside this PR.

Tests. core/src/lib.rs gains map_keys::every_key_family_is_found_after_mixing. It covers integer, double, -0, 1.0, NaN, large doubles, BigInt, object, symbol and Symbol.for keys in Map, Set and WeakMap, deletes, and insertion order. It is a semantics guard: the speedup itself cannot be observed without timing, as #1815 also notes. sys/tests/jit_patch.rs lists the new patch, and build_support/patch.rs pins its digest and the patched quickjs.c fingerprint.

Run locally (aarch64 Linux):

• cargo fmt --all -- --check
• cargo clippy -p quickjs-jit-core -p quickjs-jit-sys --all-targets (no warnings)
• cargo test -p quickjs-jit-sys --test jit_patch (13 passed)
• cargo test -p quickjs-jit-core --lib (151 passed)
• cargo test -p quickjs-jit-runtime --features compiler,test-support --test abi --test manifest_versions --test semantics --test helpers --test gpui_shell_surface -- --test-threads=1 (all passed)
• cargo test -p quickjs-jit-runtime --release --features compiler,test-support --test opcodes --test differential --test regressions -- --test-threads=1 (all passed)

Not run locally: the sanitizer, test262, Windows and wasm jobs.

Benchmark script (map-keys.js)

(function () {
  const out = [];
  const families = {
    "objects": (n) => Array.from({ length: n }, (_, i) => ({ i })),
    "symbols": (n) => Array.from({ length: n }, (_, i) => Symbol(i)),
    "ints 0..n": (n) => Array.from({ length: n }, (_, i) => i),
    "i + 0.5": (n) => Array.from({ length: n }, (_, i) => i + 0.5),
    "2**53 + 2i": (n) => Array.from({ length: n }, (_, i) => 2 ** 53 + 2 * i),
    "strings": (n) => Array.from({ length: n }, (_, i) => "key" + i),
  };
  const time = (keys, coll, get) => {
    let best = Infinity;
    const reps = Math.max(1, Math.floor(400000 / keys.length));
    for (let round = 0; round < 5; round++) {
      const t0 = Date.now();
      let hit = 0;
      for (let r = 0; r < reps; r++) for (let i = 0; i < keys.length; i++) if (get(coll, keys[i]) !== undefined) hit++;
      const dt = Date.now() - t0;
      if (hit !== reps * keys.length) throw new Error("missed a key");
      best = Math.min(best, (dt * 1e6) / (reps * keys.length));
    }
    return best.toFixed(0);
  };
  for (const n of [1000, 20000]) {
    for (const [name, make] of Object.entries(families)) {
      const keys = make(n);
      const m = new Map(keys.map((k, i) => [k, i]));
      out.push(`Map ${name} ${n} ${time(keys, m, (c, k) => c.get(k))}`);
    }
    const objs = families.objects(n);
    const s = new Set(objs);
    out.push(`Set objects ${n} ${time(objs, s, (c, k) => (c.has(k) ? 1 : undefined))}`);
    const w = new WeakMap(objs.map((k, i) => [k, i]));
    out.push(`WeakMap objects ${n} ${time(objs, w, (c, k) => c.get(k))}`);
  }
  console.log(out.join("\n"));
})();

Run by piping it as one line into target/release/rquickjs-cli.

Checklist

• Added change to the changelog
• Created unit tests for my feature if needed

Credit for the fix and its analysis goes to @pixelpax in quickjs-ng/quickjs#1815. Their commits are credited with a Co-authored-by trailer.

Prepared with AI assistance: Claude Code wrote the patch backport, the test and this description, and ran the measurements above on my machine.

🤖 Generated with Claude Code

[Copilot]

🟢 Approval recommended

The backport matches the upstream change, preserves collection semantics, and includes appropriate integrity and regression coverage.

0 open findings

What changed in this PR

Backports quickjs-ng #1815 to improve Map and Set hash-bucket distribution.

Changes:

• Adds lowbias32 mixing for numeric, object, and symbol keys.
• Adds semantic regression coverage for affected key families.
• Registers, documents, and fingerprints the integration patch.

File	Description
sys/​patches/​0033-map-hash-mix.patch	Implements hash mixing.
sys/​build_support/​patch.rs	Registers patch and fingerprints.
sys/​tests/​jit_patch.rs	Includes patch in integration tests.
core/​src/​lib.rs	Adds collection semantics tests.
sys/​patches/​README.md	Documents the backport.
CHANGELOG.md	Records the fix.

🧠 Review effort: Balanced

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.