Skip to content

refactor(codegen): migrate arrays_finds.rs + array_methods.rs onto the Layer 1 rooting API (#7615) - #7620

Merged
proggeramlug merged 3 commits into
mainfrom
refactor/layer1-slice1b
Aug 8, 2026
Merged

refactor(codegen): migrate arrays_finds.rs + array_methods.rs onto the Layer 1 rooting API (#7615)#7620
proggeramlug merged 3 commits into
mainfrom
refactor/layer1-slice1b

Conversation

@proggeramlug

@proggeramlug proggeramlug commented Aug 8, 2026

Copy link
Copy Markdown
Contributor

Layer 1 campaign slice 1b (#7615): crates/perry-codegen/src/expr/arrays_finds.rs and crates/perry-codegen/src/expr/array_methods.rs migrated end to end onto crate::rooting, following #7617's template and #7618's precedent. 2 modules, 1562 → 1763 lines; the map counts 40 raw sites and 26 hazard sites. Both land whole; the ledger notes no outstanding boundary.

Adds one combinator, rooting::with_operands_rooted_across, arriving with its two callers and with the argument for why the existing one cannot serve (below). with_operands_rooted is now its empty-across case, so "root, re-derive, or reuse?" is still answered in exactly one place — the drift that produced #7114 was two helper families answering it separately.

What the migration found

1. Expr::BufferSlice held a RAW, already-unboxed pointer across user code

buf_box    = lower_expr(buffer)        // NaN-boxed BufferHeader
buf_handle = unbox_to_i64(buf_box)     // RAW pointer, in an SSA register
start_box  = lower_expr(start)         // arbitrary user code — allocates
end_box    = lower_expr(end)           // ditto
js_buffer_slice(buf_handle, …)         // reads the PRE-MOVE address

#7453's shape with a twist that matters: what is in flight is no longer NaN-boxed, so #7280's root_reload structurally cannot repair it — that post-pass re-reads a shadow slot into a double, and the consuming call reads an i64 derived above the window. The unbox is now emitted below the group's re-read, which is the only place it can be correct.

2. The arr.find* family, in the shape it is almost always written in

find / findIndex / findLast / findLastIndex all lowered the array first and left it in a register while the callback was lowered. A callback literal lowers to js_closure_new — an allocation — so xs.find(x => x.id === id) is a live window. Same for Object.is(a, b), Object.hasOwn(o, k), path.matchesGlob(p, pat), the Map/Set positional readers, AggregateError, Buffer.concat(list, total), Object.create(proto, props), both FinalizationRegistry mutators, both ErrorNew* forms, the three-operand NativeArenaView/NativePodView, the multi-argument new Date(y, m, d, …) (each component live across the next), and the polymorphic u8[k] = v store.

3. The index arms needed a re-read point this module controls

u8[i] and buf[i] lower the receiver with lower_expr and the index with lower_index_i32, which picks between an i32 fast path (lower_expr_as_i32) and a double plus fptosi. The receiver is live across that choice. Handing the index to with_operands_rooted would force every such read back onto the NaN-boxed path — a codegen-quality regression, not a rooting fix. with_operands_rooted's re-read point is fixed at the end of its operand list, so an operand lowered before caller-controlled work would be re-read above it and stale again by the time the call runs, which is the #7114 half-measure. with_operands_rooted_across roots the group before the caller-controlled lowering and re-reads it after; across_exprs is passed as expressions rather than a bool so the "does this window collect?" question stays inside operand_protection. Two callers, both here.

The new combinator deliberately does not reopen what rooting.rs's existing note warns about: it never takes a bare register, it still owns the release (now including a bail from the operand lowering itself), and it never hands out an unrooted result.

Honest scoping — what is genuinely new vs what #7280 already covered

Following #7618's precedent, this is stated rather than glossed. For a shadow-slotted local receiver, root_reload had already repaired the find window, and the baseline IR shows it as an out-of-sequence re-read:

%r595  = load double, ptr %r9                       ; the receiver
%r596  = call i64 @js_closure_alloc_singleton(…)    ; ALLOCATES
%r1681 = load double, ptr %r9                       ; ← #7280 already fired
%r600  = and i64 %r599, 281474976710655
%r602  = call double @js_array_find(i64 %r600, i64 %r601)

-16 load double, ptr in the net delta is exactly that population: #7280 reloads replaced by root reads. What this PR actually buys is the three shapes #7280 structurally cannot cover:

  1. A receiver reassigned by its own argument. GC: #7154's residual is NOT fixed — the loop-polls config is red 0/30, and stock zod alone fails 5/40 #7280 bails there by design (operand_is_reloadable's documented miscompile: re-loading would observe the assignment). Baseline, verbatim — note the absence of any re-read:

    %r1046 = load double, ptr @perry_global_p3_finds_ts__1   ; `live`
    %r1047 = call double @…__swap()                          ; allocates AND reassigns `live`
    %r1048 = bitcast double %r1046 to i64                    ; STALE
    %r1051 = call double @js_array_find(i64 %r1049, i64 %r1050)

    After:

    %r1062 = load double, ptr @perry_global_p3_finds_ts__1
    %r1063 = bitcast double %r1062 to i64
    store i64 %r1063, ptr %r14                    ; root store, ABOVE the window
    call void @js_shadow_slot_bind(i32 5, ptr %r14)
    %r1064 = call double @…__swap()
    %r1065 = load i64, ptr %r14                   ; re-read, BELOW it
    %r1069 = call double @js_array_find(i64 %r1067, i64 %r1068)
    store i64 0, ptr %r14                         ; release
  2. A raw, already-unboxed pointerBufferSlice above. A double reload cannot repair an i64 derived above the window.

  3. Operand-to-operand windows and receivers with no slot at all — call results, property chains, the four-operand finreg.register(t, h, tok).

IR-identity evidence

Two corpora, both arms, compiled PERRY_RS4GC=0 PERRY_GC_MOVING_LOOP_POLLS=1 PERRY_INLINE_SHADOW_SLOT=0 PERRY_NO_AUTO_OPTIMIZE=1 --trace llvm, PERRY_RUNTIME_DIR pinned. Register names and block-label numbers are masked; a function is "identical" iff its instruction sequence matches up to that renaming.

A. 8 purpose-built probes reaching 28 of the 30 distinctive callees of the arms this PR changed (the two gaps are named below).

  • 172 functions compared; 164 identical, 8 differ — and all 8 are main, the only function in each probe containing these lowerings.

  • Whole-corpus net instruction delta, every line of it:

    +36 / +36 store i64 %r · js_shadow_slot_bind — the root stores
    +36 / +36 load i64, ptr · bitcast i64→double — the re-reads
    +37 / +36 store i64 0, ptr · js_shadow_slot_set — the releases (the odd one is a grown frame's prologue zero-init)
    +12 bitcast double→i64 — the store side of a push
    +1 alloca i64 — one more pooled temp slot
    −16 load double, ptrGC: #7154's residual is NOT fixed — the loop-polls config is red 0/30, and stock zod alone fails 5/40 #7280's reloads, replaced by root reads

    Non-root-plumbing added: 0. Non-root-plumbing removed: 0. This PR deletes no instruction kind and adds no callee.

B. The whole gc-root-dominance corpus (129 sources → 149 modules) compared the same way, which also re-checks the two already-migrated modules against the with_operands_rooted refactor:

  • 2452 functions; 2443 identical, 9 differ. 6 of the 9 are root plumbing only (+16 stores/binds/reads/releases, +2 allocas, −9 stale load double).
  • The other 3 have an identical instruction multiset and a different order, and are compiler NONDETERMINISM, not this change — proven by compiling one source twice with the same binary: @.str.N constants and js_register_function_name calls come out in a different order run to run, and in test_gap_class_expr_dynamic_parent_ctor three perry_method_* callees permute. Filed as Codegen is not run-to-run deterministic: string-constant numbering, function-name registration order, and method-callee selection permute #7622; it is a pre-existing build-reproducibility gap, and worth knowing about before anyone else tries a byte-level A/B here.

Behavioural A/B: all 8 probes, both arms — identical stdout and exit code, and identical again under PERRY_GC_ZEAL=1 PERRY_GC_PROTECT_FROMSPACE=1 PERRY_GC_PROTECT_FROMSPACE_DEPTH=64.

Coverage the probes do NOT reach, stated rather than glossed:

  • js_native_pod_view (lower_native_pod_view_with_layout) — NativeArena.podView needs a PerryPodView<T> annotation over a PerryPod<…> layout, which has no TS spelling reachable from a standalone probe; it is exercised only by crates/perry-codegen/tests/native_proof_* (nightly/tag only). Its migration is the same three-operand with_operands_rooted as NativeArenaView, which IS probed twice.
  • js_buffer_index_get_value (BufferIndexGet's tail) — the structural twin of Uint8ArrayGet's tail, which is probed 6 times through the same new with_operands_rooted_across call. Five index shapes were tried and all routed to lower_buffer_load or the dynamic helper.

Ledger sabotage — result recorded, per module

escape_hatch_uses strips comments, and both files named temp_root zero times before this PR, so listing them passes vacuously. The arm that makes the listing mean something was run for each module separately (the assert stops at the first offender, so one run cannot speak for both):

module injected compiles? caught by
expr/arrays_finds.rs temp_root_push_double + temp_root_truncate yes ledger test — red, naming arrays_finds.rs:279 and :280
expr/array_methods.rs temp_root_push_i64 + temp_root_truncate yes ledger test — red, naming array_methods.rs:59 and :60

Same answer #7617 and #7618 measured, now for a third and fourth module: the API does not make the bug fail to compile; the ledger is what denies the escape hatch. Both reverted. That vacuity, and the obligation to run this arm, is now written into MIGRATED_MODULES' doc comment so the next slice cannot skip it by accident.

Verification (local — the CI backlog is deep, so this is the evidence, and it is stated as such)

  • gc-root-dominance, both gated modes, post-change compiler: corpus 129/129 sources → 149 modules, 2452 functions, 9826 root stores → 0 violations; --seeded-violations 4040 planted, 40 caught, 0 missed; --unrooted-allocas --moving-only0. Baseline arm on the same corpus: 9810 root stores, 7861 gc-capable allocas, also 0/0. The +16 / +2 delta is how you can tell the gate's subject was live rather than absent.
  • All four checker static audits: --self-test, --audit-alloc-re (71 alternatives), --audit-poll-capable (54 entries), --audit-immovable-sources (4 probes, 0 failing).
  • Stale-register budget, curated corpus, both arms: 23 ≤ 39, unchanged.
  • cargo test -p perry-codegen --lib691 pass, 0 fail, including all four ledger tests. --doc — both compile_fail,E0499 arms still reject.
  • cargo test -p perry-runtime --no-fail-fast1886 pass, 0 fail, clean on the first run (no rerun needed; cargo-test: perry-runtime's suite fails a different number of tests on every run #7365's known flake did not appear).
  • Gap suite, both arms, 17 family filters over the lowerings these modules own (array, buffer, path, map, set, date, object, typed, process, iterator, weakref, error, fs, regexp, math, uint8): 60 tests, 60 pass, 0 parity fail, 0 compile fail, 0 crashed, 0 skipped — identical failure sets, because the set is empty on both. (test_gap_finaliz* matches no file; the harness exits 1 on an empty selection, which is selection, not a result.)
  • The full lint script set, enumerated from .github/workflows/test.yml rather than remembered — 27 invocations. All pass except one, which is red on main too (see below).

Not run locally: the dependency-scale (zod) dominance corpus, which needs npm ci.

Pre-existing failures found, flagged rather than smuggled in

1. benchmarks/ci_public_baseline_check.py is red on main.

public baseline error: public artifact benchmark harness changed; regenerate it with ./benchmarks/run_public_baseline.sh

This PR's diff is three files under crates/perry-codegen/src/; it cannot cause this. #7618 recorded the same step failing on every main run from 2026-07-29 onward. Because lint is a sequence and a failing step skips the rest, this is CLAUDE.md hazard 4 in the costume the workflow's own comment names — except the later steps now carry if: ${{ !cancelled() }}, so they do still speak.

2. path.resolve(base, f(), …) throws ERR_INVALID_ARG_TYPE where node returns the path — an SSO bug, not a rooting bug.

function seg(n: number): string { return "s" + String(n); }
path.resolve("/root", seg(1));   // node: /root/s1     perry: TypeError
path.resolve("/root", "s1");     // both: /root/s1

Expr::PathResolveJoin unboxes its operands with unbox_to_i64, so a short computed string's inline SSO bytes are read as a StringHeader*. Bisected by string length: a segment longer than the SSO threshold works, a short one does not. The correct helper is unbox_str_handle — but that one allocates (SSO materialisation), so applying it opens the #7213 window in the same arm, which is a rooting change with its own combinator question. Filed as #7621. Deliberately not fixed here: this PR is IR-identical-or-justified by construction, and a semantic fix to five path arms is not something to hide inside a refactor. Confirmed present on both arms.

3. Codegen output is not run-to-run deterministic (#7622). See the IR-identity section: string-constant numbering, js_register_function_name order, and perry_method_* callee selection permute between two compiles of the same source with the same binary. This does not affect correctness of any arm here, but it does mean a byte-level IR A/B needs the multiset/normalised comparison this PR used, not diff.

Not in this PR

No version bump. No behaviour change beyond the itemised rooting fixes — no instruction kind is deleted. No gate widening, no allowlist entry, no new test file (and therefore no check_test_registration.py registration to make). The SSO bug above, the benchmark-baseline gate, and the nondeterminism are reported, not repaired.

Advances #7615. Slice 2 (expr/instance_misc1.rs, expr/logical_collections.rs, lower_call/property_get/map_set.rs) is next.

Summary by CodeRabbit

  • Bug Fixes

    • Improved runtime safety when evaluating multi-step array, buffer, collection, path, date, object, and typed-array operations.
    • Preserved correct dynamic property behavior and undefined results for out-of-range typed-array and buffer access.
    • Improved array type detection and handling of array callbacks.
  • Documentation

    • Added migration notes and documented known limitations.
  • Chores

    • Updated the application version to 0.5.1355.

…e Layer 1 rooting API (#7615)

Layer 1 campaign slice 1b: `expr/arrays_finds.rs` and `expr/array_methods.rs`
migrated end to end onto `crate::rooting` and added to `MIGRATED_MODULES`.

Adds one combinator, `rooting::with_operands_rooted_across`, with its two
callers: the `u8[i]` / `buf[i]` tails root the receiver across an index lowering
whose representation the caller picks (`lower_expr_as_i32` vs `fptosi`), which
the fixed re-read point of `with_operands_rooted` cannot express.
`with_operands_rooted` is now the empty-`across` case of it, so "root, re-derive
or reuse?" stays answered in one place.
@coderabbitai

coderabbitai Bot commented Aug 8, 2026

Copy link
Copy Markdown

Review Change Stack

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 66eff6ca-9708-40d4-96d5-1f211309a99f

📥 Commits

Reviewing files that changed from the base of the PR and between bb23d2a and acf2117.

⛔ Files ignored due to path filters (1)
  • Cargo.lock is excluded by !**/*.lock
📒 Files selected for processing (2)
  • CLAUDE.md
  • Cargo.toml

📝 Walkthrough

Walkthrough

The PR adds with_operands_rooted_across and migrates array, buffer, collection, path, object, date, view, and indexed-store lowering to root operands across later expression evaluation.

Changes

GC-safe operand rooting migration

Layer / File(s) Summary
Cross-expression rooting API
crates/perry-codegen/src/rooting.rs, changelog.d/7620-layer1-slice1b-arrays-finds-rooting.md, Cargo.toml, CLAUDE.md
Adds incremental rooting, operand re-reading, cleanup on success and error paths, migration-ledger updates, changelog documentation, and version metadata updates.
Array method and constructor lowering
crates/perry-codegen/src/expr/array_methods.rs
Roots operands across aggregate errors, buffer operations, object creation, finalization registry calls, and error construction.
Array, collection, and path lowering
crates/perry-codegen/src/expr/arrays_finds.rs
Roots operands for array finds, date construction, map/set access, path operations, object checks, and callback validation.
Typed-array, buffer, and native view lowering
crates/perry-codegen/src/expr/arrays_finds.rs
Roots receivers and operands across indexed reads, writes, symbol allocation, index lowering, and native view construction.

Estimated code review effort: 4 (Complex) | ~45 minutes

Sequence Diagram(s)

sequenceDiagram
  participant ExpressionLowering
  participant with_operands_rooted_across
  participant OperandLowering
  participant RuntimeCall
  ExpressionLowering->>with_operands_rooted_across: lower and root operands
  with_operands_rooted_across->>OperandLowering: evaluate later operands
  OperandLowering-->>with_operands_rooted_across: return lowered values
  with_operands_rooted_across->>RuntimeCall: pass re-read operands
Loading

Possibly related PRs

Suggested labels: bug

Suggested reviewers: jdalton

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title clearly and concisely identifies the migration of both target modules to the Layer 1 rooting API.
Description check ✅ Passed The description thoroughly covers the migration, rationale, verification, related issue, scope, and known limitations, despite not copying every template heading.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch refactor/layer1-slice1b

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@changelog.d/7620-layer1-slice1b-arrays-finds-rooting.md`:
- Line 1: Update the changelog heading to use PR key `#7620`, matching the
7620-prefixed fragment filename and ensuring the release note references the
correct PR.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 0466fa89-1745-4ad2-bdac-1f03e03e6add

📥 Commits

Reviewing files that changed from the base of the PR and between 4f281c6 and bb23d2a.

📒 Files selected for processing (4)
  • changelog.d/7620-layer1-slice1b-arrays-finds-rooting.md
  • crates/perry-codegen/src/expr/array_methods.rs
  • crates/perry-codegen/src/expr/arrays_finds.rs
  • crates/perry-codegen/src/rooting.rs

@@ -0,0 +1,60 @@
### Layer 1 rooting migration, slice 1b — `expr/arrays_finds.rs` + `expr/array_methods.rs` (#7615)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win

Use the current PR key in the heading.

Line 1 says #7615, but the fragment filename is keyed to 7620. Change the heading to #7620 so the release note identifies the same PR as its required fragment.

As per coding guidelines, add a PR-keyed changelog.d/<PR>-<slug>.md fragment. Based on learnings, verify the PR key in each changelog fragment.

Proposed fix
-### Layer 1 rooting migration, slice 1b — `expr/arrays_finds.rs` + `expr/array_methods.rs` (`#7615`)
+### Layer 1 rooting migration, slice 1b — `expr/arrays_finds.rs` + `expr/array_methods.rs` (`#7620`)
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
### Layer 1 rooting migration, slice 1b — `expr/arrays_finds.rs` + `expr/array_methods.rs` (#7615)
### Layer 1 rooting migration, slice 1b — `expr/arrays_finds.rs` + `expr/array_methods.rs` (`#7620`)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@changelog.d/7620-layer1-slice1b-arrays-finds-rooting.md` at line 1, Update
the changelog heading to use PR key `#7620`, matching the 7620-prefixed fragment
filename and ensuring the release note references the correct PR.

Sources: Coding guidelines, Learnings

@proggeramlug
proggeramlug merged commit 38ff7ec into main Aug 8, 2026
@proggeramlug
proggeramlug deleted the refactor/layer1-slice1b branch August 8, 2026 04:16
@proggeramlug

Copy link
Copy Markdown
Contributor Author

Audit before merge — verified, merged as v0.5.1355

Ledger sabotage red on a real temp_root_push_i64 injected into
arrays_finds.rs; the per-module discipline holds for both new entries.

The find* fix verified behaviourally under a live protector: all four
find/findIndex/findLast/findLastIndex with allocating callbacks over a
5,000-element pointer array — identical to node plain AND under
PERRY_GC_ZEAL=1 PERRY_GC_PROTECT_FROMSPACE=1 with 5 retired quarantine
sets
(matching the report's own liveness figure). That was the arm holding
the array across js_closure_new; it no longer does.

Root-dominance re-run: 129/129, 0 violations, 40/40 seeded caught. Full
suites (runtime 1,886/0 first try, codegen 690/0), the full lint script set
including check_test_registration and gc_store_site_inventory, fmt — all
clean.

Three things this slice adds to the campaign's method, worth naming:

  1. The raw-i64 taxonomy point. GC: #7154's residual is NOT fixed — the loop-polls config is red 0/30, and stock zod alone fails 5/40 #7280's root_reload re-reads a double
    from a shadow slot — it structurally cannot repair a pointer already
    unboxed to i64
    above the window. BufferSlice was exactly that, and the
    distinction now anchors which sites each mechanism owns.
  2. Filing path.resolve/extname/normalize/matchesGlob read an SSO string's inline bytes as a StringHeader pointer #7621 instead of smuggling it. The path.resolve SSO bug is a
    real user-facing throw, found in-slice, and NOT a rooting bug — fixing it
    here would have broken the slice's IR-identity contract. Kept out, filed
    with the bisect evidence.
  3. Codegen is not run-to-run deterministic: string-constant numbering, function-name registration order, and method-callee selection permute #7622 (codegen run-to-run nondeterminism) contaminating 3 of 9 corpus
    diffs
    — proven by compiling one source twice with the same binary before
    attributing anything to the change. Every future slice's IR-identity claim
    needs that control; it is now on the record as required method.

The combinator addition (with_operands_rooted_across, with the plain form as
its empty case) keeps operand_protection the single decision site — the right
call versus a second parallel path.

Fallout handled separately: the harness-fingerprint redness this slice's
full-lint run surfaced traces to #7601's functional edit of
json_pipeline.ts; the baseline is regenerating now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant